Language Models Without Extra Words

The book is a concise yet comprehensive guide to building language models. It discusses the mathematical foundations of machine learning, vector representations, and the workings of neural networks. The principles of language modeling are outlined: from bags of words and embeddings to recurrent networks and transformer architecture. Complete implementations of the described models and methods are provided in Python using the PyTorch framework, including examples in the form of Jupyter notebooks. Methods for training, fine-tuning, and optimizing large language models are described: LoRA, prompt engineering, quality assessment, as well as issues of copyright and ethics. The concluding chapter offers directions for further study: mixture of experts, model compression, preference-based alignment, and multimodal models.