Training / Models & Machine Learning
Advanced · 2–3 days
Training Your Own Language Model
A two-to-three-day course for advanced developers and ML engineers: build and train a small language model in order to understand the process, not to compete with frontier systems.
2–3 days · Advanced · Models & Machine Learning · Advanced developers and ML engineers
Overview
Training a language model looks opaque from the outside. This course demystifies it by having you construct a dataset, tokenise, implement a small transformer, and run a training loop you can inspect. The objective is understanding. Nobody leaves with a model that replaces a production API, and that is stated at the start.
The course covers dataset construction, tokenisation, embeddings, architecture, the training loop, loss, optimisation, batching, validation, checkpoints and inference. GPU constraints, scaling and evaluation are discussed as the reasons large training runs look the way they do.
You train a small working model. You watch loss move, inspect generations, and connect those observations to the production models you already call.
The value is a team that can talk to ML engineers, judge training proposals, and stop treating “we should train our own model” as either magic or a strategy.
Audience
Advanced developers and ML engineers who need a grounded understanding of language-model training.
Prerequisites
Strong Python, comfort with the transformer course or equivalent, and access to a GPU environment. This is not a first course in AI.
Duration
2–3 days
Two days is sufficient for a small model and a complete loop. Three days allows more work on data, evaluation and scaling discussion.
Learning outcomes
- 01
Construct a small training dataset and a tokeniser appropriate to the experiment.
- 02
Implement or assemble a small transformer and explain each stage of the forward pass.
- 03
Run a training loop with batching, loss, optimisation, validation and checkpoints.
- 04
Generate from a checkpoint and evaluate it honestly against the training objective.
- 05
Relate GPU memory, batch size and context length to what is feasible in the course and in a real cluster.
- 06
Judge when training from scratch is the wrong proposal compared with adapting or calling an existing model.
Outline
- 01
Data and representation
- Dataset construction, licensing, quality and the limits of a teaching corpus.
- Tokenisation: what the model actually trains on.
- Embeddings and sequence preparation.
- Lab: build the dataset and tokeniser for the experiment.
- 02
Model and training loop
- A small transformer: blocks, context, parameters.
- Loss, optimisation, batching and gradient steps.
- Validation, checkpoints and the difference between a falling loss and a useful model.
- Lab: train, watch the curves, and intervene when the run is unhealthy.
- 03
Inference, evaluation and scale
- Inference from checkpoints; sampling versus the training objective.
- Evaluation that matches the claim being made for the model.
- GPU constraints, scaling laws at an intuitive level, and why frontier training is a different industry.
- Discussion: when an organisation should train, adapt, or simply call a model.
- Lab: generate, evaluate, and write an honest account of what the model can do.
Practical work
You train a small working language model. You own the dataset, the loop, the checkpoints and the evaluation. The closing artefact is the model plus a short report that states what was learned and what would not transfer to a production training programme.
Takeaways
- A small trained model and checkpoints
- A training-loop implementation and run notes
- An evaluation report with an honest scope
- A framework for judging training proposals in an organisation
Delivery
Cognitecta delivers private corporate training, on-site or as remote live training. Courses can be run as published, or adapted to your organisation’s stack, domain and experience level.
Instructor
Nicholas Johnson, AI architect and software engineer. He has a degree in Artificial Intelligence and around twenty years of professional technology training, including hundreds of courses for engineering teams and large organisations. About.
Related courses
2 days · Advanced
Transformers from First Principles
A two-day course for technically strong practitioners: build a usable understanding of transformer models by implementing simplified components, without unnecessary mathematical formalism.
2 days · Advanced
Fine-Tuning Language Models
A two-day course for ML engineers and experienced AI developers: when to adapt a model, how to do it with parameter-efficient methods, and how to evaluate and deploy the result.
1 day · Intermediate
Running LLMs Locally
A one-day practical course on local and private inference: model formats, hardware, quantisation, engines, benchmarking and the privacy and cost reasons to run models yourself.