Intermediate · 2 days
RAG: From Fundamentals to Production
A two-day course on retrieval-augmented generation as an engineering discipline: why naive RAG fails, and how to design a system that can be evaluated, secured and operated.
2 days · Intermediate · RAG & Agents · Developers, ML engineers and architects
Overview
Many RAG systems fail for ordinary reasons: poor parsing, arbitrary chunking, weak retrieval, no reranking, no evaluation, and a prompt that asks the model to sound certain. The result is fluent answers with weak grounding, which is worse than a search box.
This course teaches RAG properly. You start with architecture, embeddings and vector databases, then spend serious time on ingestion, parsing, chunking, metadata and retrieval strategy. Hybrid search, reranking, query rewriting, context construction, citations and access control are treated as the difference between a demo and a system.
Labs compare a weak pipeline with an improved one on the same corpus. The second day moves into production architecture: evaluation, hallucination reduction, permissions, monitoring, performance and cost.
You build a production-style RAG system you can explain, measure and harden — not a notebook that only works on a handful of happy questions.
Audience
Developers, ML engineers and architects who will design, implement or review retrieval-augmented systems.
Prerequisites
Professional software development experience and familiarity with APIs. Some exposure to LLMs is expected. Vector-database experience is helpful but not required.
Duration
2 days
Can be focused on a customer corpus and stack, or combined with the Agentic RAG course for teams building more complex systems.
Learning outcomes
- 01
Design a RAG architecture with explicit stages for ingestion, retrieval, generation and evaluation.
- 02
Implement parsing, chunking and metadata strategies that match the document types in play.
- 03
Compare similarity search, hybrid search, reranking and query rewriting on the same questions.
- 04
Construct prompts that use retrieved evidence, citations and refusal when evidence is missing.
- 05
Evaluate retrieval and grounded generation separately, and reduce hallucination without hiding the problem.
- 06
Apply access control so retrieval cannot leak documents the user is not allowed to see.
- 07
Specify production monitoring, performance and cost controls for a RAG service.
Outline
- 01
RAG as an architecture
- The job of retrieval versus the job of generation.
- Embeddings, vector databases and what similarity does not capture.
- Where naive RAG fails: chunking, recall, ranking, context stuffing and over-confident generation.
- Lab: run a minimal RAG path and collect its failure modes.
- 02
Ingestion that can be defended
- Document parsing: PDFs, HTML, tables, slides and the lossiness of extraction.
- Chunking: size, overlap, structure, late chunking and not treating all documents as prose.
- Metadata, document identity, versions and access-control attributes.
- Lab: ingest the same corpus two ways and compare retrieval quality.
- 03
Retrieval that can be measured
- Similarity search versus hybrid search; filters and metadata.
- Query rewriting, expansion and decomposing multi-part questions.
- Reranking, diversity and selecting what actually enters the context window.
- Context construction, citations, grounding and refusal.
- Lab: improve the weak system and show the difference on a fixed question set.
- 04
Production RAG
- Evaluation: retrieval metrics, groundedness, human review and regression sets.
- Hallucination reduction as a system property, not a prompt slogan.
- Access control, tenancy and permissions-aware retrieval.
- Monitoring, latency, caching, re-indexing and cost.
- Reference production architecture and operational checklist.
- Lab: add evaluation, citations and a permission constraint to the system.
Practical work
You build a RAG system on a realistic corpus. The first pipeline is deliberately naive. Subsequent labs improve parsing, chunking, retrieval, reranking, citations and evaluation. The closing system is production-shaped: measurable, permission-aware and operable, not merely fluent.
Takeaways
- A working production-style RAG implementation
- Weak-versus-improved retrieval comparisons on a fixed question set
- Chunking, metadata and evaluation templates
- A production architecture and operations checklist
Delivery
Cognitecta delivers private corporate training, on-site or as remote live training. Courses can be run as published, or adapted to your organisation’s stack, domain and experience level.
Instructor
Nicholas Johnson, AI architect and software engineer. He has a degree in Artificial Intelligence and around twenty years of professional technology training, including hundreds of courses for engineering teams and large organisations. About.
Related courses
2 days · Advanced
Building AI Agents
A two-day course for intermediate and advanced developers: design and build useful agents with tools, state, permissions and evaluation, rather than unsupervised demos.
2 days · Advanced
Agentic RAG Systems
A two-day advanced course combining retrieval with agentic reasoning and tool use: multi-step retrieval, query planning, structured sources and a production architecture for complex knowledge work.