Intermediate · 3 days
AI Engineering for Software Developers
A three-day programme for professional software engineers: how to build real applications with modern AI, from model APIs and RAG through agents, evaluation and production patterns.
3 days · Intermediate · AI Engineering · Software developers
Overview
Most software engineers are now expected to put large language models into applications. The gap is not access to an API. It is the engineering around it: structured outputs, retrieval, tools, evaluation, security, testing and the production behaviour of a non-deterministic component.
This course teaches those fundamentals before any particular framework. You call models, constrain outputs, build embeddings and retrieval, then add tool calling, agents and orchestration. Context engineering, evaluation and observability are treated as part of the design, not as later extras.
The labs are substantial. You write working software, compare approaches, and learn to recognise when a framework is helping and when it is hiding a design you will have to own. The course is not bound to a single vendor SDK or agent library.
After three days, you can design and implement AI-backed features with the same seriousness you would apply to any other production integration: interfaces, failure modes, tests, and an honest account of what the model is allowed to do.
Audience
Professional software developers with little or moderate AI experience. Suitable for senior developers and technical leads who will implement, review or own LLM-backed features.
Prerequisites
Professional software development experience and comfort writing backend or full-stack code. Familiarity with HTTP APIs and JSON is expected. Python or TypeScript is used in labs; the concepts are not language-specific. No machine-learning background is required.
Duration
3 days
Can be condensed to two days for experienced teams, or extended with a fourth day on your organisation’s stack.
Learning outcomes
- 01
Implement LLM API calls with streaming, retries, structured outputs and explicit model selection.
- 02
Design prompts and output schemas that can be tested rather than tuned by anecdote.
- 03
Build embeddings, vector search and a retrieval pipeline that can be inspected and evaluated.
- 04
Implement tool calling and a bounded agent loop, including failure handling and permissions.
- 05
Apply context engineering: what to put in context, what to retrieve, and what to keep out.
- 06
Evaluate quality with golden cases, semantic checks and traces, not only manual chat.
- 07
Diagnose security, cost, latency and observability issues in an LLM-backed service.
- 08
Deploy a production pattern: a service boundary, logging, and tests around a non-deterministic component.
Outline
- 01
Model APIs as an engineering interface
- Provider APIs, messages, system prompts, streaming and cancellation.
- Model selection: capability, context, cost, latency and lock-in.
- Structured outputs, JSON schemas and the difference between a request and a guarantee.
- Retries, timeouts and what “the model failed” should mean in application code.
- Lab: call, stream and constrain a model from application code.
- 02
Prompting, context and evaluation from the start
- Prompt design as an interface: instructions, examples, constraints and untrusted input.
- Context windows, truncation and the cost of stuffing.
- Golden cases, assertions and why a chat transcript is not a test suite.
- Lab: write a prompt with tests, then break it with realistic inputs.
- 03
Embeddings, retrieval and RAG
- Embeddings, similarity and the limits of vector search.
- Chunking, metadata, retrieval and assembling a grounded prompt.
- Citations, refusal and what to do when retrieval is empty or conflicting.
- Lab: build a small RAG path and measure a naive version against a better one.
- 04
Tools, agents and orchestration
- Function calling, tool schemas and validating arguments before side effects.
- Agent loops versus explicit workflows; state, memory and stopping conditions.
- Orchestration without committing the design to a single framework.
- Lab: add tools to an application and keep the agent inside a permission boundary.
- 05
Production patterns
- Observability: traces, prompt versions, token counts, retrieval records and user-visible errors.
- Security: prompt injection, data leakage, tool permissions and output handling.
- Testing, evaluation pipelines and regression when prompts or models change.
- Cost, caching, fallbacks and a deployable service boundary.
- Lab: take one of the week’s systems through tracing, tests and a production checklist.
Practical work
The course is lab-heavy. You build against model APIs, add structured outputs and tests, implement retrieval, then introduce tools and a bounded agent. The final work is a small service with evaluation, tracing and explicit failure behaviour rather than a chat demo.
Takeaways
- Working code for API integration, RAG, tool calling and evaluation
- Reference implementations that are not tied to a single framework
- Prompt, schema and evaluation templates
- A production checklist for LLM-backed features
Delivery
Cognitecta delivers private corporate training, on-site or as remote live training. Courses can be run as published, or adapted to your organisation’s stack, domain and experience level.
Instructor
Nicholas Johnson, AI architect and software engineer. He has a degree in Artificial Intelligence and around twenty years of professional technology training, including hundreds of courses for engineering teams and large organisations. About.
Related courses
2–3 days · Intermediate
Building Applications with Generative AI
A two-to-three-day build course for software engineers: take an AI-powered application from architecture through RAG, tools, evaluation and deployment, ending with an end-to-end system.
1 day · Intermediate
Evaluating and Testing LLM Applications
A one-day course on evaluating non-deterministic systems: golden datasets, semantic checks, LLM-as-judge, retrieval and agent evaluation, and the pipeline that keeps quality from drifting.
1 day · Intermediate
Securing Generative AI Applications
A one-day security course for developers, security engineers and architects: the specific risks created by LLM applications, and the controls that actually reduce them.