Advanced · 2 days
AI Platform Engineering
A two-day course for platform engineers, architects and senior developers: build the internal infrastructure that lets multiple product teams use AI safely and consistently.
2 days · Advanced · Cloud & Platform · Platform engineers, architects and senior developers
Overview
If every product team integrates a model, a vector store and a logging story independently, the organisation gets duplicated cost, inconsistent security and no way to evaluate quality. Platform engineering for AI is the work of making a paved path: model access, policy, retrieval, observability and developer experience.
This course covers model gateways, routing, authentication, policy, rate limiting and model abstraction. Prompt management, evaluation, secrets, data controls, RAG services, vector infrastructure, cost allocation, caching, resilience and developer experience are designed as platform products.
The teaching is design-heavy, with enough implementation detail to be concrete. You work through the contracts a platform should offer a product team, and the ones it should refuse to hide.
You leave with a service design, not a catalogue of tools to install.
Audience
Platform engineers, architects and senior developers responsible for shared AI infrastructure and developer enablement.
Prerequisites
Experience building internal platforms, APIs or shared infrastructure. Familiarity with LLM applications is expected. This is not a first course in generative AI.
Duration
2 days
Often delivered around your organisation’s existing platform, identity and observability stack.
Learning outcomes
- 01
Design a model gateway with authentication, routing, policy and rate limiting.
- 02
Specify model abstraction and prompt management without hiding evaluation from product teams.
- 03
Provide RAG and vector infrastructure as a service with tenancy and data controls.
- 04
Implement observability, cost allocation and caching as platform features.
- 05
Define secrets, resilience and fallback behaviour that product teams can rely on.
- 06
Shape developer experience: SDKs, golden paths, documentation and a review process.
- 07
Draw a responsibility boundary between platform and product that can be operated.
Outline
- 01
The AI control plane
- What product teams should not each build: keys, routing, policy, basic tracing.
- Model gateways: auth, routing, quotas, fallbacks and provider abstraction.
- Prompt management and versioning as a platform concern.
- Exercise: define the gateway API a product team would call.
- 02
Data, retrieval and policy
- RAG services, vector infrastructure and tenancy.
- Data controls, secrets and the path untrusted content is allowed to take.
- Evaluation services: running suites, storing results, comparing models.
- Exercise: design a retrieval service contract with ACLs.
- 03
Operations and developer experience
- Observability, cost allocation, caching and resilience.
- SDKs, templates, golden paths and self-service versus tickets.
- When the platform should refuse a request, and how that is explained.
- Exercise: a platform roadmap, SLOs and a responsibility matrix.
Practical work
Design exercises throughout. You specify gateway, retrieval, evaluation and cost-allocation services, critique a weak platform proposal, and produce a paved-path design including developer experience. Implementation snippets are used where they clarify an interface; the deliverable is a platform design.
Takeaways
- A model-gateway and paved-path design
- Service contracts for retrieval, evaluation and cost allocation
- A platform versus product responsibility matrix
- Developer-experience and SLO notes
Delivery
Cognitecta delivers private corporate training, on-site or as remote live training. Courses can be run as published, or adapted to your organisation’s stack, domain and experience level.
Instructor
Nicholas Johnson, AI architect and software engineer. He has a degree in Artificial Intelligence and around twenty years of professional technology training, including hundreds of courses for engineering teams and large organisations. About.
Related courses
2 days · Intermediate
AI Systems on AWS
A two-day course for cloud engineers, developers and architects: design and build AI systems on AWS, with architecture first and managed services in their proper place.
2 days · Intermediate
AI Systems on Azure
A two-day course for cloud engineers, developers and architects: design enterprise AI systems on Azure, including identity, search, model APIs and the architecture those services should sit inside.