Exercises + milestone requirements owned by this stage
Evidence and gate
0 · 0/9
Artifacts · checked requirements
Why it matters
Purpose of this stage
Deploy, observe, version, operate
Subjects
6 subjects · 46 canonical topics
Each subject states what you should be able to do, an order to work through it in, what to watch for, and the questions a reviewer should be able to ask you at the end. None of it is tracked or scored.
How to work through this subject
An order that works, not a checklist. Nothing here is tracked — the steps sequence this subject’s own topics and add none.
Pin your dependencies and note where the training set and the serving set genuinely differ.
Decide whether the model artifact ships inside the image or is loaded at runtime, and write down why.
Build the image and look at its size, then improve the layer order so rebuilds reuse the cache.
Move every environment-specific value into configuration read at startup.
Confirm no secret is baked into the image, including in its build history.
Finish by rebuilding from a clean checkout and confirming you get an equivalent artifact.
What to watch for
'Works on my machine' is a packaging failure, not a personality flaw.
Secrets are configuration that must never be in the image or the repository.
Training and serving have different dependency needs. Pinning them together installs a training stack in production for no reason.
A model baked into the image ships atomically but rebuilds for every retrain. Loading at runtime reverses both properties.
Build caching is determined by layer order. Put what changes least at the top.
Reproducible means someone else, on another machine, gets the same behaviour — not that it worked twice for you.
Curriculum topics
reproducible ML environments
Environments where the same code, data and dependency versions give the same model twice.
pinned dependencies for training vs serving
Training and serving often need different libraries; both are pinned so neither drifts silently.
model artifacts in the image vs loaded at runtime
Baking the model into the image makes deploys immutable; loading it at run time lets you change models without rebuilding.
image size and build caching
Ordering build steps so unchanged layers are reused, and keeping images small enough to deploy quickly.
configuration
Environment-specific values supplied from outside the image, so one artifact runs in every environment.
secrets
Credentials and keys, supplied at run time and never committed to the repository or baked into an image.
Resources for this subject
PRIMARYMade With ML — MLOps Made With ML The teaching path for production ML: packaging, lifecycle, serving, CI/CD and monitoring as one connected system. Apply each section to the model you already trained rather than to its example project.Open
Need another explanation?2 more references
REFERENCEDocker — Get started Docker Containers from zero, plus FastAPI's own deployment guide for the exact service you are packaging.Open Open
REFERENCEThe Twelve-Factor App 12factor.net Where the rule that configuration comes from the environment originates. Short, and it settles most config-versus-secret arguments. Its logs, disposability and dev/prod-parity factors are the deployment half of this stage.Open
A reviewer should be able to answer these
Answer from memory, out loud, before moving on. These are not tracked and do not affect the gate — they tell you whether you actually understand this subject.
What does a container guarantee about reproducibility, and what does it still not guarantee?
What is the difference between configuration and a secret, and where does each live?
How do you pin dependencies so a rebuild six months from now produces the same behaviour?
What would you check to prove the deployed artifact matches what you tested?
When would you load the model at runtime rather than build it into the image?
Why do training and serving environments often need different pinned dependencies?
What determines whether a rebuild reuses the cache or starts over?
Exercises
Demonstrated practice · 35% base weight
This stage defines implementation through its owned project milestone rather than a separate exercise list.
Resource status is a convenience marker only. It cannot change competencies, gates, or stage completion — in any tier below.
Start here
PRIMARY
Made With ML — MLOps
Made With ML
The teaching path for production ML: packaging, lifecycle, serving, CI/CD and monitoring as one connected system. Apply each section to the model you already trained rather than to its example project.
REFERENCEProduction ML Systems GoogleShort reference chapters on production ML systems, including serving, pipelines and monitoring. Read it for the failure modes it names, then check which of them your own deployment would actually catch.Open
Need another explanation?7 supplementary resources · not part of the canonical library
Use these for a demonstrated gap, not as a parallel course. They are authored additions and can never be the primary resource for a subject.
REFERENCEDocker — Get started DockerContainers from zero, plus FastAPI's own deployment guide for the exact service you are packaging.Bound to Stage 4, where containers are first taught, and reused in Stage 12 for ML packaging.Open Open
REFERENCEThe Twelve-Factor App 12factor.netWhere the rule that configuration comes from the environment originates. Short, and it settles most config-versus-secret arguments. Its logs, disposability and dev/prod-parity factors are the deployment half of this stage.Open
REFERENCEMLflow documentation MLflowConcrete implementations of experiment tracking and model registry, which the stage covers conceptually.Open
REFERENCEGitHub Actions documentation GitHubThe CI system this roadmap uses. Stage 4 needs only install, lint and test; Stage 12 adds data and model checks.Open
REFERENCEContinuous Delivery for Machine Learning Martin FowlerWhy ML deployment differs from software deployment, plus Google's maturity levels. Read both before designing your pipeline.Open Open
OPTIONALSite Reliability Engineering GoogleRead the monitoring and incident chapters only. They are the source of the 'wrong at 3 AM' framing this stage's gate uses.Open
REFERENCEML in Production Evidently AIPractical guides to data and prediction drift detection, which the stage lists but does not explain.Open
Stage gate
Readiness assessment
Not ready
Stage-specific requirements
Answer each prompt without notes. Expand a question only when you are ready to check your answer.
0/1
Answer: "What happens when this model is wrong at 3 AM?"
Answer
Say what you would look at first and in what order: error rate and latency, then the traces for failing requests, then the inputs to see whether the distribution has moved, then which model version is serving. Say how you would mitigate before you understand it, such as rolling back to the previous version or falling back to a safe default, and say what would tell you the mitigation worked. Then say how the root cause becomes a test.
Universal requirements
The eight conditions 01_LEARNING_RULES.md applies to every stage. Expand one to see what it actually asks for.
0/8
Core concepts can be explained from memory.
What this asks for
Every subject in this stage has review questions. The condition is met when you can answer them out loud, without notes and without looking anything up, in your own words rather than a recited definition. An explanation you can only give with the material open is recognition, not recall.
Required exercises are complete.
What this asks for
Complete means each exercise satisfies its stated requirements and its done-when criteria, not that you attempted it. Where an exercise says you should be able to explain or defend something, that explanation is part of completing it. A stage whose source lists no exercises meets this through its owned project milestone instead.
Stage project works.
What this asks for
Works means it runs end to end from a clean checkout, on the inputs it was built for, with the failure cases it claims to handle actually handled. It is not a notebook that produced a result once. Someone else following your README should be able to run it without asking you a question.
Tests or verification exist.
What this asks for
There is something that would fail if the behaviour regressed, and you have watched it fail. That can be automated tests or a written verification procedure with recorded results, depending on what the stage produces. A test that has only ever passed is evidence of nothing.
README explains decisions.
What this asks for
The README says what the project is, how to install, run and test it, and why it is built the way it is: the choices you made, the alternatives you rejected, and what you deliberately left out. A list of commands with no reasoning does not meet this condition.
You can reproduce the core implementation without copying.
What this asks for
Delete the central piece and write it again from an empty file, without your previous version, a tutorial, or generated code in front of you. It does not have to be identical, it has to work and you have to be able to explain every line. This is the condition that separates having finished from being able.
You pass the stage self-assessment at >= 80%.
What this asks for
Record a gate exam and score yourself honestly against the dimensions the roadmap defines: recall, implementation, debugging, testing and design judgment. Elapsed time, hours watched and completed resources count for nothing here. Checking the boxes without a recorded exam does not open the gate.
Claude/ChatGPT reviewer finds no foundational blocker.
What this asks for
Ask an AI reviewer to examine the work against this stage's concepts and report foundational misunderstandings, not style. A foundational blocker is a misconception that later stages would build on, such as leakage, a broken abstraction, or an explanation that is confidently wrong. Nothing in that class may remain open when you pass.
Recorded exam
No exam recorded. Checking every requirement is not sufficient.