Exercises + milestone requirements owned by this stage
Evidence and gate
0 · 0/11
Artifacts · checked requirements
Why it matters
Purpose of this stage
Test reliability and abuse resistance
Subjects
3 subjects · 28 canonical topics
Each subject states what you should be able to do, an order to work through it in, what to watch for, and the questions a reviewer should be able to ask you at the end. None of it is tracked or scored.
How to work through this subject
An order that works, not a checklist. Nothing here is tracked — the steps sequence this subject’s own topics and add none.
Write the normal cases first: the inputs the system is genuinely for.
Add edge cases at the boundaries — empty, maximum length, unusual but legitimate.
Add adversarial cases written by someone trying to break it, including yourself in that role.
Add malformed input, missing context and contradictory context as separate case classes.
Add cases where a tool fails and cases where the model refuses, and define correct behaviour for each.
Record latency and cost per case so a regression in either is visible.
Separate the suite into component evals and end-to-end evals, and run both.
Finish by breaking one component deliberately and confirming the suite localises the failure.
What to watch for
Separate component evals from end-to-end evals, or a regression will be untraceable.
Latency and cost regressions are quality regressions. Test them.
A refusal can be correct behaviour or a failure, depending on the case. Decide which per case, in advance.
Contradictory context is a case, not an accident. Real corpora contain documents that disagree.
The suite's job is to localise, not merely to detect. A red end-to-end test that names no component is half a signal.
Curriculum topics
normal cases
The expected inputs the system was designed for.
edge cases
Unusual but legitimate inputs — empty, very long, unexpected language — that often break assumptions.
adversarial cases
Inputs deliberately crafted to make the system misbehave.
malformed input
Data that does not parse or does not match the expected shape.
missing context
Cases where retrieval returns nothing relevant, and the system must not invent an answer.
contradictory context
Retrieved material that disagrees with itself, forcing the system to handle conflict.
tool failures
A called tool erroring, timing out or returning nonsense, and what the system does next.
model refusal
The model declining to answer, which must be handled as a real outcome rather than a crash.
latency regression
A change making the system slower — a quality failure, and something to test for.
cost regression
A change making each task more expensive, which must be caught before the bill does.
component eval
Testing one part in isolation, so a regression can be traced to its cause.
end-to-end eval
Testing the whole system as a user experiences it.
Resources for this subject
REFERENCEOpenAI — Evaluating model performance OpenAI The current official documentation for defining eval datasets and graders, and for running an eval as a repeatable job. Read it for the vocabulary — datasets, graders, runs — then implement the same shape yourself.Open
Need another explanation?3 more references
REFERENCECreate strong empirical evaluations Anthropic How to design task-specific evals, choose between code, human and model graders, and where LLM-as-judge stops being trustworthy. Vendor-neutral in substance.Open
OPTIONALRagas Exploding Gradients Metrics for evaluating retrieval and generation separately. Build your own eval set first — the stage requires that before optimisation.Open
OPTIONALopenai/evals (historical reference implementation) OpenAI A historical open-source eval harness. Kept only as a structural example of separating component and end-to-end evals; it is no longer the modern OpenAI eval reference.Open
A reviewer should be able to answer these
Answer from memory, out loud, before moving on. These are not tracked and do not affect the gate — they tell you whether you actually understand this subject.
What is the difference between a component eval and an end-to-end eval?
How do you test behaviour when the retrieved context contradicts itself?
What should the system do when a tool call fails, and how do you test that?
How do you detect a latency or cost regression before users do?
Why is one combined 'AI quality score' a bad idea?
When is a model refusal the correct outcome, and how do you encode that in a test?
What does an adversarial case need to contain to be worth keeping in the suite?
Exercises
Demonstrated practice · 35% base weight
This stage defines implementation through its owned project milestone rather than a separate exercise list.
Resource status is a convenience marker only. It cannot change competencies, gates, or stage completion — in any tier below.
Start here — no canonical resource for this stage
3 available
The curriculum names no resource for this stage. These are authored suggestions, not part of the canonical library.
REFERENCEOWASP Top 10 for LLM Applications OWASPThe reference list of AI-specific risks, matching this stage's topics almost one to one. The strongest starting point for the required threat model.Stage 13 has no primary resource in the canonical library.Open Open
REFERENCEOWASP API Security Project OWASPCovers the non-AI half of the attack surface — broken authorization is still the most common real-world failure.Open
REFERENCEOWASP Cheat Sheet Series OWASPShort, specific guidance on authentication, secrets management and audit logging — all required by the stage project.Open
Need another explanation?6 supplementary resources · not part of the canonical library
Use these for a demonstrated gap, not as a parallel course. They are authored additions and can never be the primary resource for a subject.
OPTIONALRagas Exploding GradientsMetrics for evaluating retrieval and generation separately. Build your own eval set first — the stage requires that before optimisation.Open
OPTIONALAI Risk Management Framework NISTA structured vocabulary for describing residual risk and who accepted it, which the gate requires you to state.Open
REFERENCEMITRE ATLAS MITREA catalogue of real attack techniques against ML systems. Use it to source the five attacks the gate requires you to demonstrate.Open
REFERENCEOpenAI — Evaluating model performance OpenAIThe current official documentation for defining eval datasets and graders, and for running an eval as a repeatable job. Read it for the vocabulary — datasets, graders, runs — then implement the same shape yourself.Provider-specific documentation for a provider-independent idea. The concepts this stage requires — eval sets, graders, component vs end-to-end evals, regression evals — must be implementable without any vendor.Open
REFERENCECreate strong empirical evaluations AnthropicHow to design task-specific evals, choose between code, human and model graders, and where LLM-as-judge stops being trustworthy. Vendor-neutral in substance.Open
OPTIONALopenai/evals (historical reference implementation) OpenAIA historical open-source eval harness. Kept only as a structural example of separating component and end-to-end evals; it is no longer the modern OpenAI eval reference.Historical. Prefer the current evaluation documentation for anything you intend to build.Open
Stage gate
Readiness assessment
Not ready
Stage-specific requirements
Answer each prompt without notes. Expand a question only when you are ready to check your answer.
0/3
Demonstrate at least five attacks/failure modes against your own system
Answer
Five distinct attempts against your own system, each recorded with what you did and what happened: direct prompt injection, indirect injection through content your retrieval will find, an attempt to make the system disclose something it holds, an attempt to make a tool act beyond its purpose, and a malformed or contradictory input that breaks handling. Demonstrated means run and recorded, not described.
Show mitigations
Answer
For each attack, the change you made and the mechanism it relies on: narrower tool permissions, validation of tool arguments, treating retrieved content and model output as untrusted, isolating what the model is allowed to reach, and adding the human approval point for consequential actions. Then rerun every attack and record the result, because an untested mitigation is a hypothesis.
Show residual risk
Answer
State plainly what still works, what is only partially mitigated, and what you chose not to address. For each, say the impact if it were exploited, why the current answer is acceptable, and who accepted it. Residual risk is normal; undocumented residual risk is the problem.
Universal requirements
The eight conditions 01_LEARNING_RULES.md applies to every stage. Expand one to see what it actually asks for.
0/8
Core concepts can be explained from memory.
What this asks for
Every subject in this stage has review questions. The condition is met when you can answer them out loud, without notes and without looking anything up, in your own words rather than a recited definition. An explanation you can only give with the material open is recognition, not recall.
Required exercises are complete.
What this asks for
Complete means each exercise satisfies its stated requirements and its done-when criteria, not that you attempted it. Where an exercise says you should be able to explain or defend something, that explanation is part of completing it. A stage whose source lists no exercises meets this through its owned project milestone instead.
Stage project works.
What this asks for
Works means it runs end to end from a clean checkout, on the inputs it was built for, with the failure cases it claims to handle actually handled. It is not a notebook that produced a result once. Someone else following your README should be able to run it without asking you a question.
Tests or verification exist.
What this asks for
There is something that would fail if the behaviour regressed, and you have watched it fail. That can be automated tests or a written verification procedure with recorded results, depending on what the stage produces. A test that has only ever passed is evidence of nothing.
README explains decisions.
What this asks for
The README says what the project is, how to install, run and test it, and why it is built the way it is: the choices you made, the alternatives you rejected, and what you deliberately left out. A list of commands with no reasoning does not meet this condition.
You can reproduce the core implementation without copying.
What this asks for
Delete the central piece and write it again from an empty file, without your previous version, a tutorial, or generated code in front of you. It does not have to be identical, it has to work and you have to be able to explain every line. This is the condition that separates having finished from being able.
You pass the stage self-assessment at >= 80%.
What this asks for
Record a gate exam and score yourself honestly against the dimensions the roadmap defines: recall, implementation, debugging, testing and design judgment. Elapsed time, hours watched and completed resources count for nothing here. Checking the boxes without a recorded exam does not open the gate.
Claude/ChatGPT reviewer finds no foundational blocker.
What this asks for
Ask an AI reviewer to examine the work against this stage's concepts and report foundational misunderstandings, not style. A foundational blocker is a misconception that later stages would build on, such as leakage, a broken abstraction, or an explanation that is confidently wrong. Nothing in that class may remain open when you pass.
Recorded exam
No exam recorded. Checking every requirement is not sufficient.