One lesson
An individual lesson
This track has no lesson ready to buy yet. Each lesson becomes available only after it is built and reviewed.
$5.00Not available yet
All course tracks
AI and machine learning · 30 lessons · up to 10 minutes each
AI Evaluation and Testing
A planned evaluation path from acceptance criteria through regression review.
Coming Soon
One-time purchase · USD
Choose what to unlock
Access is attached to your account after payment is verified. No subscription.
Full category
AI Evaluation and Testing
Permanent access to the lessons published in this track when you buy. Future lessons are not included. 0 of 30 lessons are published; 0 are in owner review and 30 are Coming Soon.
$29.0030 lessons still in progress
Beginner
Lessons 1–10
01 AI Evaluation and Testing: Sketch behavioral acceptance criteria
Topic: behavioral acceptance criteria
Draw a small map of behavioral acceptance criteria in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Beginner4 minComing Soon02 AI Evaluation and Testing: Name the parts of representative eval cases
Topic: representative eval cases
Label the main pieces in representative eval cases in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Beginner5 minComing Soon03 AI Evaluation and Testing: Trace an example of gold and reference data
Topic: gold and reference data
Follow one input through gold and reference data in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Beginner6 minComing Soon04 AI Evaluation and Testing: Compare choices for deterministic checks
Topic: deterministic checks
Write down one practical difference between deterministic checks in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Beginner7 minComing Soon05 AI Evaluation and Testing: Find the boundary in grader calibration
Topic: grader calibration
Mark where grader calibration in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Beginner8 minComing Soon06 AI Evaluation and Testing: Inspect an example of quality and cost metrics
Topic: quality and cost metrics
Annotate a minimal example of quality and cost metrics in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Beginner9 minComing Soon07 AI Evaluation and Testing: Connect regression comparisons to a neighboring concept
Topic: regression comparisons
Show how regression comparisons in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Beginner10 minComing Soon08 AI Evaluation and Testing: Spot a failure mode in failure clustering
Topic: failure clustering
Describe one likely failure for failure clustering in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Beginner3 minComing Soon09 AI Evaluation and Testing: Read an official example of human review sampling
Topic: human review sampling
Summarize an official example of human review sampling in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Beginner5 minComing Soon10 AI Evaluation and Testing: Choose a use case for release thresholds
Topic: release thresholds
Match one small use case to release thresholds in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Beginner6 minComing Soon
Intermediate
Lessons 11–20
11 AI Evaluation and Testing: Build an example with behavioral acceptance criteria
Topic: behavioral acceptance criteria
Create a small working example that uses behavioral acceptance criteria in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Intermediate4 minComing Soon12 AI Evaluation and Testing: Compose representative eval cases with another component
Topic: representative eval cases
Combine representative eval cases in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Intermediate5 minComing Soon13 AI Evaluation and Testing: Configure gold and reference data for a small case
Topic: gold and reference data
Set up gold and reference data in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Intermediate6 minComing Soon14 AI Evaluation and Testing: Validate inputs around deterministic checks
Topic: deterministic checks
Add a check around deterministic checks in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Intermediate7 minComing Soon15 AI Evaluation and Testing: Use grader calibration in a short workflow
Topic: grader calibration
Add grader calibration in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Intermediate8 minComing Soon16 AI Evaluation and Testing: Refactor quality and cost metrics into a reusable unit
Topic: quality and cost metrics
Extract quality and cost metrics in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Intermediate9 minComing Soon17 AI Evaluation and Testing: Exercise regression comparisons with sample data
Topic: regression comparisons
Run sample data through regression comparisons in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Intermediate10 minComing Soon18 AI Evaluation and Testing: Observe failure clustering outputs
Topic: failure clustering
Capture the result of failure clustering in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Intermediate3 minComing Soon19 AI Evaluation and Testing: Recover from a human review sampling failure
Topic: human review sampling
Handle one expected failure from human review sampling in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Intermediate5 minComing Soon20 AI Evaluation and Testing: Document release thresholds behavior
Topic: release thresholds
Write a short usage note for release thresholds in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Intermediate6 minComing Soon
Advanced
Lessons 21–30
21 AI Evaluation and Testing: Test a boundary around behavioral acceptance criteria
Topic: behavioral acceptance criteria
Add an edge-case test for behavioral acceptance criteria in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Advanced4 minComing Soon22 AI Evaluation and Testing: Harden representative eval cases
Topic: representative eval cases
Add the relevant access, safety, or validation control to representative eval cases in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Advanced5 minComing Soon23 AI Evaluation and Testing: Measure gold and reference data
Topic: gold and reference data
Record a small baseline for gold and reference data in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Advanced6 minComing Soon24 AI Evaluation and Testing: Reduce cost in deterministic checks
Topic: deterministic checks
Change one measurable cost in deterministic checks in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Advanced7 minComing Soon25 AI Evaluation and Testing: Diagnose a grader calibration failure
Topic: grader calibration
Use an observable signal to explain a failure in grader calibration in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Advanced8 minComing Soon26 AI Evaluation and Testing: Make quality and cost metrics repeatable
Topic: quality and cost metrics
Show that rerunning quality and cost metrics in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Advanced9 minComing Soon27 AI Evaluation and Testing: Review trade-offs in regression comparisons
Topic: regression comparisons
Compare two documented approaches to regression comparisons in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Advanced10 minComing Soon28 AI Evaluation and Testing: Improve failure clustering maintainability
Topic: failure clustering
Refine failure clustering in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Advanced3 minComing Soon29 AI Evaluation and Testing: Verify human review sampling end to end
Topic: human review sampling
Check that a complete small flow using human review sampling in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Advanced5 minComing Soon30 AI Evaluation and Testing: Audit release thresholds against its source docs
Topic: release thresholds
Check the implementation of release thresholds in the context of repeatable evaluations for AI outputs and workflows; save a short example or check that demonstrates the result.
Advanced6 minComing Soon
Reference documentation
References support the selected track topics while its lesson content is built and reviewed progressively.
This track is a catalog outline only. Each lesson remains unavailable until it is built and personally reviewed.