AI evaluation is measuring whether a model does what you need, honestly and repeatably. A single accuracy number hides how a system fails, on whom, and how sure the number is. Good evaluation chooses metrics that match the real task, tests on data the model has not seen, and reports uncertainty so decisions rest on evidence, not on impressions.
Key concepts
- Metric choice: accuracy, precision and recall, ranking and error metrics each answer a different question; pick by the cost of each kind of mistake.
- Held-out data: test sets must be separate from training and tuning data, and kept untouched until the end.
- Contamination: test examples leaking into training data inflate scores; check for overlap and duplicates.
- Uncertainty: confidence intervals and several random seeds show whether a difference is real or noise.
- Slice analysis: results broken down by group, input type or difficulty reveal failures that the average hides.
Vocabulary
- Ground truth: the labels or answers taken as correct for scoring.
- Precision and recall: the share of flagged items that are right, and the share of real items found.
- Calibration error: the gap between stated confidence and actual accuracy.
- Inter-rater agreement: how often independent judges give the same rating.
What to study, in order
- Probability and statistics: sampling error, confidence intervals, hypothesis tests.
- Compute a confusion matrix by hand and derive every metric from it.
- Evaluate one model with a bootstrap interval, and decide if a small gain is meaningful.
- Build a slice report for a classifier and find the group it serves worst.
- Design an evaluation for an open-ended task: write cases, a rubric and a way to check rater agreement.