A hot topic in the AI/ML engineering world is whether PMs should own and write evals. I tend to agree that, as the owners of the product’s scope and goals, PMs are natural owners for evals. But this ownership question glosses over the hard part: building a good evals program is actually six jobs. I approach this problem-first instead of title-first, which allows us to map clearer ownership to the underlying problems.
The six jobs of an eval program
An eval program is a build-measure-learn loop in miniature. We can think about each job as having a required output that feeds the next phase, with the whole cycle forming a learning loop.
Define “good”. Off-the-shelf evals rarely perform well in product-specific scenarios, so the bulk of every product’s evals should be tailored to its goals and moment of value. Building a useful eval program requires you to define what good outputs look like for your product. The output of this job is a framework for evaluation: categories of test cases, with criteria for evaluating them.
Define what to test. With a definition of “good”, we have what we need to construct a test set. The best test cases are a blend of real-world samples and intentionally constructed edge- or adversarial-cases. These should tie back directly to your definition of “good”. The output of this job is: specific test cases with evaluation criteria.
Measure performance. Build a scorer with rubrics, judges, and code for deterministic evaluation. As with any measurement program, we have to consider the right statistical methods and required sample size. The output of this job is: runnable evals that produce interpretable results.
Calibrate. If you’re using models to evaluate models (LLM-judges, classifiers, etc.), then these measures need to be calibrated themselves because their native outputs may not represent what you expect. Your investment here should be calibrated to your level of risk. Curate smaller, human-labeled sets to check your scorers. Output: scoring models calibrated against a validation set (evals in miniature!)
Monitor performance. One-time snapshot evals are useful, but ultimately this is a part of your CI process. At a minimum, you’re testing for regressions and evaluating model upgrades. If you’re using evals as a compass, you need ongoing measurement to know where you’re going. Where possible, you also want to be running evaluations against production traffic, or at least logging for future eval development. This job is classic MLE, and its output is a harness with logs and metrics.
Learn. This applies to the product and to the eval set itself. For the eval set: read failures as they come in, and use error analysis to refine and build on your eval framework and test cases. Evals serve as a yardstick for product decisions that connect to the models (version changes, reasoning budgets, etc.). The evals are a benchmark to guide judgement. Output: expanding eval-set (and confidence), data-informed product changes.
Mapping the jobs to your team
With these jobs defined, you can map out concrete accountabilities and ownership. The way they fit into your org depends on the structure of your team, but it’s helpful to think about them in terms of the expertise they require.
If you’re an AI startup with no data science team, you have gaps to fill in measurement and calibration. No matter what roles you have on your team, you need domain expertise. The PM fills this role in a pinch, but without a subject-matter expert, evals will tend to be defined by what’s easiest to measure. Scores might be green, but if an expert would reject the results, you’re just playing make believe.
Falling through the cracks
Calibration and monitoring often fall through the cracks because they seem optional. The models are getting better all the time, right? If you’re renting these models without running calibrated scorers on an ongoing basis, you’re putting the performance of your product in someone else’s hands on little more than faith. The same goes for your LLM judges: when the vendor’s model update comes out, your scores will shift, regardless of whether your product did.
The frontier labs are incentivized to make their models better, but better against their benchmarks does not necessarily mean better at your tasks.
So, should PMs own evals?
Sure, at least in part. The first two jobs and the learning loop clearly land in PM territory. But it takes a village to build an effective eval program. Start with the six jobs, find out who on your team is well-positioned to do them, assign accountabilities, and see what’s left. If you’ve still got a gap, now you know what you’re hiring for.