Grading AI Work: But Who Grades the Grader?

Suppose you’ve built an AI tool that drafts documents that are genuinely open-ended, e.g. a business case for entering a new market. Consistency and quality of the tool’s output are important to you, so you want to measure its quality systematically rather than take it on faith. Using the approach described in the last post for Grading AI Work, you’ve created a rubric for what “good” looks like from a variety of perspectives. 

Another “Grader AI” applies that rubric to every draft produced and scores it, generating an assessment: say, Rigor gets 4/5, Evidence is 3/5, but Balance is only 2/5. That’s very helpful, now you can notice a change that makes the output worse, and see what to fix next.

Except: the tool that’s producing the grading assessment is itself also an AI, which can be confidently wrong in exactly the same way the tool it is grading can be. And nothing in the score tells you whether it's valid. So who grades the grader? The naive answer is to point yet another AI at it - but where does it end? As they say, it’s turtles all the way down.

Nothing Shows Red When The Grader Is Wrong

How can the grader fail? That sounds like a philosophical puzzle, but in practice it can happen in three ordinary ways:

• It can be too soft, waving through a weak attempt
• It can be too harsh, blocking work that was perfectly acceptable
• Worst of all, it can be inconsistent: the same draft scores 4 one day and 2 the next; exactly the repeatability you demanded of the tool, now broken one level up

What makes this all the more dangerous is that all of these failures are silent. When ordinary software breaks, a test case goes red. But if the grader goes off-track, it doesn’t break; it returns a perfectly plausible score, in the right format, every time. You can run a broken grader for months and never know.

The Chain Ends With A Human

You won't spot any of this from the assessment itself. Picture the scores coming back as excellent on every axis, week after week. That could be a tool performing beautifully. It could equally well be a grader that has quietly stopped discriminating - and there is nothing in the numbers to tell the two apart.

Adding another AI to judge those scores doesn’t help either; it just moves the same question one step further out. This progression ends in only one place: a human reads the actual work and forms their own view. Not every document - only a handful, at regular intervals. Take a few recent drafts, judge them the way an experienced reviewer would, and hold that judgment up against the grader's. Where the two agree, you've earned some confidence in the score. Where they diverge, you've found something.

The leverage is in what you do next. Instead of limiting yourself to fixing that one score, what you’re really asking is why the grader landed somewhere you didn't. Sometimes it misapplied a sound rubric and needs better instructions, but sometimes the rubric itself is wrong - it measured exactly what you asked for, and you asked the wrong question. Either way, if you fix the standard once, every future assessment gets better. That is what makes this process affordable: the human isn't grading the output, they're maintaining the rubric.

None of this makes the grader bulletproof, and it’s not meant to. What it buys you is a score that you have some reason to believe - one that was checked recently, against the judgment of someone you would trust.

As open-ended work products increasingly get delegated to more and more powerful AI models, establishing trust by maintaining the quality and consistency of their work is critical. Making “good” explicit allows you to systematically measure work that has no right answer; keeping a human check on the grader allows you to validate the measure itself. Otherwise the dashboard stays green, and you'll be the last to know.