TPEE Toptronic® Prompt Engineering Editor

110. Loop Engineering 4 - The Check: Verifiers, Tests and Rubrics

Build the check layer: which verifier to use, how to make failures informative, and when a rubric is the only honest option.

By Jacques Botte, founder of Toptronic®. Last updated 17 October 2026.

The lesson

The check is the part of a loop that earns its keep, and it comes in four flavours. A DETERMINISTIC verifier is code: a test suite, a type checker, a schema validator, a hash comparison, a regex over an output format. A SAMPLED verifier runs part of the work and inspects the result, such as compiling one file instead of the project. A MODEL verifier asks another model, ideally a different one, to judge against a rubric. A HUMAN verifier is a person, and it is the only one that can decide whether the goal was the right goal.

Use the cheapest verifier that can actually fail for the reason you care about. Compilers, tests and schemas are free to run, exact, and immune to flattery. Reach for a model verifier only where the property is genuinely linguistic - is this summary faithful, is this answer on-topic, does this email sound like us - and accept that you have traded exactness for coverage. The mistake is using a model verifier for something a five-line assertion could have settled.

A failing check must say what failed in terms the next turn can act on. A verifier that answers false costs the loop a full turn of guessing; a verifier that answers with the failing test name, the expected and actual values, the rule number and the file and line lets the next turn go straight to the fault. Design the verifier's OUTPUT first, then its logic - the same principle as designing an error message before writing the validation.

A rubric is a fixed list of criteria, each with a level description and an example of a failure. It works because it is boring: the same words every time, no room to reward length or confidence. Write the failure examples yourself from real bad output, because a rubric with only descriptions of excellence cannot recognise the failure modes you actually see - and ask for a verdict plus a one-line reason, so a human can audit a sample later without re-reading everything.

Whatever mix you choose, record which verifier produced which verdict. When a loop ships something it should not have, the question is always the same: what said yes, and would it say yes again? A loop whose checks are named, versioned and reviewable is a system. A loop whose checks are an unrecorded opinion is a person-shaped hope with a for-loop around it.

Check yourself

Question 1: Which verifier should be used first where it can fail for the right reason?
  1. A model judge, because it understands language
  2. A deterministic verifier such as a test, schema or hash — correct
  3. A person, because they are most accurate
  4. Whichever is slowest

Answer: A deterministic verifier such as a test, schema or hash

Deterministic checks are free, exact and cannot be talked round. Use a model or a person only where the property is genuinely linguistic.

Question 2: Why should a verifier's output be designed before its logic?
  1. Because output is easier to write
  2. Because a bare false costs the loop a whole turn of guessing, while a named failure lets the next turn act — correct
  3. Because users read logs
  4. Because logs must be translated

Answer: Because a bare false costs the loop a whole turn of guessing, while a named failure lets the next turn act

The loop consumes the verdict. Naming the failing test, the expected and actual values and the rule turns a wasted turn into a repair.

Question 3: What makes a rubric useful to a loop?
  1. It rewards confident, longer answers
  2. It states the same fixed criteria every time, with real failure examples, and returns a verdict plus a reason — correct
  3. It is written by the model being judged
  4. It changes with each run

Answer: It states the same fixed criteria every time, with real failure examples, and returns a verdict plus a reason

Fixed criteria and real failure examples are what make a rubric auditable; a verdict plus a reason lets a person spot-check a sample.

← Previous lesson · All 134 lessons · Next lesson →

The full course — 134 lessons and 402 quiz questions — ships inside the app. Get TPEE to study it offline.

This lesson is the written summary of a course that ships inside the TPEE prompt editor for Windows - thirteen sections, a 0-100 score before you send anything, and it works with no network connection. The three one-time prices, and a 14-day free trial.