TPEE Toptronic® Prompt Engineering Editor

131. Distillation 2 - The Corpus Is the Ceiling: Generating, Filtering, Licensing

The dataset decides what the student can become. Build it deliberately.

By Jacques Botte, founder of Toptronic®. Last updated 17 October 2026.

The lesson

The corpus is not a convenience; it is the specification. The student can only learn behaviour that appears in it, on inputs that resemble it, and it inherits every defect in it - the teacher's hallucinations, its verbosity, its refusal habits, its formatting quirks. So the first decision is not which teacher, it is which INPUTS, sampled to look like your real traffic rather than like a benchmark.

Prompts should be drawn from three places: your own production distribution (anonymised), a deliberately constructed edge-case set (empty input, hostile input, multilingual input, very long input), and a small set of things the teacher is known to be good at that your product needs. The proportions are a product decision, and they should be written down, because the student will reproduce them faithfully.

Generation is where the money and the time go. Temperature, sampling and the request shape all matter, and one pass is usually not enough: you generate, you inspect a sample by hand, you adjust, and you regenerate a slice. Keep the teacher's raw outputs and the prompt that produced each one, so a later audit can ask 'why does the student answer this way' and get an answer.

Filtering is the part teams skip and regret. Drop duplicates near-verbatim, because they teach the student to be repetitive. Drop refusals unless refusals are the target behaviour. Drop outputs that fail a deterministic check you can write - invalid JSON, wrong length, a hallucinated citation pattern, a broken code block. Keep the ones you drop in a separate folder with the reason: that folder is where the next debugging session starts.

Licensing decides whether any of it is usable. Many providers' terms restrict using outputs to train a competing model, and some restrict it entirely; open-weight teachers have their own licences, and a derived model may inherit obligations. Check before you generate, not after you have spent a week and a budget - and record the terms you relied on, with the date you read them, in the same document as the corpus statistics.

Check yourself

Question 1: Why is the prompt distribution a product decision?
  1. Because it affects cost only
  2. Because the student reproduces the corpus faithfully, so its proportions become the product's behaviour — correct
  3. Because the teacher requires it
  4. Because it changes the tokenizer

Answer: Because the student reproduces the corpus faithfully, so its proportions become the product's behaviour

The corpus is the specification: what you sample, and in what proportion, is what the student becomes.

Question 2: Which outputs should be filtered out?
  1. Only empty ones
  2. Near-duplicates, unintended refusals, and outputs that fail a deterministic check - kept separately with the reason — correct
  3. Everything the teacher wrote at low temperature
  4. Only outputs longer than the window

Answer: Near-duplicates, unintended refusals, and outputs that fail a deterministic check - kept separately with the reason

Filtering removes behaviour you do not want taught, and keeping the rejects turns them into evidence for the next session.

Question 3: When should the licensing question be answered?
  1. After training
  2. Before generating a corpus, with the terms and the date recorded — correct
  3. Only when selling
  4. Never - outputs are always free to use

Answer: Before generating a corpus, with the terms and the date recorded

Terms on using outputs for training vary and can prohibit it; the check belongs before the spend.

← Previous lesson · All 134 lessons · Next lesson →

The full course — 134 lessons and 402 quiz questions — ships inside the app. Get TPEE to study it offline.

This lesson is the written summary of a course that ships inside the TPEE prompt editor for Windows - thirteen sections, a 0-100 score before you send anything, and it works with no network connection. The three one-time prices, and a 14-day free trial.