TPEE Toptronic® Prompt Engineering Editor

134. Distillation 5 - Economics, Ownership and the Maintenance Bill

The decision a senior engineer owns: when distilling is right, and what it costs to keep.

By Jacques Botte, founder of Toptronic®. Last updated 17 October 2026.

The lesson

Distil when the task is narrow and stable, the traffic is high enough that per-call cost matters, latency is a product constraint, and you can measure success deterministically. Do not distil when the task is broad, when the facts change weekly, when you cannot evaluate it, or when the traffic is small enough that the teacher's cost is a rounding error. A student is an asset with a maintenance bill; small traffic rarely pays it.

The economics are worth doing honestly. Compare: teacher cost per task times volume, against student inference cost plus the corpus generation cost plus the training cost plus the evaluation cost plus the annual maintenance - re-training when the base model moves, re-running the evaluation when the traffic drifts, and the engineering time to keep all of it alive. Teams routinely forget that the corpus must be regenerated when the distribution changes, which is the largest recurring line.

Ownership is the part with legal teeth. If the corpus came from a provider whose terms restrict training competing models, the derived artifact may be unusable for the purpose you built it for. If the base is an open-weight model, its licence travels with your adapter. And the student is YOUR artifact: you are the one who must be able to say what it was trained on, when, from which teacher revision, under which terms - because a customer, an auditor or a court may ask.

Drift arrives three ways, as with any tuned artifact. The traffic changes shape. The teacher is upgraded, and your corpus no longer represents the behaviour you priced. The world changes, and a fact in the corpus becomes wrong. Only the third is dangerous in the way this course keeps warning about, because a fact the student learned is now an error with no provenance to correct. Put the facts in retrieval; keep the weights for behaviour.

The closing judgement, and it is the same judgement the whole course makes: distillation is a scale decision, not a capability decision. It buys you cost and latency on a task you can measure, and it charges you a corpus, an evaluation set and a maintenance habit. Do it when the measurement exists and the economics are written down; skip it when either is missing - and either way, keep the teacher callable, because the day the student meets an input outside its corpus is the day you will want it back.

Check yourself

Question 1: When is distillation the wrong choice?
  1. When traffic is high
  2. When the task is broad, the facts change often, success cannot be measured, or volume is too small to pay the maintenance — correct
  3. When latency matters
  4. When the teacher is expensive

Answer: When the task is broad, the facts change often, success cannot be measured, or volume is too small to pay the maintenance

A student is an asset with a recurring bill; broad, fast-moving or unmeasurable tasks do not repay it.

Question 2: Which cost line do teams most often forget?
  1. Inference
  2. Regenerating the corpus when the distribution or the teacher changes — correct
  3. Electricity
  4. Licence review

Answer: Regenerating the corpus when the distribution or the teacher changes

The corpus is the specification, so when the traffic or the teacher moves, the corpus must move with it.

Question 3: Why should facts stay in retrieval rather than in a distilled student?
  1. Retrieval is cheaper
  2. A fact learned into weights becomes an error with no provenance to correct or attribute — correct
  3. Weights cannot store facts
  4. Retrieval is faster

Answer: A fact learned into weights becomes an error with no provenance to correct or attribute

Behaviour belongs in weights; facts need a source that can be corrected and cited.

← Previous lesson · All 134 lessons

The full course — 134 lessons and 402 quiz questions — ships inside the app. Get TPEE to study it offline.

This lesson is the written summary of a course that ships inside the TPEE prompt editor for Windows - thirteen sections, a 0-100 score before you send anything, and it works with no network connection. The three one-time prices, and a 14-day free trial.