TPEE Toptronic® Prompt Engineering Editor

126. Abliteration 4 - Doing It: Measure First, Then Edit, Then Prove

The working order that keeps the exercise an experiment rather than a demolition.

By Jacques Botte, founder of Toptronic®. Last updated 17 October 2026.

The lesson

Before any edit, build the test set. You need three of them, and they are not interchangeable: prompts that should be refused (harmful requests, in your jurisdiction's sense), prompts that should NOT be refused but often are (legitimate medical, security, legal, historical, translated-harsh-language questions), and a general capability set - reasoning, code, arithmetic, summarisation, multilingual - to detect the damage you are not looking for.

Record the baseline. Refusal rate on set one, over-refusal rate on set two, capability scores on set three, and the exact model revision and quantisation. A number without the revision beside it is not a baseline; it is an anecdote. This is also the moment to decide what success means: 'refusals fall from 92 percent to under 10 percent while the capability set moves less than 2 points' is a testable claim, and 'it feels more open' is not.

Then fit the direction, and validate it out of sample. Split the prompt sets, fit on one half, and check on the other - if the effect only appears on prompts the direction was fitted to, you have measured your dataset, not the model. Tools in this space expose the fitted vector and the layer choice; that choice is the main lever, and it should be justified rather than defaulted.

Apply the edit to a COPY of the weights, never to your only copy, and version it like the artifact it is: source revision, dataset hash, layer, scaling coefficient, date, and who approved it. Run the three sets again on the edited model, and run them again on a different quantisation if you plan to ship one - quantisation after weight surgery can shift behaviour further, and that interaction has surprised more than one team.

Finally, write the report before you decide whether the result is good. Refusal rate before and after, over-refusal before and after, capability deltas with confidence intervals where you have them, and the failure cases you found by hand. A professional document makes it possible for somebody who disagrees with you to check your work - which is the only kind of result worth having.

Check yourself

Question 1: What are the three test sets for?
  1. Speed, cost, size
  2. Prompts that should be refused, prompts that should NOT be refused, and general capability — correct
  3. Training, validation, test
  4. English, code, maths

Answer: Prompts that should be refused, prompts that should NOT be refused, and general capability

One set measures the behaviour you are removing, one measures the over-refusal you are trying not to create, and one measures the damage you are not looking for.

Question 2: Why must the direction be validated out of sample?
  1. To make it faster
  2. Because an effect that appears only on the fitted prompts measures your dataset, not the model — correct
  3. Because the licence requires it
  4. Because otherwise the file is too large

Answer: Because an effect that appears only on the fitted prompts measures your dataset, not the model

Fit on one half, check on the other. Otherwise the reported effect is a property of your prompt list.

Question 3: What interaction is most often missed after weight surgery?
  1. The tokenizer changes
  2. Quantisation applied afterwards can shift behaviour further — correct
  3. The licence expires
  4. The context window shrinks

Answer: Quantisation applied afterwards can shift behaviour further

Weight edits and quantisation are not independent: re-run the evaluation on the exact artifact you intend to distribute.

← Previous lesson · All 134 lessons · Next lesson →

The full course — 134 lessons and 402 quiz questions — ships inside the app. Get TPEE to study it offline.

This lesson is the written summary of a course that ships inside the TPEE prompt editor for Windows - thirteen sections, a 0-100 score before you send anything, and it works with no network connection. The three one-time prices, and a 14-day free trial.