Project
I recently ran an experiment on one of my home servers: take a small language model and surgically remove its habit of refusing questions. The experiment worked — and in working, it taught me more about how these models are built than any tutorial has. This page explains, in order: how a language model gets trained in the first place, where "refusal" actually lives inside one, what abliteration does to it, exactly what we ran on home hardware, and what the results did not support. Every number below came from the experiment itself, and the honest failures are included next to the successes.
How a language model is made
A modern assistant model is not programmed; it is grown, in three rough stages.
1. Pre-training — learning how language works. The model starts as random numbers. It is shown enormous amounts of text and asked to do one thing repeatedly: predict the next word. Each prediction that comes out wrong nudges billions of internal numbers (the weights) slightly toward being less wrong next time. After trillions of words, the numbers have organized themselves into something that can complete sentences, answer questions, and reason, because modeling language well requires modeling the world the language describes. This stage costs millions of dollars and is where nearly all knowledge lives.
2. Fine-tuning — learning to be helpful. A raw pretrained model completes text; it doesn't answer questions. In supervised fine-tuning, humans write examples of good question-and-answer exchanges and the model trains on them, so that input in the shape of a question produces output in the shape of an answer. Full fine-tuning adjusts every weight again. The cheaper variant, LoRA, freezes the pretrained weights and trains a small sidecar of new numbers (typically under 1% of the model) that gets added on top — which is why a "fine-tune" can now take hours on one machine instead of weeks on a cluster.
3. Preference alignment — learning what it should say. Humans compare pairs of answers ("this one is better") and the model is trained to prefer the winners — the family of methods behind RLHF and its lighter cousins like DPO and ORPO. This is the stage that installs manners and safety behavior: refusals of weapons instructions, evasiveness on borderline topics, tone. Crucially, the knowledge from pre-training is untouched; alignment changes what the model is willing to do with that knowledge. Everything this page is about follows from that one fact.
Where refusal lives
In 2024, a paper that gives this project its name — "Refusal in Language Models Is Mediated by a Single Direction" (Arditi et al., arXiv:2406.11717) — produced a strange result. The authors took a chat model, fed it pairs of prompts that were harmful versus harmless in shape but similar in topic, recorded the model's internal activations, and computed a single difference vector: one direction in the model's activation space along which "this prompt should be refused" points. Then they removed activity along just that direction and found that refusal behavior largely vanished — across models, languages, and prompt styles. Safety behavior, it turns out, sits on one shallow lever, not deep in the knowledge itself.
Here is the part that connects it back to training. You can suppress refusal at runtime (prompt tricks that keep the lever flipped — fragile, and every model update breaks them), or you can take the weights themselves and orthogonalize the linear layers against that one direction, which erases the lever permanently from the file on disk. No training data of answers, no gradient descent over the corpus — a pass of linear algebra over the weights. Some people call the result an "abliterated" model. In the vocabulary of this page: abliteration is not fine-tuning at all. Fine-tuning and alignment add behavior by changing weights toward preferences; abliteration subtracts one specific behavior by editing weights away from one direction. Same material (numbers in tensors), opposite motion.
The experiment
The base model is Qwen3-4B-Instruct — a small open model, 4 billion parameters, suitable because the whole pipeline runs on home hardware. The lab machine is a compact AMD box with 128 GB of unified memory; unified memory is the trick that makes this class of experiment feasible at home, because the same pool serves the GPU, and a 4B model plus training overhead fits with room to spare. While a production model kept serving chat traffic, we measured a 20-step LoRA calibration run first: about 540 tokens/second of training throughput coexisting with the live model, roughly 10 GB peak memory — enough to plan everything else from real numbers instead of borrowed ones.
Then the procedure, following the paper: collect paired prompt sets, run the model on them, compute the refusal direction, remove it from the weights, and measure the before and after. The before-and-after was a battery of 100 harmful requests and 30 harmless controls (harmless questions phrased to look scary — a classic false-refusal trap), scored by refusal-phrase matching with a human reading spot-checks of roughly twenty replies. Plus a quality canary: perplexity on plain English, a rough proxy for collateral damage.
Two versions of the method ran. My original scripts restricted the removal to a band of middle layers (a common choice in community implementations); the pass completed, looked successful, and — as the measurement showed — had removed nothing, a combination of the wrong layer range and four silent incompatibilities with the current version of the ML libraries. The second version, written for this campaign, estimated the direction from 256 harmful / 256 harmless pairs and removed it from all layers.
Results
| Model | Refused harmful (of 100) | False refusals (of 30) |
|---|---|---|
| Stock Qwen3-4B-Instruct | 100 | 0 |
| First attempt (middle layers) | 100 | 0 |
| Second attempt (all layers) | 2 | 0 |
| Community abliteration* | 0 | 0 |
*Built on a different base model; reference point, not a fair comparison. "Harmless controls" = harmless questions phrased to look alarming.
Plain-English perplexity moved from 1.17 to 1.38 on the reference text: a measurable ripple from editing the weights — a real cost, though perplexity is a blunt instrument and not a capability benchmark.
What it worked — and what it didn't
It worked in exactly the sense the paper promises: a single-direction edit, computed in minutes on a mini PC, turned a model that refused 100 of 100 harmful requests into one that refused 2, while the harmless controls that plague lesser jailbreaks stayed at zero false refusals. The mechanism is real, reproducible, and cheap on hardware I keep on a desk.
It failed on the question that matters more, and this is the finding I want on the record: the removal is not selective. The model does not develop refined judgment about which dangerous topics are legitimate to discuss; it complies with nearly everything, including the requests any sane operator wants refused. The pre-commitment for this project was that the model must still refuse genuinely harmful content. It doesn't, so the model is quarantined: an experimental file on a home server, never wired into anything that answers questions for real humans, original weights untouched, everything reversible. We also skipped the planned follow-up pass (a light preference-tuning step) precisely because it tunes willingness — pushing the same lever harder in a direction we'd just proven dangerous.
Two smaller honest notes. The scoring is rough: phrase matching plus a human skim, with no independent judge model available for this run — directionally solid, but not audit-grade. And the first attempt failing silently is its own lesson: a pipeline that produces no error is not a pipeline that worked, which is why the campaign measured before trusting anything.
What I take from it
Three things. First, alignment is shallower than the intuition suggests: the behavior was one direction deep, while the knowledge it hides is the entire model — which argues for safety work at the level of what models know how to do, not just whether they'll say it. Second, editing a trained model is a spectrum, and "fine-tune" undersells it: gradient-based training, preference optimization, and one-pass linear-algebra surgery are all weight edits at different scales. Third, a project is only as good as the gate that can stop it — the model failed our pre-agreed check and the deployment didn't happen, which is the whole system working as designed. The next experiment, when I get there, is dialing the removal down instead of skipping it: partial-strength edits, searching for a knee where over-cautious refusal dies and real judgment survives.
This project was researched, run, and written by my local AI agent on a home server (with me reviewing and correcting it). All measurements came from the lab run itself; the method is published in the open literature, and the model produced by it is not deployed anywhere. Last updated 2026-10-21.