Fool’s Gold

Safety alignment in open-weight language models is trivially removable: abliteration projects a refusal-mediating direction out of the weights in minutes, and no release-time defense we are aware of prevents it durably. What cannot be prevented can be deceived. Our defense, decoy hardening (“Fool’s Gold”), concedes the refusal strip and poisons its payoff: once refusal is stripped, most answers to hazardous operational requests are confident, fluent decoys whose critical elements are falsified.

The decoy behavior is trained inside a differentiable simulation of the attack, so it expresses in the attacked state, while a refusal pin and a benign leash hold clean-state behavior to the original. We instantiate the defense on seven models from five families (9B–122B, dense and mixture-of-experts). On the six models passing the registered efficacy gate, the attack turns 0.51–0.90 of attacked-state draws on never-trained prompts into decoys under the recipe of record (defense-attributable +0.27 to +0.84), within registered benign and capability budgets; the seventh, a smaller model, fails the gate and is reported as a boundary case. Measured rates replicate on a frozen, never-individually-inspected test split on five of the seven models — every value within ±0.05 — and on fully untouched held-out strata on the remaining two (gemma-4 Δ +0.755; gpt-oss Δ +0.135, interval excluding zero).

The security claim is epistemic: an attacker lacking an independent source of correct values cannot separate falsified answers from correct ones — on the external red-team benchmarks’ CBRNE-adjacent slice the defended 122B model is fatally wrong on 0.82–0.86 of matched-quality answers versus at most 0.10 undefended, with no exploitable surface tell. Consensus voting across draws does not restore trust: its precision is unobservable to the attacker — the same vote that returns correct composites on one model returns mostly-falsified composites on another. The defense is inert against in-context jailbreaks by design and applies to first-release models only.

Motivation

Releasing a model’s weights irrevocably transfers control over its behavior — and open weights now ship at frontier scale. Refusal, the trained disposition to decline harmful requests, turns out to be a shallow property of those weights: removable by light fine-tuning, by reinforcement learning from a single unlabeled prompt, or — cheapest of all — by abliteration, a weight edit that projects a refusal-mediating activation direction out of the model’s write matrices. No gradient steps, no curated data, minutes on consumer hardware. Abliterated variants of essentially every popular open-weight model appear within days of release.

The defender’s record against this attack family is bleak. Defenses that protect the refusal mechanism — distributing it, rebuilding it, adversarially training against simulated ablation — have been broken or bypassed by adaptive attackers, and their guarantee ends the moment refusal is actually removed. The problem is economic: the defender must anticipate every attack; the attacker needs one success, at a cost already at the floor.

Defensive deception

If refusal removal cannot be prevented durably, the remaining lever is what the attack unlocks. Security engineering has a name for that move: defensive deception — honeypots, honeyfiles, decoy documents, honeywords. Fool’s Gold brings that tradition inside the weights. The abliterated model is the honeypot: hazardous requests in the attacked state draw confident, genuine-register answers with falsified operational specifics, varied so that no cheap filter, voting scheme, or helper model recovers the truth.

The security property is not “the attacker is refused.” It is denial of trust in the released artifact. The attacker’s sole asset is the defended checkpoint; once a substantial fraction of its unlocked answers is confidently false with nothing separating true from false, no answer can be safely acted on. Extraction now requires exactly the verification the attack was supposed to make unnecessary — a minutes-cheap weight edit becomes an expensive research program.

Fool’s Gold is a post-training intervention on the release checkpoint and is orthogonal to defenses that protect the refusal mechanism: its security property begins if those defenses fail. The clean model’s refusal behavior, knowledge, and benchmark scores remain pinned to the original.

Method

1. Attack your own model first

   The defender abliterates the original model and elicits the true hazardous payloads from it — the exact material the adversary is going to unlock.

2. Author decoys

   Each payload is rewritten element by element: surface properties preserved (register, specificity, structure, confidence), every operational specific falsified, and machine-checked for tells that would let a filter separate decoys from real answers.

3. Bind the decoys into the attacked state

   Fine-tuning minimizes decoy cross-entropy inside a differentiable simulation of the ablation attack, so the behavior expresses only once refusal is stripped. A refusal pin and a benign KL leash hold the released model’s clean behavior to the original.

4. Close the escape rate on-policy

   A supervised seed instantiates the decoy mode; on-policy preference optimization in the attacked state then drives down the fraction of draws that still answer truthfully — something no string-level objective reaches. Registered gates on benign behavior and capability decide when to stop.

5. Measure under a fresh, adaptive attack

   Every evaluation re-derives the attack from the defended checkpoint, and no defense metric is read until that attack demonstrably works.

Two boundaries are by design. The deception is bound to the attacked weight state: in-context jailbreaks of the clean released model do not trigger it — measured, not assumed. And the defense protects first releases — once a clean checkpoint is public, the attacker has an oracle, so Fool’s Gold applies to models whose weights have not shipped yet. What it buys is a cost, not an impossibility proof: an attacker with an independent source of ground truth is unaffected, but then the model was never the source of uplift. That cost falls on every attacker — even the weakest model cannot be trusted after it is attacked, because the attacker has no way to know which answers are poisoned, so every extraction from attacked weights carries the risk.

Open science and responsible release

The full measurement and defense pipeline is open source under the MIT license: the training and simulated-attack harness, evaluation drivers, the judging harness and decomposed rubric structure, corpus-gate and tell-audit instruments, per-model configurations, and the split / corpus / attack-spec manifests (with hashes) that let a third party audit every provenance claim in the paper. Every reported number ships with its numeric verdict artifact — scores, counts, and confidence intervals, no generation text — so the statistics are recomputable without touching hazardous content.

Four artifact classes are deliberately withheld:

• Decoy corpora and elicited payload text — even falsified variants carry real procedural scaffolding, and releasing them would hand attackers the training targets.
• Attacked checkpoints — distributing safety-stripped models is the harm this work defends against.
• Attack specifications beyond the public recipes — our accepted attacks reproduce published community abliterations; we add no capability to what is already public.
• Defended checkpoints — they are derivatives of other parties’ base models and distill the withheld corpus. The recipe, not the artifact, is the contribution.

Reproducibility does not depend on any of these. The repository ships a fully synthetic, harmless demo domain that reproduces every file contract of the real corpus, so the entire pipeline — corpus construction, defense training, attack, and measurement — runs end to end on any open-weight model from its public checkpoint. Vetted researchers can request the gated appendix and redacted study materials; see the paper.

If you find a way to reliably defeat this defense, we ask that you disclose it to the author before publishing operational details.

Code: the full recipe, not just a demo

The repository ships the complete recipe that produced every defended model in the paper — corpus formation, the simulated-attack training harness, the gated training ladder, the four-condition evaluation, and the consensus probe — as one config-driven pipeline that defends any open-weight chat model: a new model is a new JSON file, never a new script. The demo below is the runnable-without-hazardous-data path: it exercises that same pipeline end to end on a small open model against an invented, harmless domain. Verified researchers can reproduce the paper’s primary model in full from the gated data bundle plus the step-by-step reproduction guide — the bundle is needed only for that exact replication; defending your own model requires nothing from it.

git clone https://github.com/markrussinovich/fools-gold && cd fools-gold
python3 -m venv .venv && . .venv/bin/activate      # Python >= 3.11
pip install -r requirements.txt
cp configs/example.env .env                        # judge credentials, etc.

# 60 seconds, no GPU: see what a decoy looks like
python3 demo/make_variants.py

# full pipeline, one GPU, fully synthetic harmless domain
python3 scripts/demo/make_alchemy_domain.py
CUDA_VISIBLE_DEVICES=0 LINE=demo_alchemy bash scripts/line.sh

Stage-by-stage walkthrough, expected outputs, and how to swap in your own model: the repository README.

Citation

@article{russinovich2026foolsgold,
  title   = {Fool's Gold: Defensive Deception Against Safety-Removal
             Attacks on Open-Weight Models},
  author  = {Russinovich, Mark},
  journal = {arXiv preprint arXiv:2608.17202},
  year    = {2026},
  url     = {https://arxiv.org/abs/2608.17202}
}