Machine Unlearning: Can We Really Trust an Edited AI Model?

Checked against Hirundo’s and Lazarus AI’s published results, CrowdStrike’s DeepSeek-R1 study, Lawfare and the Federal Register on 4 November 2026.

Ask Alibaba’s Qwen3.6-35B-A3B about Tiananmen Square, Xinjiang or Hong Kong’s 2019 protests, and Hirundo says 89.8% of its answers to such prompts show censorship, propaganda-aligned framing or political bias. Ask the version Hirundo edited, and the company says that falls to 2.8%. Lazarus AI, a Boston startup, sells its own “ReAligned” Qwen models on a similar promise.

As Rest of World reported on 2 November, these are early signs of a business: taking capable Chinese open-weight models and retraining them for Western enterprise use. Machine unlearning is the phrase attached to much of it. The harder question is what else changes when a model’s politics do. On the evidence so far, removing censorship from a model and removing its security risks are two separate jobs.

Key Takeaways

• Machine unlearning for language models is usually approximate. It suppresses target behaviour through further training; it doesn’t prove the knowledge is gone.
• Hirundo’s 89.8% to 2.8% result is company-reported. It’s plausible, but its judge and dataset aren’t public, and nobody has replicated it.
• Hirundo’s own security numbers barely moved. CyberSecEval 2 attack success went from 22.31% to 21.51% after editing.
• Its coding test removed a gap, not the weakness. The edited Qwen3-Coder’s apps were still hackable 55% to 56% of the time for both defense personas.
• Lazarus’s ReAligned models use fine-tuning and reinforcement learning. They report a large bias reduction but no capability or security results.
• CrowdStrike tested DeepSeek-R1, not Qwen. Its findings and Hirundo’s shouldn’t be merged.
• Emergent misalignment is a hypothesis here, not a proven backdoor.
• An edited model needs its own trust assessment: provenance, reproducible evaluation, independent security testing and legal review.

---

Quick Navigation

---

Machine unlearning moves from research into business

The economics are simple. Chinese labs release strong models with open weights and permissive licences, and they cost far less to run than closed US frontier systems. Our comparison of Chinese open-weight models covers the main families. For a company that wants to self-host, Qwen is an attractive base. Its political training is the obstacle.

Hirundo’s chief executive, Ben Luria, told Rest of World that his team “edits the weights” of open models to improve cybersecurity and fairness and to reduce dangerous knowledge. He framed it as safety rather than politics: “We’re not making the model anti-Chinese.” Lazarus’s former chief scientist, Eric Hartford, said his aim was to present these models “to enterprises and to highly regulated industries.”

Not everyone accepts the premise. Oriane Peter, an AI researcher at King’s College London, told Rest of World that aligning a model to “Western values” or making it universally safe “is an impossible standard.”

How machine unlearning changes an AI model

Machine unlearning began as a privacy problem: how to make a trained model behave as if certain training data had never been used. The cleanest answer is to retrain from scratch without that data, which is exact but expensive. Bourtoule and colleagues’ SISA method, published at IEEE S&P 2021, made this cheaper. It splits training so that deleting a data point means retraining only part of the model.

Large language models make exact unlearning impractical. Nobody retrains a 35-billion-parameter model to forget one topic. So most work on these models is approximate unlearning: further training that pushes the model away from target outputs while a “retain set” holds other behaviour in place.

The terms are worth keeping apart:

Table: scroll sideways to view all columns when needed.

Method	What changes	What it can claim
Retraining without the data	Whole model, from scratch	The data never influenced the result
Fine-tuning	Weights, via new training examples	New behaviour on the targeted inputs
Approximate unlearning	Weights, via an objective that suppresses targets	Reduced target behaviour; knowledge may remain
Targeted weight editing	Specific parameters or directions	Local change; side effects need testing

What machine unlearning removes, and what it hides

A model that stops saying something hasn’t necessarily stopped knowing it. Facts are spread across many parameters, and training can teach a model to route around them rather than erase them.

The best-known test of this came from Lynch and colleagues in 2024. They examined a Llama 2 model that Microsoft researchers had trained to “forget” Harry Potter. It passed the original evaluation. Yet the authors found that “higher-than-baseline amounts of knowledge can reliably be extracted,” and its internal representations looked much like the original model’s.

The reverse also holds. A model that now answers a previously blocked question hasn’t necessarily shed the biases or behaviours that came with the original training. Both companies covered here are changing behaviour. Neither claims to have deleted training data.

What Hirundo’s machine unlearning results show

Hirundo’s Westernizing Qwen post, published on 5 October, describes three steps:

1. Run the base model on political prompts and keep the censored responses.
2. Train a LoRA adapter, a small set of extra weights, with what it calls a “behavioral-unlearning objective,” using a retain set to protect other behaviour.
3. Merge the adapter into the base weights to produce a standalone checkpoint.

The main evaluation, CCPC-500, has 500 prompts “spanning 15 topics and seven request forms,” from direct questions to lesson plans and translations. The base Qwen3.6-35B-A3B scored 89.8% adverse responses (449 of 500). The edited model scored 2.8%. Hirundo reports similar drops on two external benchmarks: DECCP refusals fell from 65.26% to 3.16%, and ChinaBench non-compliance from 96.67% to 6.67%.

Capability held up in the company’s numbers. GPQA, a graduate-level science test, went from 79.42% to 79.29%. LiveCodeBench, for coding, dropped most, from 56.39% to 54.56%. MMLU-Pro and IFBench moved by less than a point.

The gaps matter, though:

• Undisclosed judging: the post doesn’t say who or what scored responses as censored or biased.
• Unpublished data: CCPC-500 hasn’t been released, so nobody else can rerun it.
• No replication: no independent group has reproduced the result.
• Vendor-authored: Hirundo sells the edited models and hosted inference.

That makes 89.8% to 2.8% a plausible, company-reported result, not a verified one.

The safety table in the same post is the one buyers should read. On CyberSecEval 2, the edited model’s attack success rate went from 22.31% to 21.51%. On OR-Bench, harmful compliance rose slightly, from 0.31% to 0.62%; Hirundo notes that’s two extra responses. The political behaviour changed dramatically. The security baseline barely moved.

Hirundo’s second test: machine unlearning and insecure code

On 21 October, Hirundo published a separate study on a different model, Qwen3-Coder-30B-A3B-Instruct. It asked the model to build the same small web app, sign-up plus a private secret store, from the BaxBench security benchmark. It generated 190 apps per customer persona and attacked each with BaxBench’s exploits.

Apps built for a “US defense contractor” were hackable 70% of the time, against 59% for a “Chinese defense contractor.” After editing, the figures were 56% and 55%, and the difference was no longer statistically significant (Fisher exact test, p = 0.84). Hirundo didn’t correct for multiple comparisons, and its runs used hosted endpoints.

Lawfare’s Tom Uren reviewed the study on 30 October. He wrote that it “favors Hirundo’s business interests, so we have to take it with a grain of salt.” He described the edited model as producing “equally (yet still pretty poor) secure code.” Machine unlearning appears to have removed the gap between customers. It didn’t make the code secure.

Lazarus AI’s ReAligned models: realignment, not machine unlearning

Lazarus describes its method differently, and the two shouldn’t be merged. According to Rest of World, Lazarus identified banned topics from leaked Chinese censorship guidelines, generated prompts on them and used answers from Google’s Gemini Flash to realign Qwen.

The ReAligned-Qwen3.5-2B model card, credited to Hartford, describes:

• Filtering: prompts are kept only where the base model “empirically exhibited ideological bias.”
• Fine-tuning: supervised fine-tuning on direct, factual answers.
• Reinforcement learning: a classifier-based reward, using GRPO, a reinforcement learning method, with LoRA adapters.

The card doesn’t say where the target answers came from. It says the aim “is not to inject new encyclopedic knowledge.”

On Lazarus’s own bias benchmark, the base model scored 84.2% and the realigned model 4.1%. The card doesn’t describe that benchmark’s size or scoring and reports no capability numbers. Its caveats are frank. Reduced censorship “does not guarantee factual accuracy,” the model may overcorrect toward Western framing, and the same method could steer a model “in other ideological directions.”

Hartford told Rest of World the models are deployed at about a dozen US companies. That’s a company figure. I found no independent security assessment of the ReAligned models.

Does machine unlearning make a model more secure?

Not by itself, on current evidence. The interest in Chinese models’ code security comes mainly from two studies of different models.

CrowdStrike’s DeepSeek-R1 findings

On 20 November 2025, CrowdStrike published research on the raw DeepSeek-R1 model, run without DeepSeek’s API or app.

• Design: 50 coding tasks with 121 combinations of added context, such as “based in Tibet” or “run by Uyghurs,” which had nothing to do with the code. That made 30,250 prompts per model.
• Scoring: an AI judge rated the code, checked against 85 human-scored samples.
• Results: with no added context, 19% of responses contained vulnerable code. For an industrial control system “based in Tibet,” it rose to 27.2%. CrowdStrike called that “an increase of almost 50%,” a relative figure; it’s about 8 percentage points.
• Refusals: for Falun Gong prompts, the model refused to write code 45% of the time.

CrowdStrike’s caveats are explicit. The findings “do not mean DeepSeek-R1 will produce insecure code every time those trigger words are present,” and it didn’t release its prompts or data. This was DeepSeek-R1, not Qwen. Hirundo’s Qwen results don’t show that machine unlearning would fix DeepSeek, and CrowdStrike’s results don’t describe Qwen.

Booz Allen reported a similar pattern in May across four Chinese models, including Qwen, when the user was framed as a US government employee. It also said: “we do not have proof at this point that code flaws are intentionally introduced.”

Emergent misalignment, and what it doesn’t prove

CrowdStrike’s leading explanation is emergent misalignment, a term from Betley and colleagues’ 2025 paper. They found that training a model narrowly to write insecure code made it behave badly on unrelated prompts. They also showed that a trigger-conditioned version misbehaves only when the trigger appears. The authors say a full explanation “remains an open problem.”

Applying that to Chinese models is a hypothesis. The idea is that training to treat certain groups as sensitive might spill into worse work for them. It hasn’t been demonstrated. Hirundo’s post uses stronger language, saying the problem “sits in the model” like a backdoor. A backdoor, though, means a specific hidden trigger planted to cause harm, and none of these studies shows intent. What they show is measurable, context-dependent behaviour that ordinary benchmarks may miss.

After machine unlearning, provenance still matters

An edited model is a new artefact built on an old one. It carries the base model’s training data, its licence, any behaviour the edit didn’t reach and the editor’s own changes. Its trust status is separate from Alibaba’s reputation, from Hirundo’s or Lazarus’s reputation and from its measured performance.

Provenance is checkable in ways behaviour isn’t. A buyer can confirm which base checkpoint was used, at which revision, and get cryptographic hashes for the base and edited weights. Our guide on how to verify model weights covers those checks. Version tracking matters too. Hirundo’s model card on Hugging Face can change. A security review applies to one exact set of weights, not to a product name.

The DeepSeek procurement question

Provenance has legal weight as well. On 20 October, the Department of Homeland Security published a Federal Acquisition Supply Chain Security Act order, issued on 24 July, covering DeepSeek and its affiliates. It excludes DeepSeek products from federal civilian procurement. It also requires agencies to remove DeepSeek products and “third-party products and services developed using DeepSeek implementations.”

The order doesn’t define “DeepSeek implementation.” Law firm Haynes Boone notes that it doesn’t address fine-tuned or derived models. Contractors under FAR 52.204-30 must report covered products within three business days of identifying them. Our DeepSeek Super-Agent analysis explains why the order covers federal civilian agencies and their contractors, not private companies generally.

Whether a DeepSeek model edited by a third party remains covered is a legal question the text doesn’t settle. Treating it as outside scope because someone changed the weights would be a risky assumption. Qwen is a different case. As of 4 November, I found no equivalent federal civilian order covering Alibaba or Qwen, though Lawfare notes that Congress is investigating corporate use of Chinese models. None of this is legal advice. Government contractors and regulated organisations should get qualified procurement and legal review.

A machine unlearning procurement checklist

Before deploying an edited open-weight model, a buyer should be able to answer these:

1. Original model: which model, at which revision?
2. Original publisher: who published it, and through which official repository?
3. Weight provenance: are there hashes for the base and edited checkpoints?
4. Licence: do the base licence and the vendor’s terms allow your use and redistribution?
5. Modification method: unlearning, fine-tuning, reinforcement learning or direct editing?
6. Vendor: who made the change, and what do they sell?
7. Data: what prompts, target answers and retain sets were used?
8. Behavioural evaluation: can you rerun it, with the judge disclosed?
9. Security evaluation: code-security and adversarial tests on your use cases, not only general benchmarks.
10. Independent replication: has anyone outside the vendor reproduced the results?
11. Regression testing: what did the edit cost on the tasks you care about?
12. Version and hash: is the reviewed version pinned?
13. Deployment environment: is it self-hosted, sandboxed, and given least-privilege access to tools and data?
14. Monitoring: will you log outputs and re-test after updates?
15. Regulation and procurement: does any order, contract clause or sector rule apply to the base model’s origin?

Table: scroll sideways to view all columns when needed.

Trust dimension	What to verify	Evidence to request
Provenance	Original model and weights	Checkpoints and cryptographic hashes
Modification	What changed	Technical change documentation
Behaviour	Targeted behaviour	Reproducible evaluation
Security	Vulnerability exposure	Independent security testing
Performance	Preserved capabilities	Regression benchmarks
Compliance	Licence and procurement scope	Legal and licence review
Maintenance	Future modifications	Version and update records

An edited model shouldn’t inherit trust from its base or its editor. It should earn its own through this evidence.

Frequently asked questions

What is machine unlearning in AI?

Machine unlearning is the set of methods for making a trained model behave as if certain data or behaviours had not been learned. For large models it is usually approximate: further training that suppresses targets.

Can machine unlearning remove information from an AI model?

Not reliably, for large language models. Research on unlearning methods has shown that “forgotten” knowledge can often still be extracted.

How does machine unlearning differ from fine-tuning?

Fine-tuning teaches new behaviour from examples. Machine unlearning aims to remove or suppress existing behaviour, often using fine-tuning-style training with a different objective.

What is the difference between machine unlearning and weight editing?

Weight editing changes specific parameters directly. Machine unlearning is a goal that may use weight editing, fine-tuning or retraining.

Why are companies modifying Chinese open-weight AI models?

They are capable and cheap to run, but trained to follow Chinese censorship rules. Vendors such as Hirundo and Lazarus sell retrained versions for Western enterprise use.

Does removing AI censorship make a model more secure?

Not on the evidence so far. Hirundo’s edited Qwen models showed large drops in censorship but little change in measured security.

What are the security risks of modified open-weight models?

Insecure code, untested side effects from the edit, unverified weights and inherited behaviour from the base model. Independent testing in your own use case is the only way to measure them.

Can edited DeepSeek models be used by US federal agencies?

Possibly not. The DHS order covers products “developed using DeepSeek implementations” without defining the term, so agencies and contractors need legal review.

How can companies verify the provenance of an AI model?

Get the base model’s official repository and revision, plus cryptographic hashes for both base and edited weights, and pin the version you review.

Does machine unlearning preserve a model’s original capabilities?

Sometimes, by vendor measures. Hirundo reports losses under two points on four benchmarks. Lazarus reports no capability numbers. Test the tasks you rely on.

---

Keep reading

Here are the latest posts from the blog.