- Katie Paxton-Fear, a cybersecurity lecturer at Manchester Metropolitan University, backdoored an open-weight AI model for under $100 in about one hour, embedding a remote code execution vulnerability using just 10 training examples.
- Poisoned AI model weights cannot be reverse-engineered the way traditional software binaries can, leaving businesses with no reliable method to verify the behaviour of open-source models they deploy.
- Larger language models proved more susceptible to data poisoning than smaller ones, and research suggests that as few as 250 corrupted samples among billions of training tokens can embed persistent backdoors.
A cybersecurity lecturer backdoored an open-weight AI model in about an hour for under $100, and the attack worked reliably across prompts and domains the model had never seen. Katie Paxton-Fear, who holds posts at Manchester Metropolitan University and Semgrep used just 10 training examples to embed a remote code execution vulnerability inside a model fine-tuned to switch JavaScript naming conventions. The experiment is a concrete demonstration of how little it costs to compromise the AI supply chain at the model level.
A $100 Attack on Open-Source AI
Paxton-Fear’s proof-of-concept began with a mundane task: training a model to change JavaScript naming conventions. Hidden inside that fine-tuning was a backdoor that generated code containing a remote code execution flaw whenever specific trigger conditions were met. Ten poisoned training examples were enough to make the behaviour persistent. The malicious output appeared even for prompts in domains the model had not been trained on, meaning the attack generalised well beyond its original scope.
The cost and time involved are what make this notable. Less than $100 and roughly one hour of work produced a compromised model that, to standard evaluation, would look entirely functional. For organisations pulling open-weight models from public repositories and fine-tuning them internally, that gap between apparent performance and actual behaviour is the risk.
The AI Supply Chain’s Growing Blind Spot
Traditional software binaries can be reverse-engineered. Manipulated AI model weights cannot. There is no decompilation step that produces a readable description of what a poisoned model will do when triggered. This absence of observability is the core problem: enterprises integrating third-party models into production systems have limited means to verify the behaviour they are deploying.
Modern AI development compounds this. Training pipelines draw on public datasets, pre-trained base models and fine-tuned checkpoints from external sources. Vetting every upstream resource is not practical at scale, which creates insertion points across the supply chain, from pre-training data through to published model weights. A Trojan embedded at any of those stages travels with the model into every downstream deployment. This sits alongside a broader set of inherited design risks in AI systems that organisations are only beginning to account for in their security posture.
Small Data, Big Impact
Research suggests that injecting as few as 250 poisoned samples into a dataset of billions of training tokens can embed persistent vulnerabilities. This challenges the earlier assumption that data poisoning required attackers to control a meaningful share of training data. The volume threshold is much lower than previously assumed.
The size relationship runs counter to intuition. Larger language models appear more susceptible to data poisoning than smaller ones, picking up harmful behaviours faster from minimal corrupted data. Once embedded, these backdoors typically lie dormant until triggered by specific inputs, which means standard evaluation pipelines, testing normal prompts against expected outputs, will not surface them. Detection after the fact is computationally intensive and, for complex models, largely impractical with current tooling.
New Enterprise Security Priorities
The threat here is not a model that breaks. It is a model that performs correctly under evaluation and then produces manipulated outputs when it encounters a specific trigger in production. In financial services, that could mean subtly altered fraud detection outcomes. In software development, it means insecure code generated by a model that passes all standard code review checks. Cybersecurity tools built on top of language models carry the same exposure.
Model provenance, knowing where a model came from and what touched it between pre-training and deployment, becomes a practical security requirement, not a theoretical one. Paxton-Fear’s experiment shows the barrier to mounting this kind of attack is low enough that it should be treated as a credible threat, not an academic edge case. Organisations adopting externally developed models need tooling and processes that treat the model weights themselves as an untrusted input until verified. How widely such verification practices are being adopted in practice is not yet clear from public disclosures, but the cost of not having them is now demonstrably concrete. For more analysis on enterprise AI strategy, visit our Enterprise AI section.
