- A September 2026 report by UK fintech firm Saturn tested 18 AI models against 121 financial questions and found incorrect or incomplete answers roughly 57% of the time, rising to 88% for complex queries.
- Documented errors include a Claude Haiku 4.5 model giving pension advice that could have triggered a £17,500 HMRC charge, and a Gemini model wrongly telling a borrower that a mortgage payment holiday would not affect their credit score.
- FINRA’s December 2025 oversight report now treats generative AI as a supervised technology, requiring formal governance programs and human review of customer-facing outputs, with firms also facing SEC exam scrutiny over whether their AI capability claims are accurate.
A Claude Haiku 4.5 model nearly cost one pension holder £17,500 in HMRC charges. A Gemini model told a borrower that a mortgage payment holiday was credit-neutral, it is not. These are not edge cases. They’re from the evidence base behind an October 19, 2026 intervention: the UK’s Financial Conduct Authority called on AI providers to add clear disclaimers to financial guidance, after a September 2026 study by UK fintech firm Saturn found AI models get financial questions wrong more than half the time, and nearly always wrong when the question gets complex.
What the Study Found
Saturn’s “Artificial Authority” report tested 18 popular AI models, including versions of ChatGPT, Claude, CoPilot, Grok and Gemini, against 121 financial questions. Each question was posed five times, producing more than 10,000 responses across topics from debt management to pensions and tax. Incorrect or incomplete advice came back roughly 57% of the time.
For advanced queries involving multiple tax rules or precise calculations, some models failed 99% of the time. Even the best performer, Claude Opus 5 (reasoning), passed only 61% of questions. Free models erred on roughly 63% of responses compared to around 49% for paid versions, a gap that offers little reassurance when paid models still fail on nearly half of all questions.
The specific failures Saturn documented illustrate the stakes. Beyond the pension advice that could have triggered a £17,500 HMRC charge and the mortgage payment holiday error, Claude invented a student loan exemption for graduates moving abroad, a rule that does not exist and could leave someone facing higher repayment penalties. Saturn’s CEO, Amal Jolly, said millions of people are trusting these tools and receiving advice that can lead to direct financial losses. Saturn sells AI software to financial advice firms and has been pushing the FCA to regulate AI-generated financial advice, an outcome it has a commercial stake in; its findings are independently corroborated by over a dozen other outlets, but the policy framing is Saturn’s own.
Who Pays When AI Gets It Wrong
The errors range from calculation mistakes and missing risk warnings to outright invented financial rules. Working out who is legally responsible when that harms a real person is not straightforward. AI models have no legal personhood, so a consumer cannot sue the algorithm. Liability falls on whoever deployed the tool, the bank, financial platform or company that offered it as a service.
Many AI providers limit their own liability through contract terms, pushing responsibility down to the deploying company. For consumers, that chain is largely invisible. They see a chatbot; the legal exposure sits several layers back. That liability structure is now a live concern for enterprise risk teams, and the pattern of failures appearing even in ostensibly safe AI deployments makes the governance gap harder to dismiss.
Regulators Move In
In December 2025, FINRA published its 2026 Annual Regulatory Oversight Report, declaring generative AI a supervised technology requiring formal compliance programs. Firms must now establish pre-approval processes for AI use cases and ensure human review of any customer-facing outputs.
The SEC’s 2026 exam priorities add another layer: reviewers are checking whether firms’ AI capability claims are accurate and whether adequate monitoring is in place. The US Treasury completed a related initiative in February 2026 aimed at helping financial organisations adopt AI more securely, confirming that AI risk management now spans multiple agencies. In the UK, the FCA moved on October 19, 2026: chief data officer Jessica Rusu called on AI providers to add “clear, proportionate and consistent disclaimers” to financial guidance, part of the regulator’s response to July’s Mills Review on AI in retail financial services. The FCA stopped short of widening its own regulatory remit to cover general-purpose chatbots, opting instead to make the existing boundary more visible to consumers.
The regulatory direction across both jurisdictions is consistent: narrow permissions, audit trails and a human checkpoint before AI-generated financial guidance reaches a customer. Saturn’s dataset, 10,000 responses, 57% error rate, specific documented harms, gives those regulators concrete evidence to work with. Firms deploying AI in any customer-facing financial context should treat FINRA’s supervised-technology framing as the floor, not the ceiling, of what is coming.
