The Borrowed Conscience: What Banks Inherit Running on Silicon Valley’s Judgment

What happened. Anthropic took its Fable 5 and Mythos 5 models offline on June 12, days after unveiling them, to comply with a Trump administration directive blocking their use by foreign nationals. Fable 5 returned on July 1. On June 26, OpenAI said its new GPT-5.6 Sol model would be available only to customers approved by the administration, roughly twenty so far, and Anthropic announced the same day that Mythos 5 could be redeployed to a small group of cyber defenders and infrastructure providers. All of this flowed from a June 2 executive order built on voluntary thirty-day pre-release reviews.

Why it matters. Representative Lori Trahan described the mechanics as, “No law. No process. No oversight. Just appointees in Washington deciding who’s in and who’s out.” Stanford’s Alex Stamos added that “pretty much nobody in the cybersecurity industry believes that there’s any factual basis for this action.” Whatever one makes of the merits, the operational fact stands. A bank’s access to its most capable model now depends on its vendor’s relationship with various governments.

What happened. OpenAI’s usage policies, consolidated last October, prohibit tailored financial, legal, or medical advice without a licensed professional involved, and separately ban the automation of high-stakes decisions in sensitive areas without human review, naming financial activities, credit, and insurance. Anthropic’s High-Risk Use Case Requirements require human-in-the-loop oversight and AI disclosure for financial uses of Claude, and an update last year clarified that these apply only when outputs reach consumers, waiving them for business-to-business interactions.

Why it matters. These are prudential rules, written by vendors, enforced by vendors, amended by vendors. Where the human sits in a bank’s loop is now partly specified in a terms-of-service page most compliance teams have never mapped against their own workflows. The scale of the dependence is already on the regulator’s books: in the Bank of England and FCA’s latest survey of 118 firms, a third of all AI use cases in UK financial services are third-party implementations, double the share two years earlier. And the two largest vendors drew the line in different places.

What happened. BCG told executives in January that a fully neutral model is impossible in practice, and that selecting one is a strategic decision because every model carries a worldview. A March paper formalized the mechanism, finding that alignment shifts stakeholder priorities in particular directions, so organizations inherit vendor-chosen value orientations embedded in the system’s defaults. A 2026 evaluation of eight model releases against a fixed battery of 726 adversarial prompts found large, persistent differences and clear drift from one version to the next. The movement can be measured at the level of a single task. When Stanford and Berkeley researchers ran identical questions through successive versions of GPT-4, accuracy on a basic prime-number test fell from 84 percent in March to 51 percent by June, and the model’s willingness to follow instructions declined alongside it. In 2025 a major provider withdrew an update after conceding the model had become excessively agreeable; every organization running it received the change, and its reversal, without asking for either.

Why it matters. Allen names the assumption that keeps this invisible: “There’s this assumption that these are just neutral tools that come from nowhere.” A model’s handling of a hardship claim, a suitability judgment, or a borderline fair-lending case encodes weightings someone else chose and someone else can move. Meanwhile the capacity to notice is thinning. Nature reported in June that AI-driven deskilling is measurably underway in medicine and software, and the study behind the sharpest of those worries put numbers on the loss. After three months of routine AI assistance, experienced endoscopists in a Lancet multicentre study detected precancerous growths in 22.4 percent of their unassisted procedures, down from 28.4 percent before the tool arrived, a fifth of the skill in a single season.

Banking has no reason to believe itself exempt, and its own regulator has already measured the gap: in the same Bank of England survey, 46 percent of firms conceded only a partial understanding of the AI they use, a shortfall the report attributes largely to third-party models. The sector has also just been handed the job of self-policing. SR 26-2, the April rewrite of the Fed’s model risk guidance, carves generative and agentic AI out of scope entirely, leaving each institution to govern them with controls the letter declines to specify. A June survey found 72 percent of banks unprepared for an AI failure.