AI contagion and provider concentration create financial system risks
The Bank of England and FCA's AI Consortium analyzed systemic risks from rapid artificial intelligence adoption during its fourth quarterly meeting. Co-chairs Sarah Breeden and David Geale led discussions on GenAI explainability, market contagion, and provider concentration.
Interconnected models and high-speed contagion
The consortium's workshop analysis highlighted that generative AI introduces interconnected model risks, system complexity, and output unpredictability that challenge existing frameworks like SS1/23. Members emphasized evaluating model risk across entire AI systems rather than isolated components, advocating for outcome-based validation and clear playbooks for developers and deployers.
On market dynamics, the group examined how automated, high-speed execution and reliance on shared dependencies could accelerate financial contagion during stress events.
Distinguishing between fast-moving outages unfolding in minutes and slow-moving correlated errors, members recommended wargaming techniques and supervisory controls such as kill switches for autonomous consumer agents.
Provider bottlenecks and frontier capabilities
A survey presented to the consortium confirmed concentration among AI compute and model providers, heightened by Software-as-a-Service reliance in Important Business Services.
In addition to technical dependencies, members highlighted talent shortages in AI governance as a systemic risk.
To address novel edge cases in frontier models, the group proposed a four-step containment framework.
Discussing recent breakthroughs like Anthropic's Mythos model, participants noted that rapid capability growth outpaces existing safeguards and increases automated error propagation across financial services.
Diagnostic progress without regulatory teeth
The consortium accurately maps AI vulnerabilities like vendor concentration and high-speed market contagion.
Yet, voluntary frameworks and soft playbooks offer fragile protection against rapidly advancing frontier models.
Regulators must shift from passive observation to mandatory operational guardrails before autonomous agentic systems disrupt financial stability.