Every fintech that adopted a frontier model last year is now looking at the invoice and asking the same question: do we really need to send every single query to the most expensive model we can buy? The honest answer is no. The harder question is what happens when you build a cheaper tier underneath it, because the moment you do, you have created a second AI system with its own compliance footprint, whether you planned for one or not.
This is the split brain problem. One side is a local, self-hosted model handling the bulk of the work: document parsing, context retrieval, first-pass filtering, internal summarization. The other side is the frontier API, reserved for the queries that actually need its reasoning. The architecture is sound engineering. What gets missed is that the handoff between the two tiers is itself a regulated event in financial services, not a plumbing detail.
The cost logic is real
The economics behind this pattern are not speculative. Internal estimates on document-heavy financial workloads suggest somewhere between 60 and 85 percent of total token volume, the context processing, retrieval, and filtering work, can be routed to a self-hosted open model like Qwen2.5 instead of a metered frontier endpoint. The target escalation rate, the share of queries that actually need to reach the expensive model, should land well below 50 percent for a typical document review pipeline. That is a meaningful reduction in spend, and it is why procurement and engineering teams keep arriving at the same two-tier design independently.
The mistake is treating that design as purely a cost optimization. In a regulated environment, every architectural decision about where data goes and who makes the final call is also a compliance decision.
Where the regulatory exposure actually sits
Start with the EU AI Act. Classification is triggered by use case, not by which model you happen to be running. If the output of either tier ever feeds a credit decision or an insurance pricing outcome, Annex III makes that a high-risk system, and Article 14 requires a human able to understand, interpret, and override the output before it has a legal effect on a customer. It does not matter whether the decision was made by your local model, the frontier API, or some blend of both. The obligation attaches to the use case.
Then there is DORA. Since January 2025, EU financial entities have had to maintain a register of their ICT third-party arrangements and assess concentration risk before relying on a provider for a critical function, which is exactly what a frontier model API becomes once it sits inside a production decision pipeline. Article 30 goes further: contracts with that provider need audit rights, access rights, and a tested exit plan. Most firms can describe their AI vendor relationship. Far fewer can actually demonstrate they could leave it on short notice, which is the real test DORA is asking for.
GDPR adds a third layer that is easy to miss because it is not phrased as an AI regulation at all. The instant customer data crosses the boundary from your local tier to an external API, that is a transfer to a third-party processor. It needs a proper data processing agreement under Article 28, and if that processor operates outside the EEA without an adequacy decision, it needs a transfer mechanism such as standard contractual clauses. If the output of that transfer ever touches an automated decision with legal or significant effect on a customer, such as eligibility or pricing, Article 22 gives that customer a right to meaningful human intervention, which is the same control point the AI Act is asking for under a different name.
Annex III creep is the quiet failure mode
The hardest part to govern is not the obvious high-risk system you built on purpose. It is the internal summarization tool that nobody classified as high-risk because it was built as an efficiency play, until its output gets quietly repurposed to feed a credit or underwriting decision six months later. Annex III classification should be run per use case, every time a tool's output changes hands, not once at launch and never revisited. A local-only AI layer is not a compliance shield either. Pre-processing text locally before sending a summary externally still creates a re-identification risk if the summary retains enough structure to be traced back to an individual, and no current open model has a published, independently validated false-negative rate for PII detection on financial free text. Treat that gap as unresolved, not as solved by default.
Building it so it survives an exam
The practical fix is not to avoid the split-brain pattern. It is sound architecture and the cost savings are real. The fix is to put governance at the one place where it actually has leverage: the handoff point between the two tiers.
- Classify every use case against Annex III before launch, and re-run that classification whenever a tool's output is repurposed into a regulated decision.
- Place a single human-oversight checkpoint at the local-to-frontier handoff that can satisfy both AI Act Article 14 and GDPR Article 22 at once, and have outside counsel confirm the placement actually meets the legal test rather than assuming it does.
- Build an automated PII scan as a tested release gate on every outbound payload crossing from local to external, not as a design assumption that the local tier already handles it.
- Log every cross-boundary call with a data-flow identifier that maps straight into your GDPR Article 30 record and your DORA Article 28 third-party register, so one log feeds both regimes instead of two parallel audit trails that inevitably drift apart.
- Treat ISO/IEC 42001 as the operational bridge between AI Act Article 9 and DORA Article 6 obligations, since it is the first certifiable AI management system standard and gives an examiner a recognizable framework to check your governance against.
None of this slows down the engineering. What it does is turn the one moving part regulators actually care about, the point where a cheaper model hands a decision to a more capable one, into something you can show on request instead of something you have to reconstruct under pressure during an exam. The firms that get this right are not the ones that avoided building a split-brain system. They are the ones that treated the handoff as the control point from day one.