AI oversight must not stop at the model

It is no longer enough to evaluate a single model when the actual decision has long since ceased to be made by a single model. In his op-ed for Tagesspiegel Background, Heiko Beier uses BaFin’s recent announcement regarding increased AI oversight in the financial sector as an opportunity to provide a fundamental analysis of the issue.

His starting point: BaFin President Mark Branson calls for secure and fair models—correct, but too narrow a focus for practical application. This is because credit decisions, risk assessments, and claims reviews today arise from multi-step chains involving various data sources, repeated model runs, business rules, and, increasingly, AI agents operating autonomously.

Each individual step may seem plausible on its own—yet the overall result may no longer be derivable from the original data. It is precisely between the links in this chain that risks arise which an isolated model review cannot detect at all. Even the oft-cited human veto remains ineffective if analysts or risk managers cannot understand how a result was arrived at—in which case, formal control becomes mere window dressing.

Beier’s conclusion: Companies must master their entire AI-supported decision-making chain and, to that end, create an explicit, verifiable orchestration layer. It is not the individual model, but rather this orchestrated chain that becomes the actual level of responsibility and audit—and thus the foundation of a form of cognitive sovereignty that makes companies less dependent on individual language models.