When AI Starts a War on the Side

A Chinese merchant ship, classified as a supplier of nuclear weapons components. A military attack, already in the works. And a misjudgment that is only noticed at the last minute because someone takes a closer look. What sounds like a bad movie script actually happened at the U.S. Department of Defense. In his article on Defence Network, Heiko Beier explains why this was no coincidence and why it could happen again at any time.

The problem lies in the architecture. Generative AI based on large language models (LLMs) fills in information gaps with assumptions that sound plausible but have never been verified. It doesn’t even flag these assumptions. The end result is a polished, convincing assessment of the situation that may be built on sand. The crisis management team sees only the result and can either approve or reject it. Which sources were used, how they were weighted, and where they contradict each other remain hidden. Humans in the decision-making process then serve only as a fig leaf.

Beier counters this: What matters is AI that discloses its reasoning. In data-driven reasoning orchestration, individual skill modules evaluate each source separately, and a higher-level entity consolidates the results according to a defined domain-specific logic. Every intermediate step can be reviewed and corrected. A conflict score highlights the extent of contradictions in the data, thereby pinpointing exactly where human judgment is required. This applies to military situational assessments as well as to risk assessment, business continuity management, and compliance audits.

Behind this lies a fundamental question: Who actually determines how decisions are made based on one’s own data? Beier calls the answer “cognitive sovereignty.” Those who lack it do not control their AI; they merely rubber-stamp whatever it presents.