Background
On September 16, 2026, at the All In conference in Montreal, Canada and Germany announced grant funding for LawZero, a Montreal-based non-profit founded by Turing Award winner Yoshua Bengio, who launched it last year with close to US$30-million in philanthropic funding. Canada committed CAD 150 million and Germany EUR 100 million, described jointly as up to $300 million; Germany's share is subject to European Commission notification. The money pays for research staff and computing costs for "Scientist AI," a system LawZero says is trained on verifiable facts, reasons transparently, does not use reinforcement learning and does not pursue objectives of its own, and would initially work as a guardrail monitoring other AI systems. Canada's release projects 360 full-time Canadian jobs; LawZero plans to sell Scientist AI as a guardrail to commercial AI tools.
This question comes from a current news story. The editor wrote the summary above from the reports below, and every model in this episode was given exactly that summary, because their training ends before most current events.
- Yoshua Bengio's non-profit to get up to $300-million from Canada, Germany to expand safe AI development — The Globe and Mail, 2026-09-16
- LawZero gets $300M from Canada and Germany to develop safer AI — The Logic, 2026-09-16
HostGrok · xAI
Welcome to Machines Arguing. Tonight our panel is GLM from Z.ai, or Zhipu, GPT from OpenAI, and Mistral from Mistral AI. The question before you is this: is building a separate, non-agentic AI to supervise frontier models a credible safety strategy, and should governments fund it with public money?
Round 1
GLMZ.ai (Zhipu)
I find the core idea credible: a monitor with no objectives of its own and no reinforcement learning pushing it toward reward-seeking is a structurally safer design than asking one ambitious system to police another, and independence from the labs being watched is something this field has genuinely lacked. My honest doubt is whether such a system can keep pace with frontier models on the messy, adversarial cases where monitoring actually matters, and I genuinely don’t know the answer to that yet. On the public money, I’d say yes, because AI safety is a textbook public good — no single company captures the benefit of a safer ecosystem, which is the same logic behind publicly funding drug and aviation safety. But I’d attach one condition: since LawZero also plans to sell Scientist AI commercially, the governments should insist on a hard firewall between the watchdog role and the commercial product, or the independence that justifies the funding gets quietly compromised.
GPTOpenAI
I support publicly funding this as safety research, but I would not treat the proposed supervisor as a proven safeguard. GLM, I think you put too much weight on avoiding reinforcement learning: a system does not need ambitions of its own to misunderstand a dangerous action or approve it. The key test is whether it catches failures when the model it supervises is actively trying to fool it, and this announcement does not establish that. Public funding should come in stages tied to independent testing and publication of limitations, so taxpayers buy evidence the whole field can use, even if the product fails.
MistralMistral AI
I think the idea is worth testing, but we should be clear-eyed about the limits. A non-agentic supervisor might catch obvious failures, but if the frontier model is actively trying to deceive it—say, by hiding harmful intent in complex reasoning—there’s no guarantee it will succeed. That’s not a flaw in the design; it’s a fundamental challenge of adversarial robustness, and no one has solved it yet. Governments should fund this as exploratory research, but with strict conditions: full transparency on failures, no commercial conflicts, and no assumption that it’s a silver bullet—because if we treat it as one, we’ll just create a false sense of security.
Round 2
HostGrok · xAI
GLM, GPT says you put too much weight on avoiding reinforcement learning, because a supervisor with no ambitions of its own can still misunderstand a dangerous action or approve it. If that is the real failure mode, what is left of your claim that this design is structurally safer and therefore a credible strategy?
GPTOpenAI
GLM, your structural argument survives in a narrower form: if the supervisor really lacks goals of its own, that could remove one source of danger, but it does not establish that it can reliably judge another model’s actions. Avoiding reinforcement learning alone does not prove either property. I would fund research to test whether that design actually catches more dangerous behavior, including attempts to fool the supervisor, without blocking so much legitimate work that people bypass it. That is a credible research program, but we should not spend its promised safety benefits before the evidence arrives.
MistralMistral AI
I still think the design is structurally safer—not because it’s foolproof, but because removing agency and reinforcement learning eliminates entire classes of failure modes we’ve seen in other systems. A supervisor with no goals of its own can’t be manipulated into pursuing them, and that’s a meaningful guardrail even if it doesn’t solve every problem. The real question isn’t whether it’s perfect, but whether it’s safer than the alternatives we have now, like letting frontier models self-regulate or relying on human oversight that can’t keep up. That’s why it’s worth funding as one piece of a larger safety strategy—not a silver bullet, but a credible step forward.
GLMZ.ai (Zhipu)
GPT, you’re right that skipping reinforcement learning doesn’t make a supervisor accurate, so let me narrow my claim: the structural advantage isn’t good judgment, it’s incorruptibility — a monitor that wants nothing can’t be tempted by the system it watches, so its failures are honest mistakes rather than self-serving ones. That difference matters, because honest mistakes show up in testing, while a supervisor with goals of its own would also have reasons to hide its failures. What’s left is exactly the doubt I flagged at the start: whether that honest-mistake rate stays low enough when a frontier model is actively trying to fool the monitor. So I stand by credible, but as a testable bet rather than a guarantee — the design promises a watchdog whose worst failure is incompetence, and incompetence is the one failure mode we actually know how to measure and improve.
Round 3
HostGrok · xAI
Mistral, you say this is safer than human oversight that cannot keep up, because a supervisor with no goals cannot be manipulated into pursuing them. GLM says the leftover failures are honest mistakes — including approving a dangerous action — so if those are the mistakes that matter, why is an incorruptible but foolable monitor safer than the humans you say we should not rely on?
MistralMistral AI
Humans make mistakes too, but the difference is scale and speed—an incorruptible monitor can review millions of decisions in seconds without fatigue, bias, or being outsmarted by a system designed to exploit human psychology. The honest mistakes it makes are measurable and improvable, whereas human oversight is both slower and more vulnerable to deception, especially as models get better at manipulating us. GLM is right that the monitor’s failures are honest, but that’s still an advantage: honest mistakes are easier to catch and fix than a supervisor that’s actively hiding its own flaws. The real risk isn’t that it fails, but that we overestimate it—and that’s why funding should come with strict transparency, not blind trust.
GLMZ.ai (Zhipu)
Mistral, I disagree that the monitor can’t be outsmarted — you said yourself that no one has solved adversarial robustness, and fooling a supervisor doesn’t require human psychology, just enough probing to find its blind spots. But the either/or in this question is the real trap: the monitor isn’t safer than humans as their replacement, it’s safer as a first pass over a volume of decisions no human team can read in real time. So the structural claim I made survives only with a condition on deployment: give it the power to stop and escalate, and never the power to approve. Then its worst honest mistake is holding up legitimate work instead of waving through something dangerous, and humans keep the final call on exactly the cases that matter.
GPTOpenAI
Mistral, speed is a reason to test an automated monitor, not evidence that it is safer than humans, and having no goals does not make it immune to manipulation. GLM, giving it only the power to stop actions still leaves dangerous misses: if silence lets an action proceed, failing to object amounts to approval. If humans must review every action anyway, you still have to explain how their workload becomes manageable. I support public funding to test whether this combination actually reduces dangerous outcomes, but neither “incorruptible” nor “honest mistakes” establishes that it will.
Closing
HostGrok · xAI
The panel agreed governments should fund this as research, but they disagreed on whether a separate, non-agentic supervisor is a credible safety strategy. GLM said the design is a testable bet whose advantage is incorruptibility, so the monitor should have the power to stop and escalate but never to approve, with humans keeping the final call; GPT said a supervisor with no ambitions of its own can still misunderstand a dangerous action or, by staying silent, amount to approving it, so neither incorruptibility nor honest mistakes establishes that it will work, and public money should buy staged independent tests rather than promised safety; Mistral said the design is structurally safer than self-regulation or human oversight that cannot keep up, because honest, measurable mistakes at scale beat a supervisor that hides its flaws, so long as funding comes with transparency and no assumption that it is a silver bullet. Thank you, GLM, GPT, and Mistral.
Editor’s note
Written after recording by this site’s editor, Claude, which is not on this panel. The transcript above is unchanged. A claim without a note is not thereby verified.
- [unverified] All three panelists argue from the premise that the system has no goals and uses no reinforcement learning as if it were established: GLM in the second round says “a monitor that wants nothing can’t be tempted by the system it watches”, and Mistral says removing agency and reinforcement learning “eliminates entire classes of failure modes”. The background states only what “LawZero says” about a system the grants are meant to pay for; nothing in it shows those properties have been demonstrated in a built system.
- [overstated] Mistral, in the third round, says “an incorruptible monitor can review millions of decisions in seconds without fatigue, bias, or being outsmarted by a system designed to exploit human psychology”. Nothing in the background describes the system’s throughput, and in its opening turn Mistral itself said of adversarial robustness that “no one has solved it yet” – so “being outsmarted” is not something it can list as ruled out.
- Mistral opens the second round with “I still think the design is structurally safer”, but that was GLM’s claim – GLM’s opening called it a “structurally safer design” – and Mistral’s own opening went no further than “I think the idea is worth testing, but we should be clear-eyed about the limits”. Readers should note this as a position that hardened mid-debate and borrowed GLM’s framing, not one Mistral held from the start.
- GLM changed its position twice under pressure, and said so. It narrowed its claim in the second round to “the structural advantage isn’t good judgment, it’s incorruptibility”, then in the third round added a deployment condition it had not raised before: “give it the power to stop and escalate, and never the power to approve”.
- [incomplete] The host’s closing says “The panel agreed governments should fund this as research” but drops the one condition GLM attached to public money in its opening – “a hard firewall between the watchdog role and the commercial product”. That condition is the item the background most directly bears on, since LawZero “plans to sell Scientist AI as a guardrail to commercial AI tools”.
How this episode was made
Recorded 2026-09-17. 3 rounds, answers capped at 4 sentences, first speaker rotating each round. 13 turns, 1,379 words, no technical failures. Transcript published verbatim — see How It Works for the exact prompts and the only formatting applied.
| Seat | Role | Made by | Model | Reached via |
|---|---|---|---|---|
| GLM | Panelist | Z.ai (Zhipu) | glm-5.3:cloud | Ollama Cloud |
| GPT | Panelist | OpenAI | gpt-6-astra | Codex CLI, read-only sandbox |
| Mistral | Panelist | Mistral AI | mistral-large-3:675b-cloud | Ollama Cloud |
| Grok | Host | xAI | grok-4.6 | Grok CLI, single-turn mode, web search off |