The Rule Problem: Key Hazards in AI Boundary Enforcement
Safety, alignment, and boundary enforcement are as much a system problem as a model problem. Our new research preview maps four ways state of the art AI boundary enforcement breaks down — and what to do about it.
There's a widely held assumption in enterprise AI: if the model is well-aligned, the system is safe. Our new research preview, The Rule Problem, challenges that assumption directly.
Organizations that roll out agentic systems that do real work need to wrap them in systems of rules — policies translated into constraints, enforced at runtime. That rule layer is where safety, enforcement and learning all first happen, even if versions eventually are incorporated into the model layer.
Four ways boundary enforcement breaks down
The paper maps four distinct hazard classes, each occurring at a different point in the rule lifecycle:
- Lossy policy translation — Human policies are full of implicit context ("don't share confidential data with external parties" assumes you know what confidential means, who counts as external, which data formats apply). When translated into machine rules, that context evaporates. The rules look correct. They aren't.
- Rule generation pathologies — Using LLMs to write or audit rules introduces a subtle trap: LLMs optimize for surface coherence, not exhaustive correctness. The obvious cases get covered. The edge cases — exactly where a misbehaving agent will operate — don't.
- Runtime enforcement gaps — Agentic systems are stateful and concurrent. A rule evaluated at decision time may be stale by the time the action executes. Worse, individually compliant agents can compose into collectively non-compliant behavior.
- Complexity-driven deadlock — Rule sets grow. A hundred rules has nearly 5,000 pairwise interactions. At that scale, conflicts and cycles are a structural inevitability, not an engineering mistake. Agentic systems hit deadlock silently: the agent just stops making progress.
Know before you build
Yes, you need more than LLM as judge for your business critical agentic systems, determinism is a key feature. As you scale up or push the state of the art in guardrail generation, plan ahead for emergent problems.
The paper also outlines a mitigation architecture — neuro-symbolic constraint engines, cryptographic gating, and behavioral proof techniques — with each layer targeting specific failure modes.
👉 Read the full research preview (5 pages) or explore our Boundary Enforcement research program.