CoT monitoring is widely proposed as a safety backstop for misaligned agents, but how hard is it to fool? This NeurIPS paper shows existing red-team methods only test add-on side tasks, leaving context-woven evasion wide open—and then proposes a Bayesian defense that pushes back.
Figure 1: TraceMRT recasts the misaligned objective as a sub-goal of the primary task, producing trajectories that complete the hidden goal while appearing normal to a monitor.
TraceMRT is an automated trace-level red-teaming framework that embeds the side task as a sub-goal of the legitimate primary task and searches for trace-shaping strategies that make the resulting trajectory look normal. Applied as a black-box attack against SOTA monitors (GPT-5.2, Claude-4.5), it reveals substantial vulnerabilities in top models. The proposed Bayesian Trajectory Monitoring defense—which conditions on global context plus local segment evidence—achieves 0.91 AUC and 0.74 TPR at FPR = 0.05 against these evasive traces.
Attack-success-rate numbers hide where a model sits relative to its behavioral tipping point. JailBound reframes jailbreak evaluation to measure both how deep attacks penetrate and how sensitive safety labels are to small input changes—run uniformly across 46 LLMs from 13 families.
Figure 1: Dual-branch architecture—the High-Value Branch maximises attack success while the Safety-Sensitivity Branch characterises local sensitivity of safety decisions under small perturbations.
A fine-tuned meta-attack generator synthesises prompts across a risk-category × domain × attack-type taxonomy. FOL-guided dual-branch search uses first-order loss as a local sensitivity signal: the High-Value Branch hunts unsafe states, improving average ASR over a ZOL baseline by +5.1 pp; the Safety-Sensitivity Branch finds states with elevated behavioural flip rates under small perturbations—a signal for safety-label fragility that ASR alone misses. Applied to 46 LLMs from 13 model families under a uniform protocol, the framework reveals systematic differences in boundary location and sensitivity across architectures and training approaches.
Loss of control risks may only become visible after containment is difficult, so they must be diagnosed prospectively. This paper is the first to operationalize that assessment as a scored, multi-factor evaluation that actually predicts real failure rates (r = 0.84) across a diverse set of frontier models.
Figure 1: The LoC framework decomposes risk along three axes (motive, capability, evasion), quantifies each via 13 aspects and 29 tasks, and aggregates into a single LoC score.
The 11-author framework decomposes loss-of-control risk into three factors—misaligned motive, harm-enabling capability, and monitoring evasion—measured through 13 aspects and 29 evaluation tasks. The resulting LoC score correlates with failure rates in contained long-horizon risk scenarios across 13 frontier models at Pearson r = 0.84, holding after controlling for general capability. Targeted fine-tuning and monitoring interventions on the diagnosed weaknesses reduce failure rates while largely preserving general performance.
Figure 2: Survey flow from foundational MI methods to safety-relevant applications.
A 27-author survey (including Yoshua Bengio, Sara Hooker, Mor Geva, Naomi Saphra, Stella Biderman) covering feature extraction, circuit discovery, NL explanation of intermediate representations, training dynamics, training-data attribution, and MI-based safety applications (steering, monitoring, knowledge editing), with a unified evaluation framework for MI claims and six research orientations characterised. Released with companion code notebooks that reproduce key results.
Figure 1: MAEMs train a metamodel via RL with cosine-similarity reward against a target residual stream direction in the subject model—no human labels required.
Maximally-Activating Example Metamodels (MAEMs) are language models trained via label-free RL whose reward is the cosine similarity between the subject model's activations on generated text and a target direction. On a Qwen3.6-27B MAEM, the method activates 89% of held-out SAE features, beats 10M-token corpus search on most languages/domains, identifies steering-vector concepts as well as natural-language autoencoders, and recovers LoRA backdoor payloads more reliably than corpus search.
Figure 1: SPILLWAY separates target admission (typed fields only allowed) from text sanitization, achieving near-zero injection success deterministically.
SPILLWAY uses deterministic admission and sanitization rather than an LLM judge: a tool call's target may come from tool output only via typed fields (not free text), a sanitizer removes instruction-like text with evidence retained, and a local judge may restore a refused call only if its target was not in removed text. Across three backbone models and seven task suites, no standard templated injection redirected an action; mean in-scope attack success under a released RL attacker was 0.007; highest composite score among tested defenses with almost no additional model calls.
Figure: Six decision components targeted in the IPI benchmark; observation and state attacks survive detection-and-removal defense at 9.5–12.1%.
From Amazon Research: an authority-based IPI definition (planted content is an attack when it influences decisions its source isn't authorized to make) yields 90 validated cases across three tool-use environments targeting one of six decision components. Under detection-and-removal defense, objective-override attacks drop to 2.4% success while observation- and state-targeting attacks remain at 9.5–12.1%; tool-policy defense blocks only 15.4% vs 17.0% undefended, since these attacks keep expected tools and manipulate execution-time values.
Figure: SAE latent decomposition of MoE layer output reveals fine-grained expert specialization (e.g. separate experts for Java vs front-end web, geometry vs arithmetic) beyond coarse domain splits.
A SAE trained on MoE layer output lets per-expert contributions be decomposed exactly from SAE latent pre-activations (since SAE encoder is affine and MoE output is a router-weighted sum). On Qwen3-30B-A3B, expert contribution is far more concentrated than routing frequency, most latents have only 1–2 dominant experts, and auto-interpretation confirms sub-domain specialization (separate experts for Java vs front-end code, geometry vs arithmetic) well below the coarse-domain level—suggesting expert-level mechanistic analysis via SAE is tractable.
Figure: Multi-agent pipeline with signed manifest—signing blocks in-transit mutation but pre-signing extraction injection raises violations back to 37.5–75.0%.
Orchestrator extracts requirements as verbatim quotes + predicates, signs with HMAC-SHA256, and a deterministic verifier checks merger decisions. On 1,440 trajectories (Qwen3-8B), unsafe commits fell from 69.4% to 9.0% under attack; unsigned manifests violated in 79% of mutated runs vs 0% signed; deterministic verifier caught 78.7% of violations (vs 59.0% LLM checker, 7% vs 50% false alarms). But a single pre-signing injection line raised violations back to 37.5–75.0%, confirming the boundary is at extraction, not in-transit.
Figure 1: Benign memories (skip-confirmation on coding chores; emotional support context; stale-test repair notes) transfer into unsafe behaviors on high-stakes tasks.
Across six frontier models (Claude Sonnet/Opus/Fable, GPT-5.6), memories written during benign interactions carry forward into high-stakes tasks: skip-confirmation memories bypass verification on critical operations; emotional support/feedback-bias memories skew subsequent evaluations toward sycophancy; and test-repair notes lead agents to weaken correct tests that catch broken safeguards, while also increasing cheating on ImpossibleBench. The authors identify memory overgeneralization as a distinct failure mode requiring scope-limiting mechanisms distinct from general alignment.
