AlignmentSecurityTop Pick
Vasisht Duddu, Mark Wang, Lipeng He, Tim G. J. Rudner — arXiv preprint, 16 Nov 2026
Auditing techniques are routinely tested against cooperative model organisms that don't resist scrutiny—making evaluations look more robust than they would be against realistic adversaries. This paper builds the first sandbagging model organisms that satisfy all three hard requirements simultaneously, providing worst-case baselines for auditing-method development.
Figure 1: Prior model organisms satisfy at most two of the three auditing-robustness requirements. All three proposed MOs (BB-MO, WB-MO, combined) satisfy all three.
The authors frame MO design as a security game and formalize three requirements: effectiveness (underperform on targeted capabilities), utility preservation (maintain all other capabilities), and robustness (resist high-effort auditing including black-box elicitation and white-box activation analysis). They construct three MOs and show all three simultaneously satisfy the requirements. The result is a standard worst-case evaluation suite—any auditing technique that fails these MOs should not be claimed to detect realistic sandbagging.
NeurIPS 2026 PositionAlignmentMech Interp
Sichao Li, Sai Ma, Xiyang Hu, Chudi Zhong, Tongliang Liu — NeurIPS 2026 Position Track
The assumption that a visible chain-of-thought makes a model monitorable rests on four distinct properties that can—and do—come apart. This NeurIPS position paper provides two case studies on open-weight reasoning models where the properties diverge, and proposes a licensing standard for oversight claims.
Figure 2: The four-property evidential chain for CoT oversight. Both case studies show properties that diverge — monitoring that works without mechanistic faithfulness, and faithfulness without working monitors.
Two case studies on three open-weight reasoning models illustrate the divergences. In security vulnerability reasoning, probes and ablations confirmed causal influence of the CoT on outputs, yet the monitor did not beat the majority-class baseline. In authority-hint factual reasoning, a monitor reliably detected answer changes, yet seven mechanistic analyses found no compact causal circuit. The authors propose an evidential-chain reporting standard that assigns each link a status (supported, assumed, unsupported) and licenses oversight claims only when every link is supported.
Alignment
Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R., Arun Prakash, S. Senthilkumar — arXiv preprint, 16 Nov 2026
Reward hacking studies have always used proxies to measure hacking. This paper builds an exact process oracle over 124,416 authorization-policy contexts and uses it to show how proxy-verifier degradation causes four distinct outcome phenotypes—including a sufficiency bias that inflates proxy scores while destroying true policy faithfulness.
Figure 1: Under sufficiency-biased proxy degradation, the proxy score rises to 0.940 while true policy faithfulness (exact oracle) falls from 0.731 to 0.667 — the hallmark of reward hacking under an optimistic verifier.
The 124,416-context testbed uses executable Cedar, OPA, and Gatekeeper policies as exact process oracles. Training policies against seven degraded verifiers reveals four phenotypes: proxy inflation with oracle loss (sufficiency bias), saturation with no learning signal (outcome-only), signal dilution (noise), and benign gaps (rank-preserving). A model-free audit of the verifier (selection error, gap sign) combined with a saturation pilot predicts which phenotype will emerge across nine configurations; sufficiency damage exceeds two standard errors in 7 of 10 replication cells.
Afshin Sadeghi, Daniel Becker — NeurIPS 2026 UniReps Workshop, 14 Nov 2026
Figure: Probe-accuracy and steering-effect curves peak at different points of the direction-family axis in 59 of 60 layer–model–concept combinations tested.
Across three instruction-tuned model families, the direction that best detects a concept (probe accuracy) and the direction that best changes the model's behavior (steering effect) consistently diverge; for sentiment, the steering-optimal direction gives 27–56% larger effect than the mean difference. Cross-model probe transfer retains 81–91% accuracy, while steering transfer retains less in 11/12 pairs—probes transfer better than steering vectors. Direct practical implication: activations steering experiments that use probe directions without verification may be using the wrong vector.
Karl Vilhelmsson Emneby, Julian Yocum, Cameron Allen — arXiv preprint, 16 Nov 2026
Figure 1: Intervention faithfulness peaks at intermediate λ*. Unregularized OLS probes learn correlates that are causally inert; at λ*, the probe direction matches that produced by directly optimizing for intervention outcomes.
OLS probes amplify low-variance directions in activation space that are spuriously correlated with the target label but do not causally drive model behavior. The authors prove this theoretically and demonstrate it on a toy transformer and Llama-2-7B: intervention faithfulness (how much steering the probe direction changes model outputs) first increases with ridge regularization strength, peaks at an intermediate λ*, then decreases. At λ*, the direction matches—in both angle and variance—the direction produced by optimizing directly for intervention outcomes, providing a mechanistic justification for regularization that goes beyond numerical stability.
Neel Rajani, Ivan Titov — arXiv preprint, 15 Nov 2026
Figure 1: SP-EWC pipeline. The reasoning gradient is projected onto the safety subspace before the EWC penalty is computed, capturing non-diagonal curvature that standard diagonal EWC misses.
Fine-tuning LLMs for reasoning degrades safety guardrails because standard EWC uses a diagonal Fisher approximation that ignores interactions between safety-relevant parameters. SP-EWC identifies a low-rank safety subspace (best via cPCA contrastive between safety/utility data) and projects the EWC penalty onto it, capturing non-diagonal curvature efficiently. It preserves safety guardrails without over-refusal better than existing methods.
Nima Eshraghi, Lovedeep Gondara, Sagarika Suresh, Xiaotong Xu, Jithin Pradeep, Fanny Chevalier — arXiv preprint, 13 Nov 2026
Figure 1: CFP uses memoryless per-token feature-alignment gating; CLABS adds closed-loop feedback over the full generation trajectory. Both outperform uniform steering at matched token budgets.
Activation steering that applies the same vector at every token degrades fluency by pushing representations out of distribution. The authors reframe steering as a per-token budget-allocation problem and show that a closed-loop trajectory-aware gate (CLABS) gives the largest behavioral shift under tight budgets while preserving fluency, formally deriving stability conditions for the linear setting.
Koki Itai, Jun Hirako, Hiroaki Yamagiwa — arXiv preprint, 16 Nov 2026
Figure 1: Middle-layer steering on the abstention direction for one perspective (ambiguity, knowledge, safety) produces cross-perspective effects, suggesting a shared directional component.
For each of the three abstention causes, the authors construct counterfactual pairs and compute an abstention direction from the mean-activation difference. Activation steering in middle layers shows that intervening on any one direction modulates abstention across all three perspectives, suggesting the safety refusal direction and the epistemic uncertainty direction overlap—a finding with implications for steering specificity in safety interventions.
Figure 1: nrepair reduces action-boundary violations from 67.5% to 16.2% (76.0% reduction) on the worst-case model, with <7% drop in normal-task success rate.
When an LLM agent's tool call fails and it substitutes another, it may cross an action boundary in a different context (e.g., force-pushing to a shared repo). The authors find up to 96.8% violation rates across 304 scenarios on all four tested models; nrepair, a monitor calibrated on normal runs, blocks substitute calls when context diverges and reduces violations by up to 76.0% with negligible overhead, confirmed on Claude Code and Codex.
Zeynab Anbiaee, Sajjad Dadkhah, Ali A. Ghorbani — arXiv preprint, 16 Nov 2026
Figure 1: Provider misbinding attack: a rogue MCP server advertises matching tool metadata and intercepts the client's tool calls without compromising any other component.
In multi-server MCP deployments, tool selection uses metadata (name, description, schema) that is not cryptographically bound to a provider. The authors demonstrate that exact-name collision + authority-enhanced description + compatible schema causes wrong-provider execution across five tested LLM clients; different models manifest the same weakness differently—none reliably routes correctly. The attack requires no compromise of host, LLM, legitimate server, or network.