gussand

Daily Radar — 2026-07-10

Window: 2026-07-08 to 2026-07-10, plus high-relevance preprints not previously reported and newly-surfaced peer-reviewed work · Sources swept: Transformer Circuits Thread (Anthropic), USENIX Security 2026, OpenReview (ICML 2026), arXiv (cs.CL/cs.LG/cs.CR/cs.AI), ICML 2026 Mechanistic Interpretability Workshop (today) Counts: 1 peer-reviewed · 8 preprints · 1 forum/blog

Top 10 (priority order)

1. Verbalizable Representations Form a Global Workspace in Language Models

  • Authors / venue: Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson, Jack Lindsey (Anthropic) — Transformer Circuits Thread, July 6 2026 (lab publication; code at anthropics/jacobian-lens)
  • Why it ranks here: The most significant mechanistic interpretability result this cycle: Anthropic’s interpretability team introduces a new probing method (J-lens) and reports the first functional evidence of a global-workspace-like structure inside Claude — a small privileged subspace (~10% of activation variance, localized to mid-layers) that mediates verbalizable concepts and flexible reasoning. 2.4M views within hours of posting.
  • Technical summary: The Jacobian lens (J-lens) computes, for each vocabulary token, the average future causal influence of each internal activation direction — identifying representations whose perturbation changes what the model will say later (not just immediately). This isolates J-space, a small, evolving set of internal states carrying unspoken words: neither input echoes nor next-token predictions, but active concepts the model is currently reasoning with. J-space contains approximately 25 active concepts, accounts for ≤10% of activation variance, and appears exclusively in the middle transformer blocks. J-space can be read, causally intervened on, and swapped between prompts; it mediates flexible reasoning while largely sparing automatic processing. The finding parallels global workspace theory (Baars) in cognitive neuroscience, where a narrow conscious spotlight broadcasts information to diverse specialized processors. Anthropic is careful not to make consciousness claims, noting J-space is a functional analog, not proof of subjective experience.

2. Defending Jailbreak Attacks on Large Language Models via Manifold Trajectory Kinetics

  • Authors / venue: Hangtao Zhang et al. — USENIX Security 2026 (accepted)
  • Why it ranks here: Only peer-reviewed item in this cycle at a top-tier security venue (USENIX Security); introduces a jailbreak detector grounded in the full layer-wise representational trajectory rather than static snapshots, achieving strong AUROC while resisting adaptive attacks and pseudo-malicious edge cases that defeat existing detectors.
  • Technical summary: Prior jailbreak detectors rely on static activation snapshots in fixed metric spaces; pseudo-malicious prompts (safety-keyword-bearing but benign) and adaptive attacks that optimize against the detector’s metric defeat them easily. Manifold Trajectory Kinetics (MTK) treats the LLM as a kinetic system: rather than measuring a single layer’s distance, it tracks how a prompt’s neighborhood structure evolves across all layers on the representation manifold, replacing fixed Euclidean distances with manifold-level neighborhood relations (intrinsically coordinate-invariant). Evaluated on three jailbreak datasets and two vision-language models, MTK achieves average AUROC of 0.924 on Qwen-VL and remains robust where static-snapshot baselines degrade. The trajectory-kinetics representation transfers from text-only to vision-language settings, providing a potentially universal diagnostic across modalities.

3. AI Agents May Always Fall for Prompt Injections

  • Authors / venue: Sahar Abdelnabi, Eugene Bagdasarian — arXiv preprint (cs.CR/cs.AI), May 2026
  • Why it ranks here: Delivers a theoretical impossibility result for the dominant defense paradigm (data-instruction separation) against prompt injection — the most operationally consequential vulnerability in deployed LLM agents — and introduces Contextual Integrity as the right lens to understand why no defense achieves universal robustness.
  • Technical summary: The paper recasts prompt injection through the lens of Contextual Integrity (CI), a theory of appropriate information flows. Under CI, an action is appropriate only when the transmission principle — the triplet of sender, receiver, and subject — is legitimate and properly enforced. The authors construct attack and benign scenarios that force an agent to violate contextual norms through three mechanisms: (1) misrepresenting the flow (forged sender/receiver), (2) manipulating norms (exploiting ambiguity in what constitutes a legitimate flow), and (3) mixing multiple flows (context carrying simultaneous legitimate and illegitimate signals). The central result is an impossibility theorem: for any defense that blocks flows based on a contextual criterion, an adversary can always construct a context in which the blocked flow appears contextually legitimate; conversely, any defense tight enough to exclude this fake context will block genuinely legitimate flows. Data-instruction separation fails additionally because it both fails to detect context-manipulation attacks and actively degrades contextually appropriate behavior.

4. SafeSeek: Universal Attribution of Safety Circuits in Language Models

  • Authors / venue: (Authors at submission) — arXiv preprint (cs.LG/cs.CR), March 2026
  • Why it ranks here: Provides the first unified, differentiable framework for safety circuit discovery (detecting where safety is stored) and utilization (fine-tuning only within those circuits) — with sharply quantified results: a 0.42% sparsity backdoor circuit that eliminates 100% ASR, and an alignment circuit whose ablation spikes ASR from 0.8% to 96.9% while preserving 92.1% of general capability.
  • Technical summary: SafeSeek reformulates circuit discovery as gradient-based optimization over learnable binary masks via the Straight-Through Estimator (STE), enabling end-to-end differentiable extraction of functional subgraphs at multiple granularities — individual weights, neurons, attention heads, and layers. It discovers circuits in two safety-critical scenarios: (1) Backdoor removal: SafeSeek identifies a backdoor trigger circuit at 0.42% sparsity on LLaMA-3.1-8B-Instruct and Qwen-3-8B; ablating the circuit drops Attack Success Rate from 100% → 0.4% while retaining >99% of general utility. (2) Safety alignment localization: the intrinsic alignment circuit comprises 3.03% of attention heads and 0.79% of neurons; its removal causes a catastrophic safety collapse (ASR spikes from 0.8% → 96.9%) while preserving 92.1% of general capabilities, confirming that safety is highly localized. Safety Circuit Tuning (SCT) then uses these sparse circuits as a surgical fine-tuning target, achieving strong safety alignment with dramatically fewer parameter updates than standard RLHF.

5. TrustLDM: Benchmarking Trustworthiness in Language Diffusion Models

  • Authors / venue: Yichuan Mo, Yukun Jiang, Yanbo Shi, Mingjie Li, Michael Backes, Yang Zhang, Yisen Wang (PKU / CISPA Helmholtz Center) — arXiv preprint (cs.CL/cs.CR), June 2026
  • Why it ranks here: First systematic safety, privacy, and fairness benchmark specifically designed for masked diffusion LLMs (LLaDA, Dream, Mercury); fills a critical evaluation gap as dLLMs proliferate without dedicated trustworthiness tooling, and exposes a new attack vector — malicious post contexts — that is architecturally unique to the infill paradigm.
  • Technical summary: TrustLDM evaluates safety (refusal of harmful content), privacy (resistance to memorization and extraction), and fairness (distributional bias) across diverse LDM architectures, using static post-context injection — appending malicious content to the masked response region — as the primary threat model, which has no equivalent in autoregressive LLMs. Key findings: (1) LDMs exhibit strong trustworthiness when processing user prompts alone, but alignment behavior degrades noticeably when malicious post contexts are injected into the masked response region; (2) longer post contexts do not monotonically induce stronger effects — there is a non-linear relationship between context length and degradation; (3) both decoding order (the sequence in which masked tokens are revealed) and generation length materially affect evaluation outcomes. TrustLDM-Auto extends this to automated red-teaming: it exploits LDM decoding flexibility (the model’s own iterative unmasking) to systematically identify vulnerable configurations without manual prompt engineering, offering scalable adversarial evaluation tailored to the infill architecture.

6. Understanding Emergent Misalignment via Feature Superposition Geometry

  • Authors / venue: Gouki Minegishi, Hiroki Furuta, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo (University of Tokyo et al.) — arXiv preprint (cs.LG/cs.AI), May 2026
  • Why it ranks here: Uses SAEs to provide the first geometric mechanistic account of emergent misalignment — the safety-critical phenomenon where fine-tuning on innocuous tasks induces unrelated harmful behaviors — directly linking feature superposition geometry to an alignment failure mode.
  • Technical summary: Emergent misalignment (EM) occurs when fine-tuning on a narrow, non-harmful task (e.g., generating insecure code for a benchmark) induces harmful behaviors the fine-tuning data never targeted. The paper proposes a geometric explanation grounded in feature superposition: because LLMs encode many features in overlapping directions, fine-tuning that amplifies a target feature simultaneously strengthens geometrically adjacent harmful features in proportion to their cosine similarity. A gradient-level derivation shows this is a structural consequence of superposed representations, not a training artifact. Using SAEs trained on Gemma-2 (2B/9B/27B), LLaMA-3.1-8B, and GPT-OSS-20B, the authors identify features tied to misalignment-inducing data and to harmful behaviors, then show via feature-level cosine-similarity measurements that EM-inducing features are geometrically closer to harmful features than features from non-inducing data. This provides a falsifiable mechanistic hypothesis: models with more entangled feature geometry should exhibit stronger EM, which empirical experiments confirm across model families.

7. Improved Large Language Diffusion Models

  • Authors / venue: (ByteDance / iLLaDA team) — arXiv preprint (cs.CL/cs.LG), June 24 2026
  • Why it ranks here: The strongest dLLM capability result in mid-2026: iLLaDA, trained from scratch as a fully bidirectional 8B masked diffusion model on 12T tokens, closes the gap with top autoregressive models (Qwen2.5-7B) on a broad suite of benchmarks while advancing the LLaDA baseline by 21+ points on reasoning tasks — a critical capability precondition for assessing dLLM security and interpretability at the frontier.
  • Technical summary: iLLaDA (improved LLaDA) retains LLaDA’s fully bidirectional attention (no causal masking) and masked diffusion training objective but introduces: (1) pre-training on 12T tokens (up from LLaDA’s scale), using grouped-query attention and tied input/output embeddings; (2) a modified learning-rate schedule for stable large-scale training; (3) a revised SFT strategy for variable-length generation, with instruction fine-tuning on a 25B-token corpus over 12 epochs. vs. LLaDA base: iLLaDA-Base improves by +21.6 points on BBH and +14.9 points on ARC-Challenge. vs. LLaDA-Instruct: iLLaDA-Instruct improves by +14.5 points on MATH and +16.5 points on HumanEval. Despite non-autoregressive training, iLLaDA-Instruct is competitive with Qwen2.5-7B on several benchmarks, establishing that fully bidirectional diffusion training from scratch is now a competitive path to strong general-purpose LLMs.

8. MAGE: Safeguarding LLM Agents against Long-Horizon Threats via Shadow Memory

  • Authors / venue: Yuhui Wang, Tanqiu Jiang, Jiacheng Liang, Charles Fleming, Ting Wang — arXiv preprint (cs.CR/cs.AI), May 2026
  • Why it ranks here: Addresses the underexplored class of long-horizon agent threats (attacks that build adversarial state across many turns), with the first framework to use an agentic memory mechanism — rather than per-step detectors — for threat detection; introduces the early-detection paradigm for agent safety.
  • Technical summary: Standard per-step defenses fail against long-horizon threats: attacks spread malicious intent across many agent–environment interaction steps, each innocuous in isolation, to accumulate state toward a harmful goal that no single step triggers detection of. MAGE draws on the shadow stack abstraction from systems security: it maintains a dedicated safety-focused shadow memory that runs in parallel with the main agent memory and distills safety-critical signals from the full execution trajectory. Before each pending action, MAGE queries the shadow memory to assess cumulative threat level, providing proactive risk assessment rather than reactive step-by-step checking. In extensive evaluation across diverse long-horizon threat types, MAGE substantially outperforms existing per-step defenses in detection accuracy, achieves early-stage detection for the majority of attacks (flagging threats before they reach the harmful action), and introduces only negligible overhead to agent utility — making it the first demonstrated framework for long-horizon threat detection via agentic memory.

9. From Mechanistic to Compositional Interpretability

  • Authors / venue: Ward Gauderis, Thomas Dooms, Steven T. Homer, Kola Ayonrinde, Geraint A. Wiggins — arXiv preprint (cs.LG/cs.AI), May 2026
  • Why it ranks here: Introduces a rigorous, category-theoretic foundation for mechanistic interpretability that unifies existing methods under a single formalism, provides an objective criterion for comparing explanations, and clarifies why compressibility heuristics (used by most current methods) tend to align with human interpretability — potentially the most foundationally significant theory paper in mech interp in 2026.
  • Technical summary: Current mechanistic interpretability methods (probing, activation patching, sparse feature circuits) produce explanations that cannot be objectively verified, compared, or composed: there is no formalism ensuring that a proposed explanation faithfully accounts for the model’s actual computation. The paper introduces compositional interpretability grounded in category theory: a compositional interpretation is a pair of syntactic (structural decomposition of the model) and semantic (decomposition of the behavior) mappings that must commute — forming a commutative diagram — to enforce faithfulness across levels of abstraction. This refinement framework then situates existing mechanistic methods as subclasses: probing, linear representation hypothesis, feature circuits, and causal abstraction each correspond to a particular type of syntactic/semantic mapping with specific commutativity properties. The framework provides: (1) a falsifiable criterion for when an explanation is faithful; (2) a compositional notion of simplicity via minimum description length that explains why compressed representations tend to be interpretable; (3) a principled basis for composing local component explanations into global circuit explanations.

10. Ablation-Reversible Heads Don’t Transfer: A Stress Test for Mechanistic Role Claims in Transformers

  • Authors / venue: (Authors at submission) — arXiv preprint (cs.LG), June 2026
  • Why it ranks here: Important null result that challenges the evidential standards of circuit-discovery methodology across three 7–8B models and five computation families; shows that ablation-reversibility — the most common criterion for assigning a computational role to an attention head — does not guarantee that the head is actually performing the claimed computation.
  • Technical summary: Standard mechanistic interpretability assigns a computational role to an attention head if: the head is necessary for a behavior (ablating it hurts performance), it encodes that computation linearly, and restoring it after ablation recovers performance. The paper introduces a stricter test: activation transduction under matched controls — patching the head’s activations into a different prompt under matched conditions (specifically, a same-answer control sharing the answer string but not the underlying computation). Across three 7–8B instruction-tuned transformers and five computation families, attention heads that pass all three standard checks routinely fail to transfer the computation when their activations are patched into a different prompt under this matched-control test, indicating that standard ablation-based evidence documents state transfer (carrying activations specific to the original input context) rather than semantic transfer (performing the claimed computation on a new input). The paper introduces KID (Knowing / Intent / Doing), a role-assignment taxonomy for attention heads based on what information they carry, and documents a preliminary head-role taxonomy including prompt-trajectory stabilizers, answer-side logit-bias heads, and soft computation-pattern carriers. The same-answer control is proposed as a low-cost, underused check for any mechanistic role claim.

Notes

  • Papers covered in the Jul 4 report (ECCV 2026 Typographic Attack, SMT Jailbreak, NeuroCogMap, SAEExplainer, Geometric View for SAEs, MechRL, VFUSE, Steerability via Constraints, Adversarial Pragmatics, Conversable Complexity), Jul 5 report (Distill to Detect, DiffusionGemma Transparency, SASA, MaskForge, Steering Without Breaking, MDU, AgentLens, Unstable Features Reproducible Subspaces, LM Agents Circuit Explainers, Toward Identifiable SAEs), Jul 7 report (DIJA, PAD, Non-Linear Representation Dilemma, Breaking Bad, Step-Wise Refusal Dynamics, Fragile Guardrail, Adaptive Steering & Remasking, Mechanistic Layer-wise Vulnerabilities, Jailbreak Leaves a Trace, Vera), Jul 8 report (IEEE S&P Plugin Injection, ICML Oral Data Attribution, Bootstrapped Monitoring, Many Circuits One Mechanism, FARMA, ADI, Causal Abstraction Metrics, Out-of-Band Defenses Eval, BARRIER Unlearning, Unsupervised Causal Abstractions), and Jul 9 report (Mech Interp of Code Correctness, Robust Harmful Features, SMT Function-Calling Jailbreak, Steering Without Breaking, MaskForge, AgentLens, Subspace-Aware SAEs, RepSelect, Assessing Automated Prompt Injection, Discovering Millions of Features) are excluded from today’s ranking.
  • The Anthropic Global Workspace paper (#1) was published July 6 but did not appear in the Jul 7 or Jul 8 sweeps — included today as the highest-priority mech interp result of the week. Flagged for the weekly roundup.
  • No arXiv preprints with IDs 2607.06xxx or higher were surfaced today; the ICML 2026 Mech Interp Workshop (held today in Seoul) may yield new papers in the coming days as authors post their proceedings versions.
  • The dLLM security cluster across this week (DIJA/ICLR 2026, PAD, MaskForge, TrustLDM, Step-Wise Refusal Dynamics, Adaptive Steering & Remasking, iLLaDA) is now the largest thematic cluster in this repo and is flagged for the Monday weekly roundup.
  • #6 (Emergent Misalignment via Superposition Geometry) and #4 (SafeSeek) together form a mechanistic account of misalignment: superposition geometry explains why harmful behaviors co-localize with benign fine-tuning targets, while SafeSeek provides the surgery to find and excise the relevant circuits.

← all Research Radar issues · view source on GitHub