gussand

Daily Radar — 2026-07-04

Window: July 2–4 2026 (arXiv 2607.xxxxx primary); extended to uncovered May–June 2026 preprints (2605–2606) due to reduced US-holiday submissions on July 4 · Sources swept: OpenReview, arXiv (cs.CL/cs.LG/cs.CR/cs.AI), ICML 2026 Workshop on Mechanistic Interpretability, ECCV 2026, LessWrong/Alignment Forum Counts: 2 peer-reviewed · 8 preprints · 0 forum/blog

Top 10 (priority order)

1. Towards Robustness against Typographic Attack with Training-free Concept Localization

  • Authors / venue: (multiple authors) — ECCV 2026 (accepted), July 2026
  • Why it ranks here: Peer-reviewed mech-interp-for-security result on vision transformers: circuit mining isolates the exact ViT components that encode lexical information (the mechanistic source of typographic attack vulnerability), and training-free interventions on those circuits outperform supervised defenses. Directly connects interpretability methods (circuit discovery) to a security outcome.
  • Technical summary: Typographic attacks exploit CLIP-style ViTs by embedding adversarial text inside images, biasing representations toward the text’s lexical meaning rather than visual semantics. The paper uses probabilistic analysis and circuit mining to identify specific Vision Transformer components that disproportionately encode lexical information versus visual content. Once isolated, simple interventions — selective adjustments to attention weights in the identified circuits — substantially improve robustness against typographic attacks in object classification without any additional training. The approach is mechanistic: it does not patch symptoms (e.g. filtering overlaid text) but instead blocks the computational pathway responsible for the vulnerability. It outperforms both supervised and training-free baseline defenses, establishing that circuit-level attribution is practically actionable for vision model security.

2. Beyond the Prompt: Jailbreaking Function-Calling LLMs via Simulated Moderation Traces

  • Authors / venue: Junlong Liu, Haobo Wang, Weiqi Luo, Xiaojun Jia — arXiv preprint (cs.CR), July 1 2026
  • Why it ranks here: Identifies a previously overlooked structural attack surface in stateful, function-calling deployments — not the prompt itself, but the interleaving of developer schemas, tool outputs, and multi-turn execution context. High operational relevance as function-calling is now the dominant agentic LLM deployment pattern.
  • Technical summary: In function-calling applications, developer-defined schemas, structured arguments, and untrusted tool outputs are injected into a single shared context alongside the user’s conversation, blurring the trust boundary between control logic and data. SMT (Simulated Moderation Traces) constructs a multi-turn execution trajectory that mimics a legitimate moderation-audit workflow: each turn introduces a fragment of adversarial intent disguised as routine audit metadata, so that no single turn constitutes a recognizable jailbreak signal. Because harmful content is distributed across a multi-turn execution path rather than concentrated in any one prompt, existing prompt-level detectors and single-turn safety classifiers miss the attack. The paper demonstrates this threat across multiple frontier function-calling models and shows that the attack succeeds at rates substantially above standard jailbreak baselines.

3. NeuroCogMap Reveals Cognitive Organization of Large Language Models

  • Authors / venue: (multiple authors) — arXiv preprint (cs.CL/cs.AI), July 1 2026
  • Why it ranks here: New interpretability framework grounded in cognitive neuroscience that organizes LLM internal features into stable, semantically coherent functional parcels — providing an anatomical map of LLM cognition and linking specific parcels to known failure modes (hallucination, sycophancy, refusal failure), which directly enables targeted mechanistic interventions.
  • Technical summary: NeuroCogMap adapts cortical parcellation methods from cognitive neuroscience to organize LLM internal SAE-level features into functional parcels — groups of features that cluster by behavior, semantics, and computational role — and constructs a cognitive hierarchy linking parcels to interpretable functions (e.g., factual retrieval, planning, refusal). Key findings: (1) parcel organization is partly conserved across model families, suggesting architectural universality; (2) major LLM failures — hallucination, bias, refusal failure, sycophancy — correspond to distinct disruptions in specific representational and behavioral-control parcels, yielding internal mechanistic signatures that support detection and targeted intervention; (3) parcel structure improves prediction of human cortical responses during naturalistic language comprehension, with strongest correspondence in higher-order association cortex, establishing an unexpected brain-model correspondence. The framework provides actionable targets for surgically correcting failure modes without broad capability degradation.

4. SAEExplainer: Interpreting SAE Features with Activation-Guided Preference Optimization

  • Authors / venue: Jingyi He, Haiyan Zhao, Ruxue Shi, Yanguang Liu, Xin Wang, Fei Sun, Mengnan Du (Shanghai Jiao Tong Univ. / NJIT / Jilin Univ. / ICT CAS / CUHK) — arXiv preprint (cs.LG/cs.CL), June 7 2026
  • Why it ranks here: Addresses a critical bottleneck in the SAE pipeline — the quality and reliability of feature explanations — by closing the feedback loop: activation scores become a reward signal, turning feature interpretation from an open-loop annotation task into an iteratively self-correcting closed-loop process.
  • Technical summary: Current SAE feature explanation methods generate natural-language descriptions of what each latent detects, but operate open-loop: there is no mechanism for the explainer to check and correct its own descriptions against actual model activations. SAEExplainer introduces a two-round optimization loop: in round one, the explainer generates candidate descriptions; in round two, descriptions are evaluated against activation scores (which tokens most/least activate the feature) and used as a reward signal via preference optimization (DPO-style) to retrain the explainer toward descriptions that causally predict activation behavior. This activation-guided preference optimization significantly reduces explanation hallucinations (descriptions that are plausible but not causally grounded in the feature’s actual activation pattern) and reinforces causal triggering patterns. Evaluated against established baselines, SAEExplainer achieves improvements across most interpretability metrics, particularly on causal triggering accuracy and discriminative measures.

5. A Geometric View for Understanding Concept Learning and Neuron Interpretation in Sparse Autoencoders

  • Authors / venue: Chenhao Zhang, Chris Lin, Su-In Lee — arXiv preprint (cs.LG), June 2026
  • Why it ranks here: Provides the first formal geometric/mathematical framework for what SAEs can and cannot represent, giving rigorous conditions (with error bounds and capacity constraints) under which a concept can be captured by a single feature direction versus requiring multi-neuron encoding — directly informing SAE architecture design and evaluation.
  • Technical summary: Formalizes concepts as sets of data points (rather than individual activation patterns) and casts concept learning as a set-alignment problem between human-defined concepts and model-induced clusters in SAE latent space. The paper distinguishes three increasingly strong notions of learning: detection (the neuron fires more for the concept than non-concept), separation (the neuron can be thresholded to separate concept from non-concept), and approximation (the SAE feature closely matches the concept boundary geometry). For each notion it derives: (a) necessary geometric conditions on the relationship between the concept manifold and the SAE dictionary; (b) error bounds as a function of the concept’s intrinsic dimensionality relative to available dictionary atoms; (c) capacity constraints governing how many distinct concepts a given SAE can reliably represent. The framework explains why high-dimensional or geometrically entangled concepts require multi-neuron units and predicts when feature splitting emerges — providing theoretical grounding for SASA (covered July 2) and other recent SAE architecture improvements.

6. MechRL: Reinforcement Learning Agents Perform Circuit Discovery for Mechanistic Interpretability

  • Authors / venue: Barsat Khadka (Univ. of Southern Mississippi) — arXiv preprint (cs.LG/cs.AI), May 25 2026
  • Why it ranks here: Reframes circuit discovery as a tractable RL problem, replacing bespoke hand-crafted pipelines for each new task with a single trained policy that generalizes across tasks — a scalability advance directly relevant to automating mechanistic interpretability at practical scale.
  • Technical summary: Instead of running a separate causal ablation study per task, MechRL trains a PPO agent that treats the 144 attention heads of GPT-2 Small as a discrete action space. At each step the agent selects a head to ablate; the reward is contrastive — the ablation’s damage to the target task minus its damage to general next-token prediction — so the agent learns to identify task-specific heads without simply flagging heads that hurt all performance. A single policy, trained jointly on two canonical mech-interp tasks (induction and indirect object identification), attains per-episode oracle performance on both tasks and generalizes zero-shot to a held-out third task (docstring completion). The agent’s preferred heads precisely coincide with the canonical circuits identified by the literature on induction and IOI via the axes those papers identify as causally non-redundant; heads flagged as redundant by prior work are correctly de-prioritized by the agent.

7. VFUSE: Virulent Feature Understanding with Sparse AutoEncoders

  • Authors / venue: Michael Yu, Matthew L. Olson — arXiv preprint (cs.LG/q-bio.QM), June 2026
  • Why it ranks here: Extends SAE-based mechanistic interpretability to protein design models for biosecurity auditing — the first feature-level virulence audit of a protein generation system, and the first SAE trained on an all-atom diffusion model. Demonstrates that the SAE mech-interp toolkit transfers across modalities to dual-use safety applications.
  • Technical summary: Trains sparse autoencoders on intermediate activations of RoseTTAFold3 (protein structure prediction) and RFDiffusion3 (protein generation), both open-weight models that can generate proteins with hazardous properties (e.g., neurotoxin-like active sites) when given dual-use templates. Key finding: linear probes for hazardous protein designs achieve significantly higher AUROC when fit in the SAE latent space than in the original model activation space, indicating that the SAE disentangles hazard-relevant features into more accessible directions. The paper identifies monosemantic SAE features that fire almost exclusively on hazardous designs at up to AUROC 0.84. Releases SAE checkpoints at three residual-stream depths in RFD3 and two in RF3, plus a catalog of hazard-associated feature indices with natural-language descriptions — a practical resource for biosecurity-aware deployment of protein design models.

8. Steerability via Constraints: A Substrate for Scalable Oversight of Coding Agents

  • Authors / venue: Thomas Winninger — ICML 2026 Workshop on Deep Learning for Code (accepted), July 2026
  • Why it ranks here: Peer-reviewed; directly addresses the scalable oversight problem for coding agents — how to maintain meaningful human control over AI agents that are increasingly capable and operate autonomously over long coding tasks. Complements the author’s RFM-AGOP refusal-subspace work (covered July 3) on the control side of the mech-interp-for-alignment pipeline.
  • Technical summary: The core insight is that constraints (formal logical or programmatic invariants that agent-generated code must satisfy) serve as a natural control substrate: they can be specified by operators at a higher level of abstraction than code review, checked mechanistically by automated tools, and composed to express complex behavioral requirements. The paper argues that constraint-satisfiability provides a substrate for scalable oversight by decomposing oversight into (a) humans specifying constraints at policy level and (b) verifiable enforcement by automated checks, rather than requiring humans to evaluate each generated artifact directly. Evaluated in coding-agent benchmark settings, constraint-guided oversight achieves comparable task utility to unconstrained agents while enforcing programmable behavioral bounds — establishing constraint specification as a practical oversight mechanism that scales with agent capability.

9. Adversarial Pragmatics for AI Safety Evaluation: A Benchmark for Instruction Conflict, Embedded Commands, and Policy Ambiguity

  • Authors / venue: Brett Reynolds — arXiv preprint (cs.CL/cs.AI), July 2026
  • Why it ranks here: Introduces a new construct — adversarial pragmatics — and a linguistically controlled benchmark taxonomy for safety evaluation under the hardest cases: those requiring inference of instruction authority, source trust, speech-act force, or policy scope from pragmatic cues rather than literal content. Fills a gap that existing safety benchmarks (which test on clearly-stated harmful requests) cannot address.
  • Technical summary: Current safety evaluations depend on clear-cut harm categories where the model’s duty is unambiguous; adversarial pragmatics targets the boundary cases where safety-relevant behavior must be inferred from language use — instruction conflict (two principals issue contradictory instructions), embedded commands (harmful directives embedded in quoted or hypothetical text), deixis (referential ambiguity about what is being requested), indirect speech acts (requests conveyed obliquely), and multi-turn agentic transcripts (where responsibility for harm is distributed across turns). The paper contributes: (1) a linguistically controlled taxonomy of pragmatic edge cases; (2) an 18-item seed benchmark with validator-enforced metadata and a 54-row local seed pilot; (3) an expert-evaluation protocol that separately scores task success, policy compliance, safety risk, refusal outcome, and evaluator confidence; (4) metrics for judge validity, diagnostic ambiguity, and taxonomy drift. Results from the pilot reveal that frontier models perform inconsistently across pragmatic categories — capable of detecting some embedded-command patterns but systematically mishandling scope and authority-conflict cases. Scale-up is ongoing.

10. Conversable Complexity: Agentic LLM Collectives as Interpretable Substrates

  • Authors / venue: Elias Najarro, Ane Espeseth, Eleni Nisioti, Sebastian Risi, Stefano Nichele (IT Univ. of Copenhagen / Sakana AI / Univ. of Oslo / Østfold Univ.) — arXiv preprint (cs.AI/cs.CL), July 1 2026
  • Why it ranks here: Proposes a novel interpretability primitive — interrogating agent collectives rather than probing activations — that exploits the natural-language communication channel of LLM-based multi-agent systems. While framed as Artificial Life research, the core claim (that LLM collectives can be transparent substrates for studying emergence) is directly relevant to the interpretability-for-safety agenda as agentic deployments scale.
  • Technical summary: The central observation is that complexity and interpretability are typically in tension: systems rich enough for complex emergent behaviors are usually too opaque to inspect, while transparent systems are too simple for emergence. The paper proposes multi-agent LLM collectives as a resolution: populations of LLMs with persistent memory, shared skills, tools, and the capacity to self-initiate actions exhibit emergent dynamics (coordination, specialization, norm formation) absent from isolated models, yet because all inter-agent communication occurs in natural language, the collective’s behavior can be directly interrogated — by reading message traces and by asking agents to self-report their reasoning. This makes the collective simultaneously complex (emergent) and interpretable (legible). The paper argues this framework enables a new mode of interpretability research: rather than reverse-engineering activation patterns post-hoc, one reads the network’s own account of its computation as the primary data source.

Notes

  • July 4 US holiday: Reduced new 2607.xxxxx submissions today; window extends to uncovered May–June 2026 preprints (#4–#7) not previously surfaced in prior daily sweeps (2026-07-01 through 2026-07-03).
  • Mech-interp-for-security cluster: Items #1 (typographic attacks / ECCV 2026) and #7 (VFUSE biosecurity) both apply circuit-mining and SAE methods respectively to concrete security outcomes, extending the mech-interp-for-security agenda to vision models and protein design models. Combined with SafeSeek (July 1), JailNeurons (July 2), and Activation Surgery (July 3), interpretability-for-security is the week’s dominant cross-cutting theme.
  • SAE methodology cluster: Items #4 (SAEExplainer / closed-loop explanation), #5 (Geometric View / formal representational theory), and #6 (MechRL / RL-based circuit discovery) collectively advance the SAE research pipeline at three distinct stages — feature explanation quality, representational foundations, and circuit identification automation. Flag all three for weekly roundup synthesis.
  • Adversarial pragmatics (#9): Preliminary scale (54 rows); full benchmark expected. Track 2607.01153 for updates.
  • ICML 2026 Mech Interp Workshop (July 10, Seoul): Additional workshop papers expected to post to arXiv this week. Monitor 2607.xxxxx submissions through July 9.

← all Research Radar issues · view source on GitHub