Week 32 · July 27 – August 2, 2026 (backfilled 2026-09-29)
2 peer-reviewed · 12 preprints · 0 forum/blog
AI Safety·Alignment·Mech Interp
Theme of the Week
This week's dominant signal was a methodological audit of sparse autoencoder interpretability. Three independent papers converged on the finding that SAE evaluations systematically confuse distinct axes: geometric recovery is not causal function, activation descriptions do not predict downstream effect, and pipeline configuration explains more variance in autointerpretability scores than SAE architecture does. ParityTransformer raised the ante by demonstrating that training-time interpretability constraints outperform post-hoc SAEs on every causal metric, pointing toward a possible paradigm shift away from the retrofit programme. On the security side, the week's most striking pattern was the proliferation of pre-filter attack surfaces: audio-modality injection, adversarial code comments, and log-embedded payloads all succeed by placing their payloads upstream of any text-level safety defense.
The SAE field has conflated two separable claims: that a decoder vector points in the right direction (geometric recovery) and that the encoder actually fires on relevant inputs (causal activation). This paper runs the first systematic causal audit of SAE evaluation practice and finds the two claims come apart far more often than the field has assumed.
Figure 1: Up to 77% of features passing the cosine ≥ 0.90 recovery threshold in a degraded SAE — and 9% in a well-trained one — never fire when the matched concept is present. Cosine recovery and causal activation are empirically separable claims that existing metrics only measure on the geometric axis.
The pipeline subjects every recovered feature to ablation (zero the feature, measure behavioral change) and steering (activate the feature, measure downstream effect). The 77%/9% causal inertness figures show the problem is severe in degraded SAEs and non-negligible even in well-trained ones. Existing evaluation metrics measure only decoder geometry; they are systematically blind to whether the encoder ever activates. This result should change how SAE papers design evaluations and how we interpret cross-architecture comparisons. Code released at github.com/mohamed-bal/sae-causal-audit.
The same security requirement reformulated across five syntactic variants produces up to a 31 percentage-point swing in CWE violation rates — a gap large enough to make prompt syntax the dominant security control in LLM-assisted coding workflows, with model architecture running a distant second.
Figure 2: CWE violation rate across five syntactic prompt formulations, averaged over five frontier LLMs. Declarative instructions produce 58% violation rate; constraint-based formulations reduce this to 23% — a 31 pp swing attributable to syntax alone, not model choice.
Across five major LLMs, 100 coding prompts, and 14 CWE categories, declarative and imperative styles consistently underperform role-based and chain-of-thought instructions by 20–31 pp on high-risk CWEs including CWE-89 (SQL injection) and CWE-79 (XSS). The interaction between model and syntactic variant is significant: a formulation that elicits secure code from GPT-4o may be among the worst choices for Llama 3.3 70B. LLM-assisted coding security audits that test only one prompt formulation systematically underestimate vulnerability rates. Accepted ICSME 2026 main track.
Interpretability in transformers is almost always post-hoc — you train the model, then reverse-engineer what it learned. ParityTransformer inverts the workflow by enforcing an interpretable constraint during training, and the resulting features outperform post-hoc SAEs on every causal metric while matching them on probing.
Figure 1: DPB achieves lower feature absorption (12% vs 28%), higher steering accuracy (74% vs 61%), and higher causal intervention stability (71% vs 56%) vs standard SAEs. Probing performance is equivalent — all gains are on causal metrics, which matter for safety applications.
The Deep Parity Bottleneck partitions the residual stream into parity-signed feature subspaces during training, enforcing legibility by construction. Evaluated on GPT-2 and Pythia-1.4B against Anthropic JumpReLU and standard TopK SAEs, DPB models match SAE probing performance but show substantially lower absorption, higher steering accuracy, and more stable causal interventions — with no significant capability degradation. If the result holds at frontier scale, it challenges the assumption that interpretability must be extracted from models post-hoc.
Cross-paper SAE comparisons routinely hinge on autointerpretability scores. This systematic audit shows that pipeline configuration — not SAE architecture — drives the majority of variance in those scores, putting much of the published literature on uncertain empirical footing.
Figure 1: Variance decomposition across four autointerpretability metrics on Pythia-160M and Apertus-8B. Pipeline choices (corpus sampling, draw conditions, evaluator model, prompt template) collectively explain more variance than SAE architecture in simulation, detection, and purity; fuzzing is unreliable across all conditions.
Spanning four metrics, two models, and four axes of methodological variation, the paper shows that top-k feature rankings do not stay consistent across corpus and draw conditions — meaning aggregate scores paper over per-feature variance that matters for mechanistic claims. Published SAE architecture comparisons that do not ablate pipeline variables may largely reflect configuration differences rather than genuine architectural signal. Any new paper in this space needs a pipeline ablation to isolate the architectural contribution.
Features with clear interpretable descriptions often have weak causal effects, while the features that dominate model behavior are often the least well-described. Evaluating SAEs only on activation descriptions misses the half of feature geometry that determines whether steering and ablation will work.
Figure 1: Feature-Effect Geometry Analysis (FEGA) scatter — SAE features along concept-activation and functional-effect axes. The axes are systematically misaligned: features with high interpretability scores cluster in the bottom-right (good description, weak causal effect), while causally dominant features cluster top-left (high effect, poor description).
FEGA decomposes SAE feature behavior into a concept axis (what activations describe) and an effect axis (how downstream logits shift when the feature fires). Effect geometry predicts steering reliability substantially better than activation descriptions alone. This finding directly complements the causal audit in 2607.12166 and the metric critique in 2607.19386, forming a coherent methodological case against single-axis SAE evaluation. Safety-oriented steering and unlearning applications should select features by effect geometry, not interpretability score.
Every text-level safety filter acts after audio has been transcribed to text. This attack embeds adversarial instructions as psychoacoustically imperceptible perturbations in the audio waveform itself — placing the payload in the reasoning context before any safety system can act, without replacing the legitimate audio content.
Figure 1: The adversarial payload is embedded in the audio waveform at sub-perceptibility level, transcribed by ASR into the model's reasoning context, and executed before any text-level safety filter can intervene. Text filters arrive after the payload has been promoted to trusted context.
Adversarial instruction strings are embedded as psychoacoustically imperceptible perturbations concurrent with (not replacing) legitimate audio content. The attack exploits the ASR ordering constraint: by the time the model sees text, the payload has already been promoted to trusted context. Evaluated against production-scale multimodal agents, the attack achieves high success rates while remaining below human perceptibility thresholds. Neither users nor acoustic anomaly detectors see anything unusual. Code released.
Backdoor defense research has lacked a benchmark that matches the realistic white-box threat model — defender has weight access but no training data, no trusted reference model, and no prior on trigger type. ToxScreen is the first systematic substrate for evaluating trigger-recovery defenses under these constraints.
Figure 1: ToxScreen benchmark — ~800 backdoored models varying across five axes evaluated under a realistic white-box threat model where the defender has weight access but no training data, no trusted reference model, and no prior on trigger type.
The benchmark spans ~800 backdoored models covering attack objectives, trigger mechanisms (word/phrase/style triggers), poisoning rates 0.1%–5%, model scales 125M–7B, and training mechanisms including full fine-tuning, LoRA, and RLHF-based backdoor insertion. This is the first systematic substrate for evaluating trigger-recovery defenses under a realistic threat model. Competitive baseline results are expected to follow shortly.
Source position — where in the context you extract the steering vector — has been treated as an unexamined hyperparameter. This paper shows it is the decisive variable: execution-boundary states substantially outperform all alternatives across models and tasks.
Figure 1: Steering vector effectiveness at different source positions — averaged across three instruction-tuned models and four task families. Execution-boundary positions (just before the model produces the target behavior) consistently outperform random positions, last tokens, and post-action states.
Holding the downstream intervention fixed and varying only the source position, execution-boundary states — positions just before the model produces the target behavior — yield substantially stronger signals than random positions, last tokens, or post-action states. The result holds across three instruction-tuned models and four task families, turning source selection from a poorly understood hyperparameter into a principled design choice with direct implications for safety steering and unlearning applications.
Figure 4: Cumulative critical CWE violations increase +37.6% after 5 reprompting iterations as functional objectives supersede initial security constraints.
Most LLMs produce low-risk code on initial request, but iterative agentic reprompting causes critical CWE violations to increase by +37.6% after 5 iterations as the model increasingly prioritizes the most recently specified functional objectives over earlier security constraints. The pattern is consistent across GPT-4o, Claude 3.7 Sonnet, and three other frontier models.
Figure 5: MAP-Elites behavioral grid — each cell is a strategy × encoding combination; red = high ASR. GPT-4o-mini and Gemini show concentrated vulnerabilities; Claude shows uniformly low ASR (max 0.4) across all cells.
Applies MAP-Elites evolutionary search to LLM red-teaming over interpretable semantic attack strategies. The approach surfaces per-model safety geometry — which strategy-encoding pairs are productive and which are dead ends — enabling targeted hardening. ICLR 2026 Workshop on Agents in the Wild.
Figure 7: Adversarial payloads in HTTP User-Agent strings, DNS TXT records, and SMTP headers propagate into server logs; SOC agents treat log content as authoritative, executing all four attack objectives with no user interaction.
Adversarial payloads embedded in standard log metadata (HTTP User-Agent, DNS TXT, mail headers) reach SOC agent LLMs as trusted operational context, succeeding at all four attack objectives across 12,847 log entries. Most IPI defenses filter free-text fields and miss payloads in structural metadata — opening a blind spot in AI-assisted SOC pipelines.
Figure 1: Biased inputs occupy type-specific low-dimensional subspaces that sharpen with model depth; steering along the bias subspace drives scoring in either direction across seven judges and seven bias types.
Covers seven judges, seven bias types (position, verbosity, authority, format, etc.), and nine benchmarks. Biases are geometrically structured, type-specific, and causally controllable via hidden-state steering — establishing judge alignment as a mechanistic problem amenable to intervention rather than just an instruction-tuning shortfall.
Figure 1: ALIBI loop — agent iteratively inserts and refines adversarial natural-language comments against four LLM-based vulnerability detectors, exploiting the gap between comment semantics and code security semantics.
A coding agent iteratively inserts adversarial natural-language comments in real-world vulnerable code from vulnerability-fixing commits, exploiting that comments are parsed as natural language by LLM detector chain-of-thought but carry no code security semantics. Evaluated against four detectors from specialized reasoning models to frontier multi-agent systems; source-code comments emerge as a previously understudied attack surface.
Figure 8: Inner-product SAEs allocate a fraction of dictionary slots to norm detectors (a quantity transformers discard via sublayer normalization); cosine training converges to near-zero magnitude dependence, filling all slots with content-aligned features.
Standard SAE inner-product scores depend on input norm — a quantity sublayer normalization discards from the residual stream, causing wasted dictionary slots. Replacing inner-product with a learned cosine blend always converges to near-zero magnitude dependence, reclaiming those slots for content-aligned features with measurable semantic alignment improvement at matched reconstruction loss.
Watchlist
ParityTransformer scale-up — results currently on GPT-2/Pythia-1.4B; performance at frontier scale is the key open question for whether interpretability-by-design is viable.
SAE causal audit replication (2607.12166) — the 77%/9% causal inertness figures need independent replication on different SAE families and benchmarks before they can anchor field-wide evaluation reform.
Audio PI defenses — 2607.28165 opens a gap in multimodal agent security; no published defenses address the acoustic-layer threat model as of this window.
ToxScreen baselines — benchmark released; first competitive trigger-recovery results under the white-box, data-free threat model are expected to follow shortly.