Mohamed Abdessalem Bal — arXiv 2607.12166, July 13, 2026
The SAE evaluation field has been measuring the wrong thing. Cosine similarity between ground-truth directions and learned decoder atoms checks geometry — but a feature can be geometrically well-recovered and causally invisible at the same time. This paper runs the first systematic causal audit of SAE features and finds the disconnect is substantial.
Figure 3 (SAE Causal Audit): In degraded SAEs, up to 77% of features passing the cosine ≥ 0.90 geometric recovery threshold never fire when their matched concept is present — including at cosine ≈ 1.000. Even well-trained SAEs have ~9% causally inert features. The paper separates two empirical claims the field has conflated: decoder geometry and encoder activation behavior.
The experimental pipeline subjects every geometrically-recovered feature to two causal tests: ablation (zero the feature, measure behavioral change) and steering (activate the feature, measure downstream effect). Both must hold for a feature to support mechanistic claims, but standard evaluation metrics measure only cosine similarity between decoder atoms and ground-truth directions — checking geometry, not behavior. The 77%/9% causal inertness figures show the gap between geometric and causal validation is severe in degraded SAEs but non-zero even in well-trained ones. The result directly challenges the validity of mechanistic claims built on SAE features without causal validation, and should change evaluation standards for new SAE papers. Code at github.com/mohamed-bal/sae-causal-audit.
Zixiang Xu et al. — arXiv 2607.11871, July 13, 2026
Figure 8: LLM-as-judge bias occupies a low-dimensional type-specific subspace in activation space. Clean inputs (blue) cluster tightly; biased inputs (red) are displaced along a direction that sharpens with model depth. Steering along the reverse direction (orange dashed) restores baseline scoring on biased inputs — making judge alignment a mechanistic intervention target.
A representation-level account of LLM-as-judge bias across seven judges, seven bias types, and nine benchmarks. Biases aren't random activation noise — they're geometrically structured, type-specific, and causally controllable via hidden-state steering. The finding that bias direction sharpens with model depth implies that deeper judges concentrate bias in later layers, making late-layer interventions a practical correction target. 58 pages; 13 figures; comprehensive scope.
Shengfang Zhai et al. (NUS, Peking U, SJTU) — arXiv 2605.09397, May 10, 2026
Figure 10 (BadDLM): Four qualitatively distinct attack targets, all achieved with a single unified trigger-aware training objective that up-weights loss at target-relevant token positions in poisoned samples. High attack success rate maintained across all four categories while preserving clean-utility parity.
Broader in scope than SHADOWMASK (#9): where SHADOWMASK targets the forward-process mechanism, BadDLM targets the diversity of attacker objectives. The trigger-aware training objective up-weights loss at target-relevant positions, enabling precise control over what the backdoored model produces when triggered across four qualitatively different attack categories. Clean-utility parity makes BadDLM backdoors difficult to detect via behavioral testing on untriggered inputs.