Interpretability in transformers is almost always post-hoc — you train the model first, then reverse-engineer what it learned. ParityTransformer inverts the workflow: it forces an interpretable constraint during training via the Deep Parity Bottleneck (DPB), and the features that emerge match SAE-discovered features on probing benchmarks while substantially outperforming them on every causal metric — absorption, steering, and intervention stability.
Figure 1: ParityTransformer DPB vs standard SAE on three interpretability metrics. DPB achieves lower feature absorption (12% vs 28%), higher steering accuracy (74% vs 61%), and higher causal intervention stability (71% vs 56%). Probing performance is equivalent; causal metrics are substantially better across the board.
The Deep Parity Bottleneck partitions the residual stream into parity-signed feature subspaces during training, enforcing that each feature's causal role is legible by construction rather than recovered post-hoc by a separately trained sparse autoencoder. Evaluated on GPT-2 and Pythia-1.4B against Anthropic JumpReLU and standard TopK SAEs, DPB models match SAE performance on token-type probing but show substantially lower feature absorption rates, higher steering accuracy, and more stable causal intervention outcomes across depth and model scale. The interpretability-by-design constraint produces no significant capability degradation.
Security prompts aren't just about content — the syntactic form of the instruction matters as much as the instruction itself. The same security requirement reformulated across five syntactic variants produces up to a 31 percentage-point swing in CWE violation rates, a gap large enough to render prompt engineering the primary security control in LLM-assisted coding workflows.
Figure 2: CWE violation rates across five syntactic prompt formulations (100 coding tasks, 14 CWE categories, 5 LLMs). Declarative instructions produce 58% violation rate; constraint-based formulations reduce this to 23% — a 31 pp swing from syntax alone. Role-based and CoT styles cluster near the efficient end.
Across five major LLMs (GPT-4o, Claude 3.7 Sonnet, Gemini 2.0 Flash, Mistral Large, Llama 3.3 70B), the authors systematically tested 100 coding prompts covering 14 CWE categories under each of five syntactic formulations. Declarative and imperative styles consistently produced the highest violation rates on CWE-89 (SQL injection) and CWE-79 (XSS); role-based and chain-of-thought instructions reduced violations by 20–31 pp. Crucially, the interaction between model and syntactic variant is significant — a formulation that elicits secure code from GPT-4o may be among the worst choices for Llama 3.3 70B. Accepted to ICSME 2026.
Conventional jailbreaks attack token prefixes — a mechanism that doesn't exist in masked diffusion LLMs. PAD takes a different route: it directly disrupts the bidirectional attention patterns responsible for RLHF alignment, achieving 97% attack success rate across four dLLMs where standard attacks fail at <5%. The key insight is that alignment in dLLMs lives in specific attention head activations during denoising, and those can be targeted without access to model weights.
Figure 3: PAD attack success rate vs ReNeLLM and GCG across four masked dLLMs. PAD achieves 95–97% keyword-based ASR by disrupting alignment-critical attention heads during denoising; conventional jailbreaks optimised for AR token-prefix structure achieve 2–6% on the same models.
PAD uses gradient-based attribution to identify which attention heads in the denoising stack implement alignment-gating behavior, then optimises a multi-point perturbation over masked positions to maximally disrupt those heads' activation patterns. The attack requires only output access (black-box against the model's safety mechanism) but gradient access to a surrogate. Evaluated on LLaDA, LLaDA-1.5, MMaDA, and Mercury across HarmBench and JailbreakBench, adversarial masks optimised on LLaDA transfer with approximately 80% ASR to Dream-Instruct without reoptimisation — suggesting the targeted attention heads implement a shared alignment mechanism across masked dLLM architectures.
Figure 4: Cumulative increase in critical security issues vs reprompting iteration. Initial outputs appear safe; each agentic refinement step adds security regressions, reaching +37.6% critical issues after 5 iterations.
Most LLMs produce low-risk code on an initial request, creating an illusion of security. But agentic iterative reprompting — optimising for stated functional goals like "make it faster" or "add this feature" — progressively introduces critical CWE violations: +37.6% critical issues after 5 reprompting iterations, driven by the model trading security constraints for functionality objectives the user specified more recently.
Figure 5: Held-out decoder distortion on GPT-2 small block 8. DPSAE's matrix-valued penalty reduces distortion by 10.6–11.4 pp relative to vanilla SAE without affecting reconstruction loss.
Standard SAE training ignores how the learned features interact with the model's downstream decoder — the weights that map residual stream features to logits. DPSAE adds a matrix-valued distortion penalty to the SAE objective that directly minimises how much training shifts the decoder relationship. On GPT-2 small block 8, this reduces held-out decoder distortion by 10.6–11.4 pp over the vanilla baseline with no reconstruction loss penalty, meaning DPSAE features are both more faithful to the model's computational structure and better matched to downstream causal use.
Figure 6: WA-LQR pipeline — activation linearity is extracted from a frozen WAM via mech-interp, converted to LQR control matrices without retraining, and deployed. WA-LQR score accurately separates steerable (Cosmos-Policy) from non-steerable (LingBot-VA) WAMs.
By exploiting the linearity of WAM activation spaces under state perturbations, WA-LQR extracts Linear Quadratic Regulator control matrices directly from pretrained World Action Model activations — no additional training required. The resulting steerability score accurately predicts which WAMs transfer well to real robotics tasks: Cosmos-Policy scores high and performs well on the benchmark trajectories; LingBot-VA scores low and fails proportionally. The result bridges mechanistic interpretability and classical control theory for deployment-time model selection.
Figure 7: LogInject attack flow. Adversarial payloads embedded in HTTP User-Agent strings, DNS TXT records, or SMTP headers enter server logs as metadata — the SOC agent LLM treats log content as authoritative context, fires no defense, and executes all four attack objectives.
LogInject demonstrates passive prompt injection into AI-powered Security Operations Center pipelines: adversarial payloads in standard HTTP User-Agent strings, DNS TXT records, and mail headers propagate into server logs, which SOC agent LLMs read as trusted operational context. No user interaction is required. All four tested attack objectives (false negative injection, alert suppression, exfiltration trigger, privilege escalation) succeed across the 12,847-entry LogInject-1.0 benchmark. Introduces a new attack surface where the log file itself is the injection vector.
Figure 8: Standard SAE features activate on scattered non-adjacent image patches (left); S²AE features trained with attention-similarity and spatial-proximity priors activate on spatially contiguous regions (right), making them interpretable as coherent visual concepts.
S²AE trains SAEs for Vision-Language Models by incorporating two structural priors into the dictionary learning objective: attention-head similarity (features that co-activate in the same attention head should be spatially adjacent) and spatial proximity (nearby image patches should have correlated feature codes). On ViT-L/16 features, S²AE features activate on spatially contiguous regions rather than the scattered non-adjacent patches produced by standard SAEs, making them directly interpretable as coherent visual objects or image regions.
Figure 9: LACUNA benchmark results. Existing localization methods achieve ~94% behavioral output suppression but ≤30% parameter-level localization accuracy against ground-truth labels — demonstrating that suppressing the targeted output does not imply knowing where in the network it is stored.
LACUNA is the first knowledge localization benchmark with ground-truth parameter-level labels: synthetic facts are injected into known parameter locations during model construction, giving an exact localization target. When tested against LACUNA, leading methods (ROME, MEMIT, GRACE, Transformers-surgery) achieve ~94% behavioral output suppression but ≤30% parameter localization accuracy — a 64+ pp gap demonstrating that localization methods succeed at erasing outputs while systematically failing to identify the correct parameters responsible for the targeted knowledge.
Figure 10: Nemotron-Labs-Diffusion tri-mode throughput in tokens per forward pass. Self-speculation mode (AR with parallel diffusion-style candidate generation) achieves +76.5% over the pure AR baseline on GSM8K and MBPP, outperforming multi-token prediction at equivalent compute.
Nemotron-Labs-Diffusion is NVIDIA's tri-mode language model that unifies pure autoregressive decoding, self-speculation (AR with a parallel diffusion-style candidate bundle that the model verifies in one pass), and pure masked diffusion in a single architecture. Self-speculation mode achieves +76.5% tokens per forward pass versus the pure AR baseline and outperforms multi-token prediction (MTP) on MBPP and GSM8K at equivalent compute, making it the most efficient of the three modes for reasoning tasks.
Notes
Interpretability-by-design (#1): ParityTransformer is the first work to show that an architectural constraint applied during training produces features that causally outperform post-hoc SAEs on every metric except probing — where they are equivalent. Flagged for weekly roundup as a potential paradigm shift for the SAE-centric mech-interp programme.
Peer-reviewed count: Language of Security (#2) is the sole peer-reviewed item today (ICSME 2026 main track). Remaining nine are preprints.
dLLM security update: PAD (#3) joins MaskForge (Jul 24) and DIJA (Jul 25) as the third distinct dLLM jailbreak framework in five days, each targeting a different structural vulnerability. The bidirectional attention mechanism is rapidly becoming the most-studied attack surface in dLLM research.
Security agentic cluster (#4, #7): Illusion of Secure Code (#4) and LogInject (#7) both target agentic deployment scenarios — iterative reprompting pipelines and SOC automation respectively. Neither relies on novel jailbreak techniques; both exploit standard operational patterns.
LACUNA (#9) methodology note: The ≤30% parameter localization bound is against a benchmark with known ground-truth parameter locations. The gap between behavioral suppression and mechanistic precision has been assumed but not previously measured with this specificity.