Alignment researchers have spent years hardening autoregressive LLMs against jailbreaks that exploit the left-to-right prefix. Masked diffusion models look safer by default — they never commit a harmful prefix because there is no committed prefix. DIJA is the first systematic study showing that this intuition is backwards: the bidirectional context window is not a defense, it is an attack surface, and RLHF alignment transferred from AR-LLMs provides essentially no coverage for it.
Figure 1: In AR-LLMs (left), jailbreaks must place the harmful payload as an explicit token — RLHF alignment sees it and refusal fires. DIJA (right) places the payload in masked positions ([MASK]) that are filled bidirectionally at decode time — alignment never encounters the harmful token as input, and the attack succeeds with up to 100% ASR on Dream-Instruct (+78.5 pp over the strongest prior baseline).
DIJA (Discrete diffusion Injection Jailbreak Attack) exploits three structural properties simultaneously: bidirectional context that induces harmful content from surrounding text; the parallel mask-filling mechanism that commits tokens in a single denoising sweep without sequential safety checks; and parallel decoding that limits dynamic filtering capacity. Evaluated on LLaDA, LLaDA-1.5, Dream, and MMaDA, DIJA achieves up to 100% keyword-based ASR on Dream-Instruct and outperforms ReNeLLM by 78.5 pp (evaluator-based ASR on JailbreakBench) and 37.7 pp (StrongREJECT). The result confirms that RLHF alignment methods transferred from AR-LLMs provide insufficient coverage for the dLLM threat model — the bidirectional infilling mechanism creates attack vectors orthogonal to those studied for sequential generation.
Masked diffusion LMs have been closing the capability gap with autoregressive models at 7B scale, but until iLLaDA the best results (LLaDA-8B) still fell well short of Qwen2.5-7B on most benchmarks. ByteDance's iLLaDA is the first masked dLLM to exceed a competitive AR baseline on average across a standard benchmark suite — showing that the remaining gap was engineering, not a fundamental architectural ceiling.
Figure 2: iLLaDA-8B base model surpasses Qwen2.5-7B on MMLU (+2.9 pp), BBH (+7.4 pp), and GSM8K (+3.0 pp), achieving an average of 63.9 vs 63.3 — the first masked dLLM to edge out a competitive AR baseline on average across a standard suite. LLaDA-8B trails substantially on reasoning-heavy tasks (BBH: 49.7).
iLLaDA is an 8B masked diffusion language model trained from scratch with fully bidirectional attention, scaling pre-training to 12T tokens and fine-tuning on a 25B-token instruction corpus for 12 epochs. Key improvements over LLaDA: grouped-query attention, a larger and higher-quality pre-training corpus, and an extended SFT schedule. The base model surpasses Qwen2.5-7B on MMLU, BBH, and GSM8K; the instruct model narrows but does not close the gap with reinforcement-learning-aligned AR models, with the remaining difference attributed to the absence of RL alignment in the dLLM pipeline. This establishes that fully bidirectional diffusion training from scratch is a competitive path to strong language models, not just a niche inference-efficient alternative. Model checkpoint: HuggingFace GSAI-ML/iLLaDA-8B.
Intervention-based circuit discovery works beautifully on small models with small SAE dictionaries but blows up at the scale where deployed models live — path patching and EAP scale as O(k³) in the number of SAE features, making them computationally prohibitive for dictionaries of 16K or 65K features. CircuitLasso reformulates circuit learning as sparse linear regression over feature activations, drops the cost to O(k log k), and matches the structural accuracy of the expensive methods at production scale.
Figure 3: CircuitLasso replaces O(k³) intervention-based patching with sparse linear regression (O(k log k)) over Top-K SAE feature activations, making circuit discovery practical on production-scale dictionaries (16K–65K features). Structural accuracy matches state-of-the-art intervention-based methods; discovered circuits also enable domain generalization at comparable accuracy.
CircuitLasso fits a LASSO-regularized linear regression from upstream SAE feature activations (layer l) to downstream feature activations (layer l+1); non-zero regression coefficients identify circuit edges. Because it is regression-based rather than patching-based, it requires no counterfactual forward passes and scales to the 16K–65K feature dictionaries of production SAEs. On standard circuit benchmarks, CircuitLasso matches the structural accuracy of path-patching and EAP while also recovering explicit semantic dependencies — e.g., which semantic features at layer 8 causally activate which at layer 12. Discovered circuits support a domain-generalization use case: circuits from one distribution provide routing priors that improve OOD performance at comparable accuracy to expensive baselines. IBM Research.
Shadow memory distils safety context per turn; risk-assesses before action execution.
First framework targeting long-horizon threats distributed across multiple agent turns; shadow memory (inspired by shadow stack) retains safety-critical context across the full execution trajectory and enables early-stage detection for the majority of attacks with negligible utility overhead. Stony Brook University / Cisco Systems, May 2026.
Fake safety audit (moderation trace) injected as a tool response overrides RLHF alignment.
Exploits the structural context of function-calling environments — developer-defined schemas and untrusted tool outputs interleaved in the shared model context. Adversaries inject simulated moderation traces (fake safety-audit tool responses) that override RLHF refusal without modifying the user-visible prompt. arXiv July 1, 2026.
TAP (black-box) substantially outperforms GCG (white-box) — inverting the usual gradient advantage.
Systematic evaluation adapting GCG and TAP to agentic settings across 80 task pairs (AgentDojo). TAP outperforms GCG, inverting the usual white-box advantage; task-universal attacks transfer across unseen domains; but attacks optimised on open-source models fail to transfer to frontier models. Prompt injection is a credible but model-dependent threat. arXiv June 2026.
TILG + History Gate enforce proposal stability before committing, reducing denoising steps.
Training-free dLLM decoder combining Temporal Implicit Logits Guidance (exponential moving average of past logits as self-reference) with a History Gate (enforces proposal persistence before commitment). Reduces denoising steps and increases tokens-per-forward on LLaDA, Dream, LLaDA2-Mini with preserved or improved accuracy on code and math benchmarks. arXiv July 3, 2026.
Marker gate defers commitment; last-n layers re-run for joint decoding before argmax.
dLLMs commit multiple positions independently, accumulating factorization error when positions are dependent. CoCommit inserts a marker-gated coordination pass: the backbone's last-n layers re-run on the marked commit set, approximating joint-mode decoding. Reuses existing weights (one extra partial forward pass); improves accuracy on all 6 benchmarks on LLaDA2.1-mini with largest gains on reasoning tasks. arXiv July 5, 2026.
Mid-to-late layers (16–25) consistently produce most vulnerable feature subgroups across all grouping methods.
Applies three SAE feature-grouping strategies across all 26 layers of Gemma-2-2B, amplifies top features per subgroup, and scores outputs for harmful content. Consistent finding: layers 16–25 are substantially more vulnerable to harmful-output steering than early layers — localising jailbreak susceptibility to a specific depth range with implications for layer-targeted defences. arXiv April 2026.
Heads passing CSS + SVD routinely fail transduction under matched controls — role claims don't generalise.
Introduces KID (Knowing/Intent/Doing): three-stage pipeline (CSS → SVD encoding → activation transduction under matched controls). Across 3 instruction-tuned 7–8B models and 5 computation families, heads passing all three checks routinely fail to transfer computation across prompts. The "same-answer control" — a transduction target sharing the answer but not the computation — exposes broad state transfer masquerading as semantic specificity. Philip Quirke (Martian), arXiv June 4, 2026.
Notes
DIJA vs. Hao Peng DIJA (W30 #2): Two distinct ICLR 2026 papers on dLLM jailbreaks — W30 covered 2507.02983 (Hao Peng et al.); today's #1 is 2507.11097 (Zichen Wen et al., DIJA framework with quantitative ASR results across four models).
Substack-style layout: Today's report extends the figure-per-item format to all 10 entries. Items 4–10 carry thumbnail SVG figures in the HTML and direct arXiv HTML image URLs in the .md — same editorial treatment for every paper.
iLLaDA (#2) weekly nomination: First masked dLLM base model to exceed Qwen2.5-7B on average. The remaining instruct-model gap is RL alignment, not architecture — strong W31 candidate.
CircuitLasso (#3): Makes circuit discovery practical at production SAE dictionary scale (65K features); direct unblock for mech-interp research teams running large-scale SAEs.
Ablation-Reversible Heads (#10): Methodological critique — has implications for how the field evaluates mechanistic role claims, particularly for induction heads and copy-suppression heads that have been widely cited.