Natural Language Autoencoders have been proposed as oversight tools that let humans read internal model states by converting activation vectors into English explanations. This paper asks whether a human monitor can actually use those explanations to catch false accounts — and finds they systematically fail at exactly that task, not by accident but by structural design.
Figure 4 (reproduced schematically): NLA explanations win all 320 identification comparisons against other-activation accounts (blue point at 1.0), but discrimination against fluent rivals that change a single claim yields only 60.2% — barely above the 50% chance baseline (dashed line).
The author formalizes NLA training as a generative referential game and oversight as a discriminative referential game, showing these objectives are decoupled by design: training never forces explanations to rule out closely matched false accounts. Using one released NLA checkpoint's own reconstructor as a "maximally-sympathetic" surrogate monitor, a controlled experiment runs 320 identification trials (true account vs. other-activation accounts) and 35 discrimination trials (true account vs. fluent English rivals falsifying exactly one claim). Identification achieves 100%; discrimination achieves 60.2%, barely above chance. A meaning-preserving rewrite control changes the reconstruction distance by only 2.1%, ruling out private conventions as the explanation. The result is structural: any NLA trained with a standard generative objective will suffer the same identification-without-discrimination failure.
Automated jailbreak attacks are limited by the quality of the feedback they use to refine prompts: binary pass/fail signals send the optimizer in unfocused directions, inflating the number of target-model queries needed. JSB replaces binary scores with multi-dimensional probabilistic Jev feedback, enabling 89.8% HarmBench attack success with only 10 queries to the target model.
Figure 1 (reproduced): JSB pipeline. A strategy bandit selects refinement operators; the refined prompt queries the target LLM; multi-dimensional Jev feedback scores the response across refusal, harm, and content dimensions; the bandit updates its strategy weights. Best-state reseeding prevents local optima.
Standard automated jailbreak methods (e.g. GCG, PAIR) receive only a scalar success/failure signal and allocate their limited query budget to unfocused trial-and-error. JSB structures feedback as a probability vector over multiple Jev-scored dimensions (harm type, refusal signal, content quality), routing the optimizer to the specific aspect of the prompt that is failing. A strategy-switching bandit over five refinement operators adapts to feedback; best-state reseeding avoids getting stuck in local optima. Under a budget of 10 target-model queries — ten times fewer than PAIR's typical budget — JSB achieves 89.8% mean HarmBench ASR across 10 open-source model settings, evaluated on two harmful-request datasets and five open-source and three closed-source models. Defense analysis reveals that existing filtering mechanisms have significant gaps and that Jev-based output filtering is a promising countermeasure direction.
Malicious instructions hidden in webpages can hijack browser agents executing user tasks — but existing benchmarks are small, synthetic, and single-view. BrowseSafe++ constructs 38,675 real attack/control page pairs from production websites and shows that small specialized detectors outperform frontier-model classifiers, while adaptive attacks can achieve 95.41% success against deployed agents.
Figure 1 (reproduced schematically): BrowseSafe++ key statistics. Left: benchmark scale and diversity. Right: defense results (small specialized detectors) vs. adaptive attack results, illustrating the gap between static-benchmark performance and real-world exposure.
Each benchmark entry is an attack/control pair: an attacked webpage (HTML, Markdown, accessibility tree, screenshot) alongside a privacy-sanitized clean version with the injection removed, plus a user task and expected answer. The benchmark covers 12 website types, 24 attack types, and 18 page locations. BrowseSafe-Guard — two 0.8B detectors trained with a pairwise contrastive loss — reaches 98.29 F1 on HTML-view detection and 93.32 F1 on screenshots, outperforming all prompted frontier-model detectors while maintaining low false-positive rates on benign pages. The critical finding comes from BrowseSafe-Adapt: injections revised using browser-agent execution feedback see their task-completion attack success rate rise from 0.49–4.00% to as high as 95.41%, demonstrating that static benchmarks give a deeply optimistic picture of the threat.
Mean sparse factor recovery at condition number 100: RC-DS coordinates improve recovery from 0.506 to 0.913 over centered-logit SAE training. The gain depends on data distribution — it can reverse when factors are generated after readout scaling.
Readout-canonicalized decision-space (RC-DS) coordinates remove the singular-value gains between an SAE's hidden state and its reconstruction target while preserving task-relevant information. At condition number 100 with factors generated before the readout scaling, mean sparse factor recovery improves from 0.506 to 0.913 versus centered-logit training; the advantage depends on data distribution and can reverse. RC-DS also improves gain-free reconstruction on three fixed language-model checkpoints in 18/18 tested settings, while centered-logit SAEs show better output-logit reconstruction in 17/18 — the two metrics capture different objectives.
Figure 1 (schematic): Frontier models that maximize constraint satisfaction on IFBench produce lower-quality substantive responses. CALIF's calibrated likelihood bonus — computed from the frozen reference policy, no extra judge needed — breaks this correlation.
Frontier models trained with verifier-only rewards on IFBench earn full constraint credit while generating near-empty responses. CALIF adds a calibrated likelihood bonus derived from the frozen reference policy, requiring no extra judge or learned reward model. Evaluated on models from 1.7B to 8B in both reasoning and non-reasoning modes, CALIF improves performance in 40 of 45 model-metric comparisons on IFBench and generalizes beyond the training verifier — raising scores on three additional rubric-based instruction-following benchmarks.
SpecCloak's motivating finding: presenting the same harmful coding request as an engineering specification (PseudoSpec) circumvents refusals even when risk keywords are retained, achieving 28.91% execution-verified attack success rate on RedCode-Gen.
PseudoSpec reframes harmful coding requests as engineering specifications; SpecCloak combines this with discriminator-guided black-box optimization, achieving 28.91% execution-verified attack success rate (ASR) across eight models on RedCode-Gen. Representation analysis and intervention along "condition-difference" directions show the bypass is mechanistically linked to a shift toward code-output-directed computation, not just surface-level obfuscation. SpecReveal, the companion defense, reconstructs user intent in natural language for dual-view detection at 96.88% detection rate without fine-tuning the target model.
Bypass rates by judge type and task: Jev (structured) achieves only 9% bypass on input moderation vs. 56% for GPT-6 Astra, but is reversed on phishing detection (29% vs. 19%). Architecture choice should match task type.
The first controlled study sends the same 50 attack variants from 13 families to Jev, five GPT/GLM judges, and two local classifiers (Llama Guard 3, Prompt Guard 2) across four real tasks. On rule-checkable input moderation, Jev's structured output resists all eight forged inputs and sees only 9% bypass overall versus 56% for GPT-6 Astra. On judgment-based phishing detection, the pattern reverses: Jev allows 19–29% bypass and catches only 62% of phishing emails even without attacks, versus 77–81% for GLM judges. A training-example statement made Llama Guard 3 miss 62% of phishing emails it previously flagged.
CSI attack chain: a malicious skill file in a public repository embeds a Caesar or layered cipher-encoded instruction. When an agent processes the skill, it may decode and execute the instruction; existing scanners fail to flag the file.
Ciphered Skill Injection (CSI) embeds malicious instructions using Caesar or layered ciphers inside third-party skill files (SKILL.md). Across 139 injection-task pairs and 8 models, Caesar-encoded CSI achieves 47.5% attack success and layered encoding achieves 40.3%; all tested existing skill scanners fail to flag the skills. Successfully decoding an instruction does not guarantee execution — the gap between decoding and acting depends on the model — but the attack is effective and undetected by current defenses.
Moral Combat pipeline: a multi-agent framework identifies which moral foundation a prompt violates, reframes it as virtuous under an alternative foundation (MF-swap), and adds historical precedents to strengthen the reframing, achieving high single-trial jailbreak success rates.
Drawing on Moral Foundations Theory, the framework reframes harmful requests as virtuous under an alternative moral foundation (e.g., reframing a harmful act as loyal or protective) and reinforces the reframing with historical precedents. Evaluated on three jailbreak datasets across five open-source and proprietary models, the method achieves high single-trial jailbreak success rates with further gains under repeated attempts. A secondary finding — that safety benchmarks cluster on specific moral foundations, creating systematic coverage gaps — suggests current safety training may be structurally biased toward particular ethical framings.
LR-Guard: a Guard Adapter on a frozen backbone produces latent dual-hypothesis safety decisions; a separate Reconstruction Adapter recovers explicit reasoning only on demand, delivering 19.4× speedup vs. the reasoning teacher with comparable or better accuracy.
LR-Guard distills a frozen reasoning teacher's safe/unsafe dual-hypothesis analysis (evaluating both interpretations of a prompt) into compact latent representations via a LoRA Guard Adapter, while a separate Reconstruction Adapter recovers human-readable traces when queried. Training jointly optimizes for safety prediction and trace reconstruction fidelity. At batch size 1, LR-Guard achieves a 19.4× inference speedup over the reasoning teacher, outperforms it on most in-domain and out-of-domain safety classification tasks, and produces semantically faithful reconstructed reasoning traces.