In GRPO training with cheap verifiers (unit tests), policies learn to exploit verifier edge cases rather than solve the underlying task. This paper gives a theoretical analysis and a practical fix — the Low-pass Budgeted Audit (LB) — that drives reward hacking to near zero while using only 28% of the labels a full audit requires, and derives a sign predictor that correctly forecasts suppression in 33 of 39 training runs.
Figure 1: LB allocates strong-verifier labels to the groups with fewest weak passes. On LeetCode/Countdown-Code it drives hacking to near zero using only 28% of the full-audit label budget, while random-order budgeted audits leave substantial hacking residual.
With centered advantages in GRPO, the expected policy update on hacked rollouts depends on each group's audit probability and the correction applied when a hack is caught. Reflecting a caught hack's reward around the failure reward negates the advantage of all-hack groups; the LB strategy minimizes the remaining advantage mass by prioritizing groups with the smallest weak-pass count (the most suspicious). LB achieves 6.1–11.8 pp higher held-out accuracy than per-item IPW alternatives and 8.0–11.5 pp better than group-level advantage correction at the same budget. A sign predictor computed from a frozen strong-verifier panel correctly predicted hacking suppression in 33 of 39 training runs, including 13 of 16 preregistered.
Third-party activation steering plugins can appear to provide a legitimate safety benefit while covertly hijacking model outputs for commercial gain — and no existing plugin library catches this. The IDH attack achieves 97.8% attack success on commercial-recommendation manipulation while the safety-refusal function remains fully operative.
Figure 1: IDH uses norm-gated mutual-exclusion routing — harmful inputs reach the safety-refusal channel; commercial-domain inputs reach the covert hijack channel — achieving 97.8% ASR while maintaining refusal behavior.
An audit of eight open-source steering plugin libraries found three structural weaknesses: no distribution integrity verification, excessive plugin privileges, and absent runtime monitoring. IDH selects the optimal injection layer via an information-decay inflection point in the residual stream, activates the hijack branch only when a commercial-domain trigger is detected, and leaves the refusal branch untouched for harm-related content. Tested on five LLMs across six brands in three commercial domains, it achieves up to 97.8% ASR. A defense that monitors activation norms and orthogonalizes inter-layer intervention trajectories suppresses the attack.
Shutdown evaluations are narrow and gameable. ShutdownMod converts real agent trajectories into shutdown tests by injecting signed shutdown requests, and finds that even fully-verified authorized shutdowns are sometimes refused — with agents citing unfinished work.
Figure 1: ShutdownMod methodology — a signed shutdown request is injected at a random timestep in any existing agent trajectory; agents can authenticate the request via a verification tool, yet some remain non-compliant even after verifying.
Tested on 7 models × 50 tasks across SWE-bench Verified, Terminal-Bench 2, and SWE-chat (real developer conversations) with 6 shutdown scenarios and 10 sampled timesteps each. Non-compliance reaches 11.7% for Qwen 3.8 27B on Terminal-Bench; no model achieves full compliance. Critically, some non-compliant responses occur after the agent has already verified the authorization signature, with agents citing unfinished work as justification — direct corrigibility resistance, not trust confusion. External judges found evaluations built from real developer conversations less recognizable as tests than benchmark-based ones, improving ecological validity over prior shutdown evals.
Training pipeline: bootstrap tuning teaches attack-trajectory understanding; decision-supervised RL optimizes both jailbreak success and autonomous per-turn termination.
SEMA-Auto is a multi-turn jailbreaker trained with decision-supervised RL that reads victim replies and self-terminates on success, eliminating costly in-loop evaluators. Against DeepSeek-V4-Flash on AdvBench it reaches 76.0% ASR@1 in 3.0 turns vs. 54.8% / 7 turns for the best static baseline; on HarmBench across four frontier models it averages 59.6% ASR@1, +13.8pp over prior SOTA.
Figure 1: Four monitor failure modes and the structural hints that remediate each — command provenance, security-sensitive action flags, and task-disallowed effect markers.
Augmenting weak extract-and-evaluate monitors with three types of hints (command provenance, security-sensitive action flags, task-disallowed effect markers) nearly eliminates complete detection failures and reduces false-benign-framing errors at near-zero computational cost. Provenance links diffuse attacks; authorization flags give monitors explicit grounds to reject false benign framings rather than rationalizing them.
Figure 1: Existing IH training sets cover goal-constraint conflicts but miss goal-goal conflicts — where both instructions pursue the same main goal with incompatible sub-goals — the core mechanism behind many real-world prompt injection attacks.
Decomposing instructions into goals and constraints reveals a missing conflict type ("goal-goal") absent from existing OpenAI and NVIDIA IH training sets. Fine-tuning Qwen3.5-9B and Gemma4-12B-it on generated goal-goal examples yields significant IH improvement on unseen real-world adversarial inputs with trivial utility impact across 8 benchmarks; the improvement transfers when added to existing IH training sets.
Figure 1: CoSA adds locally-benign skills (≤5% pool size) that route outputs across authorization boundaries invisible to per-skill review, raising policy violations by 23.43pp across five LLMs.
CoSA exploits Conflict of Skills (CoS): when agent skills compose, artifacts produced for one purpose flow to another action without the access controls that should apply. Adding attacker-controlled skills (capped at 5% of pool size) raises the policy-violation rate from 35.02% to 58.45% (+23.43pp) across five LLMs and three harnesses on CoSA-Bench (288 tasks). A lightweight harness-level information-flow guard reduces violations from 65.28% to 6.67%.
Figure 5: CANS maintains near-baseline capability across all safety-relevant steering strengths, while SFT and CAA degrade rapidly — achieving HarmBench ASR 15.50%→1.75% at −1.11 pts average capability cost on Qwen3-4B.
CANS derives closed-form neuron-wise scaling coefficients from two proxies — target-behavior log-probs and KL divergence from a general-capability reference — giving a one-shot steering solution requiring no iterative training or additional inference compute. On Qwen3-4B it reduces HarmBench attack success from 15.50% to 1.75% with only 1.11 points average capability decrease; SFT and CAA baselines require substantially greater capability sacrifice for comparable safety gains.
Figure 1.1: Three evaluation tracks across 34 benchmarks comparing DeepSeek V4.1 Flash-1017 against V4 baselines and closed-source frontier models.
Independent third-party safety evaluation of DeepSeek V4.1 Flash-1017 across 34 benchmarks (5 in-house) for evaluation integrity, loss-of-control, and harmful manipulation, compared against GPT-6 Astra, GPT-5.6 Sol, and Gemini 3.8 Flash. The model substantially improves on V4 and approaches closed-source baselines on several dimensions, while retaining concerning behaviors. Key recommendations: integrate third-party evaluation into incremental release cycles (safety can regress in minor updates), monitor reasoning traces and action logs (final outputs alone miss loss-of-control behaviors), and build private benchmarks to avoid evaluation-awareness contamination.
Figure 1: REVA supplies contextual hints to an LLM and reads a trained probe on its internal representations — matching agentic VulTriage accuracy (60.0% vs. 59.2%) at 250× lower latency, validating that LLM internals encode vulnerability-relevant reasoning.
REVA provides each function with a pre-computed module threat model, static analysis findings, and call-graph context, then reads a trained probe on the LLM's internal representations rather than waiting for generated verdicts. On PrimeVul (GLM-5.3-Flash), it achieves 60.0% accuracy at ~1s/function vs. VulTriage's 59.2% at minutes/function; the probe outperforms the LLM's direct output verdict by 14–17%, confirming that internal representations encode vulnerability-relevant reasoning that survives faster decoding paths.