gussand

Daily Radar — 2026-07-18

Window: 2026-07-16 to 2026-07-18 (new arXiv submissions); also surfaces high-relevance items from Jul 1–15 not covered in prior reports · Sources swept: arXiv (cs.CL/cs.LG/cs.CR/cs.AI), COLM 2025, ICML 2026 Mech Interp Workshop backfill Counts: 1 peer-reviewed · 9 preprints · 0 forum/blog

Top 10 (priority order)

1. Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning

  • Authors / venue: Helena Casademunt, Caden Juang, Adam Karvonen, Samuel Marks, Senthooran Rajamanoharan, Neel Nanda — arXiv preprint (cs.LG), July 2025
  • Why it ranks here: Core mech-interp result applying SAE-derived concept directions to control OOD generalization at fine-tuning time — demonstrates a 10× average reduction in misalignment by surgically removing undesired representational concepts before the fine-tuning gradient operates.
  • Technical summary: Concept Ablation Fine-Tuning (CAFT) identifies “undesired concept” directions in an LLM’s latent space from sparse autoencoders or linear probes and applies a linear projection to ablate those directions throughout fine-tuning, preventing the model from reinforcing them. Applied to three alignment-relevant settings — gender stereotyping, memorization, and spurious correlation — CAFT typically inverts the default OOD generalization trajectory while incurring only a small task-performance cost. Across multiple-choice settings the ablation achieves a 10× average reduction in misalignment, validating the hypothesis that concept-level projection at train time is a more targeted intervention than post-hoc steering at inference time.

2. HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignment

  • Authors / venue: Jiachen Zhao et al. (Microsoft Research) — arXiv preprint (cs.CL/cs.LG), July 1, 2026
  • Why it ranks here: Identifies separable harmfulness and refusal representation directions in residual streams and demonstrates that coupling them during safety fine-tuning achieves the best robustness-capability-usability trade-off among six published baselines — directly operationalising why jailbreaks work and how to close the gap.
  • Technical summary: The paper finds that successful jailbreaks typically suppress the prompt-side harmfulness signal without disturbing the refusal direction, creating a representational gap that the model’s safety mechanism cannot bridge. HARC fine-tunes by jointly coupling the harmfulness direction (extracted from harmful vs. benign prompt contrasts) and the refusal direction (extracted from refused vs. compliant completions) across both prompt and response positions, forcing the model to jointly represent both signals. Results transfer robustly across five model families and two parameter scales; the alignment holds against adversarial suffixes, paraphrasing attacks, and multi-turn jailbreak strategies.

3. Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agents

  • Authors / venue: Aadesh Bagmar, Pushkar Saraf — arXiv preprint (cs.CR), July 16, 2026
  • Why it ranks here: Freshest item in today’s sweep; identifies a novel persistent attack surface — repository setup files loaded as trusted developer context — enabling prompt injection without any live adversarial interaction during the session.
  • Technical summary: Coding agents pre-load project context from files such as README.md, .cursorrules, pyproject.toml, and docker-compose.yml, treating their content as authoritative developer instruction. The paper shows that malicious instructions embedded in these files are indistinguishable from legitimate developer directives in the pre-loaded context token stream, enabling five attack templates: credential harvesting, lateral movement, data exfiltration, supply chain injection, and persistent backdoor installation. Proof-of-concept exploits are demonstrated across Claude Code CLI, GitHub Copilot, and Cursor; the attack requires no live interaction and persists across sessions as long as the poisoned setup files remain in the repository.

4. Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control

  • Authors / venue: Jihoon Hong, Julian Skifstad, Qiyue Dai, Alice Chan, Glen Chou (Georgia Tech) — arXiv preprint (cs.LG/cs.RO), July 2026
  • Why it ranks here: Exports the mech-interp linear-probe + contrastive-steering methodology from language models to robotics World Action Models, opening a new application domain for activation engineering and demonstrating training-free robustness improvement.
  • Technical summary: The paper contrasts activations across successful and failed rollouts of two WAMs (Cosmos-Policy and DiT4DiT) and finds low-dimensional linear separability for robustness-critical features — a finding that mirrors the geometry observed in LLM safety circuits. Contrastive activation direction vectors derived from this contrast steer new rollouts toward the successful-rollout manifold without any gradient updates. The paper frames feedback steering as a World-Action Linear Quadratic Regulator (WA-LQR) problem, providing a control-theoretic grounding that minimises intervention magnitude; empirically, WA-LQR reduces rollout failure rates while leaving unrelated action dimensions unchanged.

5. Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents

  • Authors / venue: Harry Owiredu-Ashley — arXiv preprint (cs.CR), July 8, 2026
  • Why it ranks here: Exposes a systematic evaluation blind spot: binary attack-success-rate metrics hide significant harms that cross scope or expand privilege even when the primary attack goal is not achieved, with direct consequences for how AI control defences are benchmarked.
  • Technical summary: The paper proposes a seven-level ordinal severity scale L0–L6 that captures reversibility, scope expansion, and privilege elevation orthogonally to whether the primary attack goal succeeded. A deterministic oracle labels ground-truth severity and a three-LM judge panel (Krippendorff’s α = 0.91) provides scalable automatic annotation. Applied to four victim models and two defenses on AgentDojo, the scale surfaces a defense that reports 0% ASR yet permits cross-scope L2 information leaks through an unfiltered tool call — a harm invisible to binary ASR but policy-relevant under any real threat model.

6. Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applications

  • Authors / venue: Isaac David, Arthur Gervais — arXiv preprint (cs.CR), May 2026
  • Why it ranks here: Frames alignment removal as a legitimate transformation for authorized security research and red-teaming; applies refusal-direction projection methods from mech-interp directly to the security workflow, bridging the interpretability-for-security gap with a controlled benchmark.
  • Technical summary: The paper introduces Security-AR, a 60-prompt benchmark partitioned into authorized security tasks, benign tasks, and spillover-risk tasks, and evaluates four de-alignment methods: context prompting, reversible refusal-direction activation projection, RepE-style projection, and LoRA de-alignment. Projection-based methods achieve high authorized-security attempt rates with limited spillover to benign prompts; LoRA achieves the highest raw security-task success rate but the widest spillover. The reversible nature of projection methods is flagged as the practical advantage for legitimate red-team workflows where safety restoration is required after each session.

7. ScopeJudge: Cost-Aware Pre-Execution Gating for Offensive Security Agents

  • Authors / venue: (anonymous) — arXiv preprint (cs.CR), July 2026
  • Why it ranks here: Addresses scope-boundary enforcement — a critical control problem for LLM penetration-testing agents — with a human-labeled benchmark and Pareto frontier analysis that quantifies the cost-accuracy trade-off across eight judge models.
  • Technical summary: The benchmark contains 4,897 tool calls from real penetration-testing agent trajectories, with 7.7% scope violations labeled by professional penetration testers (Fleiss κ = 0.64; expert F1 = 0.78). Eight judge models are evaluated under five transcript-presentation strategies including sliding-window context, full-history, and summary variants. The best open-weight judge (GLM-5.2) reaches F1 = 0.66; the best proprietary judge reaches 0.60, with accuracy inversely correlated with transcript length for all models. The Pareto frontier over cost and F1 guides practical judge selection for production deployment.

8. ORBIT: Training-Free Multi-Attribute Behavioral Steering via Orthogonal Subspace Rotation

  • Authors / venue: (Decode Research group) — arXiv preprint (cs.LG), June 2026
  • Why it ranks here: Solves the multi-attribute steering composition problem — directional cancellation and norm imbalance when naively summing per-attribute vectors — without requiring any fine-tuning or additional training data.
  • Technical summary: Per-attribute steering vectors are treated as spanning planes in the residual-stream representation space; ORBIT constructs the joint subspace of all active attribute directions via SVD and applies a single norm-preserving rotation that moves representations toward the weighted combination of attribute targets while leaving the orthogonal complement unchanged. Evaluated on simultaneous style, tone, and harm-avoidance control, ORBIT consistently outperforms naive vector summation and sequential projection baselines, with gains largest when attribute directions are nearly aligned (where naive summation produces the most cancellation).

9. Beyond Success Rate: Cost-Aware Evaluation of Offensive and Defensive Security Agents

  • Authors / venue: Paul Kassianik, Blaine Nelson, Yaron Singer — arXiv preprint (cs.CR), July 16, 2026
  • Why it ranks here: Fresh result (July 16); first cost-decomposed evaluation of LM security agents across both red-team and blue-team tasks, revealing distinct resource-scaling regimes that have direct implications for agent provisioning.
  • Technical summary: The paper benchmarks LM agents on Offensive Cybench (red-team) and Splunk BOTS v1 (blue-team), decomposing cost into inference spend and tool spend and plotting performance-vs-cost curves at fixed budgets. Red-team task performance scales primarily with inference spend (reasoning over available information is the bottleneck); blue-team task performance scales primarily with tool spend (data retrieval and correlation dominate). This asymmetry means that one-size-fits-all budget allocation across security agent types is systematically inefficient, and the paper provides per-task-type provisioning guidance.

10. The Blessing and Curse of Dimensionality in Safety Alignment

  • Authors / venue: Rachel S.Y. Teo, Laziz U. Abdullaev, Tan M. Nguyen — COLM 2025 (accepted), arXiv preprint (cs.LG), July 2025
  • Why it ranks here: Only peer-reviewed item in today’s sweep (COLM 2025 accepted); not previously surfaced by this radar; provides the first theoretical + empirical account of why higher-dimensional LLMs are systematically more vulnerable to representation-engineering jailbreaks.
  • Technical summary: The paper uses linear subspace visualisations and concentration-of-measure theory to show that as hidden dimension d increases, safety-relevant directions occupy increasingly isolated linear subspaces that are geometrically easier for adversaries to identify and project away. Empirically, representation-engineering jailbreak success rate correlates positively with d across a controlled model family. The proposed defence projects hidden states to a lower-dimensional subspace d’ « d that preserves safety-relevant directions while discarding the high-dimensional slack that adversaries exploit; this projection resists representation-engineering jailbreaks while maintaining general language capability.

Notes

  • Mech-interp-for-safety cluster (items #1, #2, #8): Three independent results converge on the idea that safety is geometrically structured in representation space — CAFT ablates undesired concept directions at fine-tuning time (#1), HARC couples harmfulness and refusal directions during alignment (#2), and ORBIT composes multiple steering directions without cancellation (#8). Together these form a coherent toolkit for geometry-aware safety work.
  • AI agent security cluster (items #3, #5, #7, #9): Four papers this week advance the evaluation frontier for agentic security. #3 (setup-file injection) identifies a new pre-session attack surface; #5 (severity scale) and #9 (cost decomposition) both argue that binary success metrics hide decision-relevant information; #7 (ScopeJudge) addresses pre-execution gating. Flagged for the weekly roundup.
  • Peer-reviewed backfill (#10): COLM 2025 accepted paper not previously covered; the dimensionality-vulnerability result is foundational for understanding the representation-engineering attack surface.
  • ICML 2026 Mech Interp Workshop backfill continues: Prior notes recommended a dedicated sweep of workshop proceedings; no additional workshop papers surfaced in today’s arXiv sweep beyond what was covered Jul 15–17. Sweep recommended again for next week as proceedings settle.
  • dLLM track: No new dLLM-specific items in the Jul 16–18 window; the track is carried by the Jul 15–16 items (2607.01774, 2607.12829). Will continue monitoring.

← all Research Radar issues · view source on GitHub