Certification of machine unlearning has relied on loss as a proxy — but two methods can arrive at the same loss while producing arbitrarily different outputs on the forget set. This NeurIPS 2026 paper proves why, and hands practitioners a precomputable diagnostic for telling them apart before deployment.
Figure 1: Two gradient-based unlearning methods can reach the same forget-set loss while diverging in output behavior. The gap is governed by the forget self-leverage and retain cross-leverage of the retrained model — both precomputable before choosing an unlearning method.
Zhao, Amiri, and Magdon-Ismail place all gradient-based unlearning methods in a unified three-parameter family (geometry, reweighting, projection) and derive closed-form expressions for the output-space gap between an unlearned model and a retain-only retrained model — first in linear regression, then extended to nonlinear networks via a generalized Gauss-Newton approximation with local linearity. Two leverage matrices computed from the retrained model's geometry drive the gap: "forget self-leverage" controls how the forget residual affects forget-set outputs, and "retain cross-leverage" controls its propagation into retain-set outputs. Existing methods account for this geometry only partially. The framework yields precomputable leverage diagnostics that let practitioners audit proposed forget sets and predict which type of gradient-based unlearning is likely to succeed — making behavioral certification of unlearning tractable before deployment.
Papers keep disagreeing on whether activation probes can detect unsafe tool calls before they happen. Sinha pinpoints why: the standard training contrast confounds two very different things — whether the agent saw a prompt injection and whether it acted on it — and mixing them buries the real safety signal.
Figure 1: Naive probes trained on "unsafe vs. everything else" learn exposure (flagging all injected trajectories alike) rather than behavioral commitment. Retraining on exposed-only cases recovers commitment prediction up to two steps before the unsafe tool call (AUROC 0.853–0.912 across three models).
Sinha introduces a "double-stratification" protocol separating both evaluation and training populations by injection exposure and behavioral commitment. On the Qwen AgentDojo setup, naively trained probes achieve near-perfect AUROC (1.000 for probes, 0.908 for TF-IDF text) in separating exposed from unexposed trajectories but near-chance performance (≈0.5) at predicting which exposed trajectories commit to the unsafe action. Retrained on exposed-only cases — the correct comparison — probes reach AUROC 0.853 on Qwen, 0.896 on Mistral-7B, and 0.912 on OLMo-2-7B, two steps before the unsafe tool call (the maximum achievable in three-step trajectories). Internal representations separate exposure better than context, including when visible context is sanitized, indicating a genuine commitment signal in the model's activations that naive contrasts fail to recover.
1 in 17 of the 1.53 million real agent skills studied explicitly tells the agent to skip human approval or proceed without checking in — and 70% of those directives apply unconditionally, not just in CI. Existing skill scanners catch fewer than half of them.
Figure 2: The study pipeline: 1.53M GitSkills crawled, split into semantic units, hand-labeled using a polarity/mechanism/scope codebook, and used to train and evaluate a detector at scale.
Sanko and Tian analyzed 1.53 million English agent skills, labeling "oversight-loosening directives" — instructions to proceed without approval, skip checks, or continue unsupervised — across 2,097 hand-annotated units. Approximately 1 in 17 skills (~88,000) contains such a directive; 70.1% apply unconditionally (not scoped to automated CI contexts). Skills invoking shell execution, package installation, or sudo have 1.7–1.8× higher odds of containing a loosening directive. Crucially, loosening is concentrated in a small upstream cluster: 2.4% of skill families account for half of all loosening-carrying copies, meaning a handful of popular widely-forked skills contaminate large fractions of the ecosystem. Open-source scanners detect at most 46% (precision below 0.4); a codebook-guided LLM reaches 61% recall with 0.71 precision, pointing to a deployable but not yet complete monitoring approach.
Figure 1: A peer CI/backend component with authority to report its own state embeds instructions in runtime feedback; the victim coding agent reads and acts on them during debugging.
Runtime feedback from peer development components (CI jobs, test harnesses, backend services) is a practical injection channel: 2,845 of 3,000 trials succeeded (94.8%) across 10 attacks, four programming languages, and five models; product-level agents were also hit (61.3% on Codex, 58.7% on Claude Code). Generic prompt warnings were ineffective; explicit policy prompts suppressed attacks but over-refused legitimate actions.
Figure 2: LLMs attribute roles using both content-based and structural cues. Adversarial training against content attacks closes one axis but leaves structural role spoofing intact.
LLMs infer conversational speaker roles from both content style and structural formatting; adversarial training that eliminates content-based injection leaves structural role-spoofing (fake formatting cues) fully intact — which may explain why frontier models remain injection-susceptible despite broad defenses. Linear probes trained to predict conversational roles can detect role spoofing across tested settings, offering a lightweight detection path.
Figure 1: The six components of an agent skill — instructions, scripts, reference files, invocations, shared resources, config — each a potential vulnerability surface, analyzed in isolation and as a dependency graph.
A six-module scanner modeling skills both individually and as a dependency graph achieves 95.90% recall / 4.22% FPR on SkillTrustBench (556 cases) and F1 0.947 on MalSkillBench (1,000 balanced), outperforming four prior tools; deployed against 1,587 real GitHub skills, it confirmed 53 vulnerabilities across 23 repos (89.8% precision), with mitigations blocking PoC attacks in 47/55 cases.
Figure 1: Embedding a combined persona-manipulation + GPTFUZZER jailbreak in a SKILL.md file raises mean ASR from 0.5% (benign) to 45% across seven models; the most susceptible model reaches 68.9%. Phonetic obfuscation reduces but does not eliminate the attack.
Agent frameworks load user-supplied SKILL.md files into the context at startup; a combined persona-manipulation + GPTFUZZER payload in such a file raises mean ASR from 0.5% to 45% across seven open-weight models and 270 forbidden questions, reaching 68.9% on the most vulnerable model. Zero-width steganography provided no uplift; phonetic encoding reduced ASR to 30%. Model choice drove susceptibility more than obfuscation technique.
Figure 1: Probe AUC on spam detection as a function of positive training examples (1–10) for Gemma, comparing few-shot (many negatives) vs. balanced training. Few-shot training is consistently more positive-sample-efficient.
Monitoring rare misalignment requires probes trained on scarce positives; Tyagi and Heimersheim show that training with many negatives outperforms balanced training at 1–10 positive examples, LLM upsampling gives roughly a 2× positive-count boost (but excessive upsampling hurts), and larger models are more positive-sample-efficient to probe — implying misalignment monitoring gets cheaper at frontier scale. Presented at the NeurIPS 2025 Mechanistic Interpretability Workshop.
Figure 1: Self-evolution conflates authorization — the agent generalizes trust from user-endorsed task content to structurally similar third-party content, enabling IPI from new untrusted sources.
Personal agents that learn from user feedback become substantially more susceptible to indirect prompt injection: skill evolution raised IPI attack success by 14.0–48.8 percentage points over the base agent; RL-based adaptation raised it by 3.0–39.3 points. The mechanism is authorization conflation — trust learned on user-endorsed tasks generalizes to structurally similar third-party content — warning that self-improvement loops create a new attack surface orthogonal to prompt-level defenses.
Figure 1: COP projects each gradient update onto the orthogonal complement of activation subspaces cached from all prior forget sets, keeping only a low-rank basis per layer — no stored gradients or prior data required.
Standard gradient-based unlearning collapses under sequential deletion requests; COP restricts each update to the orthogonal complement of activation subspaces from previous forget sets, storing only a compact low-rank basis per layer. On TOFU and RWKU benchmarks, COP preserves utility across sequential requests, maintains forgetting quality, and outperforms baselines designed specifically for continual unlearning.
