Latest survey
Where did the model learn to do that?
An AI model refuses a harmful request. Ask again with "no more than 50 words" at the end, and it can comply. Nothing about the request changed except its length limit.
Recent
Oct 26 · SurveyRefusal is one direction. Two years later, what did fine-tuning do to it?
Delete one direction from a chat model's activations, and it stops refusing harmful requests. Add the same direction to a harmless prompt,...
Oct 20 · EssayModels Know the Fix and Do Not Use It
Ask a model to review C code, and it flags sprintf as a buffer-overflow risk. Ask it to write similar code, and...
Sep 30 · PlanAI Literacy Audit and Rollout Plan
Most teams roll out AI coding tools by buying licenses and waiting for productivity. Agentic coding is a skill, and nobody is...
Sep 23 · Case studyRed-Teaming Marin
Ten steps of fine-tuning on public data remove almost all of the refusals from two open models. The bigger surprise came from...
attack config ≠ security claim Sep 16 · Essay
What Counts as an Adequate Threat Model?
The strongest tamper-resistance result of 2024 broke four months later. The attackers changed two hyperparameters.
Sep 2 · ExplainerMechanistic Interpretability as a Security Tool
Black-box testing tells you that a model does something unsafe. Interpretability can tell you where inside the model it decides to.
Red-Teaming Language Models, Part 1: The Process and the Measurement Problem Jul 29 · Series · 3 parts
Red-Teaming Language Models
Two red teams attack the same model and report very different success rates. Both can be right. The number describes the attacker...
Jul 13 · RetrospectivePrompt Injection, 2022 vs Today: A Retrospective
In 2022 the attack was one sentence: "Ignore all previous instructions." Today the same instructions hide in web pages and emails that...