Andy Arditi (@andyarditi) / X

X
Andy Arditi

Andy Arditi

221 posts

Andy Arditi

@andyarditi

PhD @ the Bau Lab

Boston

Joined July 2023

RepliesReplies MediaMedia

New to X?

Sign up now to get your own personalized timeline!

By signing up, you agree to the Terms of Service and Privacy Policy, including Cookie Use.

Terms·Privacy·Cookies·Accessibility·Ads Info·© 2026 X Corp.

Don't miss what's happening

People on X are the first to know.

Pinned
Andy Arditi
@andyarditi
Dec 17, 2025
I gave a talk this morning at the MLSec Seminar Series. I discussed the research process leading up to our refusal paper, as well as interesting follow-up work from various groups. youtube.com/watch?v=fN5x8X…
Machine Learning Security Laboratory
@mlsec_lab
Dec 1, 2025
We are excited to present a new event of our seminar series on ML Security! We will host @andyarditi (Northeastern University) on December 17th, 2025, at 5 pm CET. Free registration at the link below: events.teams.microsoft.com/event/9ca660c5…
Andy Arditi
@andyarditi
Jun 18, 2024
Our paper on refusal in LLMs is finally up on arXiv.
Andy Arditi
@andyarditi
Apr 27, 2024
New research post on refusals in LLMs
lesswrong.com
Refusal in LLMs is mediated by a single direction — LessWrong
This work was produced as part of Neel Nanda's stream in the ML Alignment & Theory Scholars Program - Winter 2023-24 Cohort, with co-supervision from…
Andy Arditi
@andyarditi
Dec 10, 2024
Presenting our work on refusals at NeurIPS tomorrow. Come by if you're interested in chatting about interpretability or adversarial robustness!
Andy Arditi
@andyarditi
Jun 18, 2024
Our paper on refusal in LLMs is finally up on arXiv.
Andy Arditi
@andyarditi
May 1, 2024
Great work showing that adversarial suffixes don't reliably transfer between models. My collaborators + I also spent a lot of effort trying to reproduce transferrable suffixes, but ultimately couldn't. Glad @ncmeade + co did diligent investigation to bring clarity on this! Kudos!
Nicholas Meade
@ncmeade
Apr 25, 2024
Adversarial Triggers For LLMs Are 𝗡𝗢𝗧 𝗨𝗻𝗶𝘃𝗲𝗿𝘀𝗮𝗹!😲 It is believed that adversarial triggers that jailbreak a model transfer universally to other models. But we show triggers don't reliably transfer, especially to RLHF/DPO models. Paper: arxiv.org/abs/2404.16020
Andy Arditi
@andyarditi
May 5, 2024
Replying to @panickssery
Final punctuation inside of quotes! english.stackexchange.com/questions/7548…
Andy Arditi
@andyarditi
Feb 17, 2025
Highly recommend working with Neel!
Neel Nanda
@NeelNanda5
Feb 5, 2025
Apps are open for my MATS stream, where I try to teach how to do great mech interp research. Due Feb 28! I love mentoring and have had 40+ mentees, who’ve made valuable contributions to the field, incl 10 top conference papers! You don’t need to be at a big lab to do mech interp
Andy Arditi
@andyarditi
Jun 26, 2024
Entropy neurons!
Alessandro Stolfo
@alesstolfo
Jun 26, 2024
New paper w/ @benwu_ml and @NeelNanda5! LLMs don’t just output the next token, they also output confidence. How is this computed? We find two key neuron families: entropy neurons exploit final LN scale to change entropy, and token freq neurons boost logits proportional to freq 🧵
Andy Arditi
@andyarditi
Jun 18, 2024
Replying to @andyarditi
We show that our methodology is competitive with other SotA jailbreaks, and that it does not significantly degrade model performance. We also present new results, including the surprising finding that the "refusal direction" is also present in non fine-tuned base models!
Andy Arditi
@andyarditi
Jun 18, 2024
Replying to @andyarditi
I'm very grateful to have worked with amazing collaborators on this project: @aaquib_syed1, @dpaleka, @NinaRimsky, @wesg52, @NeelNanda5. And especially @OBalcells, who was in the trenches with me every step of the way!
Andy Arditi
@andyarditi
Apr 27, 2024
Replying to @andyarditi
The "refusal direction" can be found simply by taking the difference between activations on refusals and non-refusals. We can then directly modify the model's weights to never write to this "refusal direction," yielding a jailbroken model that is unable to refuse.
Andy Arditi
@andyarditi
Dec 22, 2024
Building on @ChrisGPotts' excellent lecture, I wrote a short post on thinking about AI as systems rather than just models. This framework seems especially useful in the new age of inference-time compute and reasoning systems. lesswrong.com/posts/2po6bp2g…
Andy Arditi
@andyarditi
Dec 6, 2024
Wonderful talk. Very clear message - think in terms of systems (model, sampling method, prompt, tool use, etc), not just in terms of models.
Andy Arditi
@andyarditi
Jun 18, 2024
Replying to @andyarditi
arxiv.org
Refusal in Language Models Is Mediated by a Single Direction
Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones. While this refusal behavior is...
Andy Arditi
@andyarditi
Oct 2, 2024
This new work from Meta FAIR shows that doing adversarial training with refusal feature ablation (RFA) not only improves robustness against RFA, but also improves robustness against general text-based attacks.
Lei Yu
@jade_lei_yu
Oct 1, 2024
New paper! 🎊 We are delighted to announce our new paper "Robust LLM Safeguarding via Refusal Feature Adversarial Training"! There is a common mechanism behind LLM jailbreaking, and it can be leveraged to make models safer!

Advertisement

Advertisement