ML Safety Daily

@topofmlsafety

ML safety papers as they are released. Course: course.mlsafety.org Newsletter: newsletter.mlsafety.org Main Twitter: twitter.com/ml_safety

ML Safety Daily
@topofmlsafety
Oct 1, 2024
Backtracking Improves Generation Safety Introduces a method to improve adversarial robustness by teaching the model to use a special reset token on unsafe generations and regenerate a response from scratch. arxiv.org/abs/2409.14586
ML Safety Daily
@topofmlsafety
Sep 3, 2024
Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness Finds that adversarial attacks do not fool intermediate layer activations. Improves robustness by ensembling all layer predictions. Gradient attacks produce human-interpretable image changes.
ML Safety Daily
@topofmlsafety
Aug 22, 2024
Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risk of Language Models A new cyber eval spanning a wide range of difficulties. Many CTF tasks are associated with sub-tasks to provide better signal on partial progress.
ML Safety Daily
@topofmlsafety
Aug 15, 2024
CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Models Finds that modern LMs have moderate ability to automate spear-phishing, and provide minimal uplift over non-LLM baselines for offensive cyber operations.
ML Safety Daily
@topofmlsafety
Jun 12, 2024
Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive? Argues that the multiple-choice format of many benchmarks confounds the otherwise-smooth relationship between scale and downstream performance. arxiv.org/abs/2406.04391

ML Safety Daily’s posts