Sharing a little late update (before itβs no longer news): I wrapped up my PhD at the end of last year and recently joined @OpenAIβs reasoning team πβ¨!
- π€ Q: How do you find low-quality data? π‘ A: Corrupt the good ones and watch where they go! Sharing this simple yet generalizable data pruning idea I worked on this summer at the ENLSP workshop #NeurIPS2023 as an *Oral* presentation! arXiv: arxiv.org/abs/2312.02418 (π§΅1/)
- π Two of my papers have been accepted this week at #ICLR2024 & #AISTATS! Big thanks and congrats to co-authors @xxchenxx_ut & Eric Gan, mentors Atlas Wang & @gkdziugaite, and especially my advisor @baharanm! π More details on both papers after the ICML deadline!
- We released *DuoGuard*, a 0.5B multilingual guardrail model! Its two-player RL framework adversarially co-evolves a generator and classifier to generate safety data. DuoGuard offers fine-grained probabilities across 12 subcategories with customizable thresholds. Check it out! π₯³New paper & model release! Excited to introduce DuoGuard: A Two-Player RL-Driven Framework for Multilingual LLM Guardrails, showcasing our new DuoGuard-0.5B model. - Model: huggingface.co/DuoGuard/DuoGuβ¦ - Paper: arxiv.org/abs/2502.05163 - GitHub: github.com/yihedeng9/DuoGβ¦ Grounded in a
- 1/ I'll be at #NeurIPS2024 presenting our work SmallToLarge (S2L): Data-efficient Fine-tuning of LLMs! π Whatβs S2L? Itβs a scalable data selection method that trains a small proxy model to guide fine-tuning for larger models, reducing costs while preserving performance. π
- Excited to share our #NeurIPS2023 paper tackling spurious correlations in machine learning! Grounded in theoretical analysis, our PDE algorithm improves efficiency and worst-group accuracy under group imbalances. Discover more in our code and project page!πCodes and project page are released for our #NeurIPS23 paper on spurious correlations in robust learning! π π Project: uclaml.github.io/PDE/ π Code: github.com/uclaml/PDE/ Key Insights: π We discovered in theoretical analysis that spurious features overtake initial
- Excited to introduce SecCodePLTπ‘οΈ: a unified platform for evaluating security risks in code generation AI! Since summer, weβve been building a comprehensive tool to assess AI models' potential for insecure coding and facilitating cyberattacks. π§΅1/π
- Honored to have my work featured alongside others from our lab in the #ICML2024 tutorial on Foundations of Data-Efficient Learning! The tutorial was well-designed and comprehensive on theoretically grounded dataset curation techniques. Recording is out for anyone interested!ππ’Excited to share the recording of our #ICML2024 Tutorial on Foundations of Data-Efficient Learning: youtu.be/30VkdWuwmdA Truly grateful to everyone who attended β it was incredible to see the enthusiasm for theoretically principled techniques for dataset curation!
- ποΈWe are thrilled to announce that we will be presenting our latest paper with @besanushi @hmd_palangi @baharanm (arxiv.org/abs/2304.03916) at #ICML2023 ! π Join us as we share insights and solutions for β¨spurious correlations in vision-language modelsβ¨. (π§΅1/8)
- Itβs been a new and exciting experience to be part of founding @VirtueAI_co! Iβve had the privilege of working with top minds in the fields β I'm incredibly grateful for this invaluable experience. Check out our website and blogs, and come hang out with us in SF this summer! π₯³πReplying to @uiuc_aisecureInterested to learn more? Please contact us virtueai.com/contact-us/ and talk to our team. More to come! @sanmikoyejo @dawnsongtweets @guestrin @EasonZeng623 @MinzhouP @YUYANG_UCLA
- π’Our (Hao Kang, @baharanm) paper (arxiv.org/abs/2306.01244) will appear at #ICML2023! Introducing CREST: the first coreset selection algorithm theoretically guaranteed to speed up training of deep neural networks!π(π§΅1/7)
- Our finding shows longer code files are often lower quality, and pruning these files can significantly enhance performance. Excited to have contributed to this project led by @Aaditya6284, extending my internship work @AIatMeta! π Check out our paper π arxiv.org/abs/2407.00434Long (code) files may not be as high quality as you thinkβ¦ Excited for our new work, "Brevity is the soul of wit: Pruning long files for code generation". We find that long code files are often low quality and show benefits from pruning such files for code gen. Read on πβ¬
- Excited about training on synthetic data? Different stages of training might need different synthetic data! π§ π‘ Check out our #ICLR2024 paper on Progressive Dataset Distillation (PDDπ) at PS#2 Halle B#9! It tailors synthetic data to each training stage for better performance!Youβre unable to view this Post because this account owner limits who can view their Posts. Learn more
- Don't miss our poster session today at #NeurIPS2023! π€@Yihe__Deng will be presenting our work on "Robust Learning with Progressive Data Expansion Against Spurious Correlation." π Great Hall & Hall B1+B2 (level 1) #707 β° 5:15 p.m. - 7:15 p.m. CST π neurips.cc/virtual/2023/pβ¦Happy to share that our work "Robust Learning with Progressive Data Expansion Against Spurious Correlation" has been accepted to #NeurIPS2023! π arXiv: arxiv.org/abs/2306.04949
