Log inSign up
user avatar
Anna Rogers
@annargrs
Associate professor @ITUkbh: LLM interpretability, generalization, AI & society. Co-editor-in-chief @ACLRollingReview. Blog: hackingsemantics.xyz
Copenhagen, Denmark
annargrs.github.io
Joined September 2018
874
Following
9,032
Followers
  • Pinned
    user avatar
    Anna Rogers
    @annargrs
    Aug 13, 2024
    📢 AI 'news' are easy to make and hard to detect! Did you know that there's already 1K+ scammy LLM-powered 'news' sites? In #ACL2024 paper led by @gpuccetti92, we show that such 'news' are too easy to produce (even beyond English) - and nearly impossible to detect 'in the wild'.
    Image
    13K
  • user avatar
    Anna Rogers
    @annargrs
    Mar 6, 2022
    ❌"NATO expansion to the East" ✅ "Eastern Europe leaving the ex-empire" The NATO frame makes it about Russia vs NATO, denying agency to Eastern European countries. The fact is: they've all been jumping the Soviet ship for decades, at the first opportunity. It's kinda clear why.
  • user avatar
    Anna Rogers
    @annargrs
    Feb 28, 2020
    Have you been drowning in BERT papers? We have. So... the first ever... Primer in BERTology is out! arxiv.org/abs/2002.12327 We survey over 40 papers on BERT's linguistic knowledge, compression, architecture tweaks, multilinguality, and more! With OlgaKovaleva & @arumshisky.
    arXiv logo
    arxiv.org
    A Primer in BERTology: What we know about how BERT works
    Transformer-based models have pushed state of the art in many areas of NLP, but our understanding of what is behind their success is still limited. This paper is the first survey of over 150...
  • user avatar
    Anna Rogers
    @annargrs
    Feb 26, 2022
    📢 tech & ethics folks: I've collected 🧵 of pointers to official statements and discussions on what the big tech has done/should do for 🇺🇦 war. Again, this is not just about 🇺🇦: what we do now sets precedent for the future. Comments, updates & corrections welcome! /1
  • user avatar
    Anna Rogers
    @annargrs
    Jul 19, 2019
    Here's a summary post on problems with huge models that dominate #NLProc these days. I put together several different discussion threads with/by @yoavgo, @jaseweston, @sleepinyourhat, @bkbrd, @alex_conneau, @SeeTedTalk.
    Image
    How the Transformers broke NLP leaderboards
    From hackingsemantics.xyz
  • user avatar
    Anna Rogers
    @annargrs
    Mar 4, 2024
    🥳🍾 It's official - I'm a tenured associate professor! This job is incredible luck and privilege, and @ITUkbh is an amazing place to work at!!! More PhD student and postdoc positions will be announced soon.
    15K
  • user avatar
    Anna Rogers
    @annargrs
    Jan 16, 2020
    A belated blog post for our BERTology EMNLP paper (by Olga Kovaleva, Alexey Romanov, yours truly and @arumshisky). My favorite experiment in this work is showing that for most GLUE tasks BERT works pretty well even *without pre-training*!
    text-machine-lab.github.io
    The Dark Secrets of BERT
    BERT and its Transformer-based cousins are still ahead on all NLP leaderboards. But how much do they actually understand about language?
  • user avatar
    Anna Rogers
    @annargrs
    Dec 10, 2023
    @chrmanning at #EMNLP2023 to #NLProc PhD students, who are having an existential crisis over LLMs: Aeronautics students do not build Boeings for their PhD theses. They do smaller models - and still make meaningful contributions. There's plenty of such opportunities for us too.
    Image
    Image
    46K
  • user avatar
    Anna Rogers
    @annargrs
    Jun 14, 2022
    Life update: I'm now an assistant professor at @CPH_SODAS! Very grateful to be in the right place to learn from all those psychologists, anthropologists, social & political scientists - all the people who actually know what we're baking into NLP models, apart from language.
  • user avatar
    Anna Rogers
    @annargrs
    May 31, 2021
    🎈 #NLPaperAlert: Changing the World 🌍 by Changing the Data 🗃 arxiv.org/abs/2105.13947 A soul-searching piece that made it to ACL 2021: - how NLP resources affect the world - what does it even mean to 'work in NLP' - how we can make better use of our subcommunities. /1
  • user avatar
    Anna Rogers
    @annargrs
    Apr 3, 2023
    Will reviewers now ask "where's GPT4?" We propose a #BenderRule-like rule: That which is not open & reproducible* can't be a requisite baseline hackingsemantics.xyz/2023/closed-ba… w @b_niranjan @LeonDerczynski @JesseDodge @alkoller @SashaMTL @MaartenSap @royschwartzNLP @nlpnoah @strubell
    Image
    Closed AI Models Make Bad Baselines
    From hackingsemantics.xyz
    162K
  • user avatar
    Anna Rogers
    @annargrs
    Apr 29, 2023
    > ChatGPT was more empathetic than humans 🤦 No it wasn't! It produced text that humans, unaware of its origin, interpreted as empathetic. Dear journalists, I know that tweets are short, but these language shortcuts do matter in how public perceives LLMs. It's not just jargon.
    user avatar
    Tamar Haspel
    @TamarHaspel
    Apr 28, 2023
    Researchers pittted ChatGPT against genuine doctors, using actual patient questions. ChatGPT was more accurate and more empathetic. By a lot. HT @adamcifu jamanetwork.com/journals/jamai…
    Image
    Readers added context
    The study itself claims in its section on Limitations that “the additional length of the chatbot responses could have been erroneously associated with greater empathy; and evaluators did not assess the chatbot responses for accuracy or fabricated information.”
    48K
  • user avatar
    Anna Rogers
    @annargrs
    May 16, 2020
    Personal news: I'm moving to Copenhagen! Excited to join the interdisciplinary team of political scientists, psychologists, anthropologists, economists, and computer scientists at @CPH_SODAS. I'll be working on both core and social NLP topics.
  • user avatar
    Anna Rogers
    @annargrs
    Oct 9, 2020
    New paper📜: What Can We Do to Improve Peer Review in NLP? arxiv.org/abs/2010.03863 with @IAugenstein TLDR: In its current form, peer review is a poorly defined task with apples-to-oranges comparisons and unrealistic expectations. /1
    Image