Image
user avatar
Michael C. Frank
@mcxfrank
Cognitive scientist at Stanford. Open science advocate. @stanfordsymsys director. Bluegrass picker, slow runner, dad.
Palo Alto, CA
stanford.edu/~mcfrank/
Joined June 2012
1,447
Following
14.3K
Followers
  • Pinned
    user avatar
    Michael C. Frank
    @mcxfrank
    Jan 20, 2023
    Do you want to do a psychology experiment while following best practices in open science? My collaborators and I have created Experimentology, a new open web textbook (to be published by MIT Press but free online forever). experimentology.io Some highlights! 🧵
    Image
    240K
  • user avatar
    Michael C. Frank
    @mcxfrank
    Apr 4, 2023
    People are testing large language models (LLMs) on their "cognitive" abilities - theory of mind, causality, syllogistic reasoning, etc. Many (most?) of these evaluations are deeply flawed. To evaluate LLMs effectively, we need some principles from experimental psychology.🧵
    Image
    549K
  • user avatar
    Michael C. Frank
    @mcxfrank
    Mar 27, 2023
    How do we compare the scale of language learning input for large language models vs. humans? I've been trying to come to grips with recent progress in AI. Let me explain these two illustrations I made to help. 🧵
    Image
    Image
    2.6M
  • user avatar
    Michael C. Frank
    @mcxfrank
    Aug 17, 2022
    Wordbank is an open database of children's vocabulary development across languages, archiving parent reports of what their children say. Today we are updating Wordbank, adding 10k children, 9 new languages, and data on bilinguals. wordbank.stanford.edu
    Wordbank homepage
  • user avatar
    Michael C. Frank
    @mcxfrank
    Jun 22, 2018
    Everyone makes mistakes during data analysis. Literally everyone. The question is not what errors you make, it's what systems you put into place to prevent them from happening. Here are mine. [a thread because I'm sad to miss #SIPS2018]
    user avatar
    simine vazire
    @siminevazire
    Jun 21, 2018
    Some thoughts as I get ready for #SIPS2018 : Some people say the credibility revolution has made researchers afraid of making mistakes. 1/4
  • user avatar
    Michael C. Frank
    @mcxfrank
    Apr 10, 2023
    What does it mean for a large language model (LLM) to "have" a particular ability? Developmental psychologists argue about these questions all the time and have for decades. There are some ground rules. 🧵
    diagram of developmental change referenced later in the post
    diagram of abstractions linking to observations referenced later in the post
    188K
  • user avatar
    Michael C. Frank
    @mcxfrank
    Aug 6, 2018
    A thought on grad advising. When I was a second year, an announcement went out to our dept. with the abstract for a talk I was giving in the area talk series. A senior faculty member wrote back with a scathing critique (cc'd to my advisor, @LanguageMIT). /1
  • user avatar
    Michael C. Frank
    @mcxfrank
    Jun 27, 2023
    How do we use methods from developmental psychology to assess AI models? My comment, "'Baby steps' in evaluating the capacities of large language models" is now out in Nature Reviews Psychology: nature.com/articles/s4415…
    Image
    75K
  • user avatar
    Michael C. Frank
    @mcxfrank
    Dec 19, 2018
    Errors in data analysis code are a major problem for scientists and for science. Two practices that can help you avoid them (or track them down more easily) : 1. never copy and paste code, and 2. never write a code block longer than your screen. [thread]
  • user avatar
    Michael C. Frank
    @mcxfrank
    Apr 19, 2023
    Can a large language model be used as a "cognitive model" - meaning, a scientific artifact that helps us reason about the emergence of complex behavior and abstract representations in the human mind? My answer is YES. Why and under what conditions? 🧵
    Figure showing a computational artifact being used as a model of a human system, with potential interventions of interest listed
    106K