Jack Bai (@jackbot

Jack Bai

421 posts

Jack Bai

@jackbot_cs

CS PhD @UofIllinois, Research @NVIDIA | Prev @Berkeley_ai, @MSFTResearch

Champaign, IL

Joined January 2024

Replies Articles Media

Pinned
Jack Bai
@jackbot_cs
Jun 9
😈 WebGym is now 300% faster with better performance. We introduce AsyncWebRL, with 300% end-to-end throughput and better performance over WebGym (which uses REINFORCE) with async GRPO. Two algorithmic takeaways we learnt: (1) do asynchronous multi-step agentic RL with the
00:00
Jack Bai
@jackbot_cs
Mar 7, 2025
We just made Q function work on 7B VLMs with TD learning. If you work on end-to-end RL with Q functions, you know it's extremely hard. tbh most people give it up right after they finish the first wandb run. Let me show how we got through: A thread 🧵 1/n arxiv.org/abs/2502.15760
Jack Bai
@jackbot_cs
Feb 21, 2025
🚨🚨🚨 We found that 1k trajectories + offline RL with VLM can achieve performances of previous online RL methods on Android-in-the-Wild. 23% to 71% boost, scalable, no online interactions, and no fancy RL tricks. Check out our ICLR paper Digi-Q: digiq-agent.com 1/15
00:00
Jack Bai
@jackbot_cs
Feb 21, 2025
🚨🚨🚨 We found that 1k trajectories + offline RL with VLM can achieve performances of previous online RL methods on Android-in-the-Wild. 23% to 71% boost, scalable, no online interactions, and no fancy RL tricks. Check out our ICLR paper Digi-Q: digiq-agent.com 1/15
00:00
Jack Bai
@jackbot_cs
Mar 6, 2025
Personal update: I am thrilled to resume my PhD study from MS at UIUC CS (@IllinoisCDS), advised by Prof. Tong Zhang(tongzhang-ml.org)! If you happen to drop by Champaign, come in and you'll get a cup of free coffee ;p
tongzhang-ml.org
Tong Zhang
Tong Zhang is a professor of Computer Science at the University of Illinois Urbana-Champaign. His research interests include machine learning theory, algorithms, optimization, reinforcement learnin...
Jack Bai
@jackbot_cs
Jun 21, 2024
My collaborators have posted excellent summaries of the work, so I’m going to write a slighted more in-depth analysis of the development of this work. Website: digirl-agent.github.io ArXiv: arxiv.org/pdf/2406.11896 Code: github.com/DigiRL-agent/d… 1/
00:00
Jack Bai
@jackbot_cs
Mar 24, 2025
I am going to give a talk at @Stanford during 10-10:30 AM, March 25 (tomorrow). The location will be in Simonyi Conference Center at CoDa. This talk will discuss major challenges and properties of pre-training language models with built-in sparsity. Major points to discuss: 1.
Jack Bai
@jackbot_cs
Mar 7, 2025
Replying to @jackbot_cs
To the best of our knowledge, this work is the first to successfully scale state-action Q-value functions to realistic agent tasks with VLMs and show significantly improved performance. I hope this work also inspires people in the empirical RL field! 🧑‍🍳 🧵 n/n
Jack Bai
@jackbot_cs
Feb 21, 2025
Replying to @jackbot_cs
We came up with a way to fine-tune the VLM representations with an unsupervised approach. We simply need to make VLMs pay more attention to where the cursor is on the screen, and it worked pretty robustly. Here's how we did it: 8/15
Jack Bai
@jackbot_cs
Feb 21, 2025
Replying to @jackbot_cs
Sounds like offline algorithms require significantly more data than online learning as they’re sub-optimal? NO - we managed to use even less data (1296 trajs) than DigiRL online learning (1600 trajs). Now for our method: let's start with introducing a reliable Q function. 5/15
Jack Bai
@jackbot_cs
Feb 21, 2025
Replying to @jackbot_cs
Why does sample efficiency matter? In tasks like mobile device control, where real rollouts are slow and costly, we need efficient learning. Can we eliminate interactions entirely during agent training? 4/15
Jack Bai
@jackbot_cs
Apr 3, 2025
I’ll be at NYC this summer! Anyone wants to have a chat? I can host friends in the weekend wherever you come from 🎃
Jack Bai
@jackbot_cs
Dec 10, 2024
I’m at NeurIPS! On Wed 11 Dec (tomorrow), in the morning session, I will present DigiRL -- a SOTA VLM-based agent trained with RL to perform tasks on your Android device, in West Ballroom, A-D, #7301. Looking forward to have chat with people in agents/ RL/ LLM! DMs are open.
Jack Bai
@jackbot_cs
Mar 7, 2025
Replying to @jackbot_cs
If you look at the bellman equation of Q, you know what I'm talking about. Empirically, Q^π(s, a) where a is in the dataset learns from Q^π(s, π(s)), where π(s) is not necessarily in the dataset. This requires the Q function to be good at marginal generalizations. 🧵 3/n
Jack Bai
@jackbot_cs
Feb 21, 2025
Replying to @jackbot_cs
Our previous work DigiRL solved the Android-in-the-Wild challenge quite well by designing an online policy-based RL algorithm, which utilizes a doubly robust V estimator and automatic curriculum. 2/15
00:00