Tzu-Heng (Brian) Huang (@zihengh1) / X

Tzu-Heng (Brian) Huang

236 posts

Tzu-Heng (Brian) Huang

@zihengh1

CS PhD @WisconsinCS. Prev: @Apple, @Meta. Focusing on multimodal models, RL for VLMs, and data-centric AI (curation, synthesis, and auto-labeling).

Mars

Joined March 2021

Pinned
Tzu-Heng (Brian) Huang
@zihengh1
Mar 13
Scaling expert-annotated image captions is expensive. Supervised distillation from VLMs helps but has a diversity ceiling: models memorize the teacher's style and generalize poorly. Can RL fix this without a verifiable "ground truth"? Introducing RubiCap: arxiv.org/abs/2603.09160
12K
Tzu-Heng (Brian) Huang
@zihengh1
Oct 7, 2024
Annotating your data with state-of-the-art large language models can be costly and opaque. What can we do about this? Simple idea: instead of prompting LLMs for labels, we distill them into programs you can run locally for free. Introducing Alchemist, a Spotlight at #NeurIPS2024!
37K
Tzu-Heng (Brian) Huang
@zihengh1
Dec 9, 2024
🚀 Excited for #NeurIPS2024 next week! We will present Alchemist, our 500x cheaper alternative to LLM-as-annotator, distilling LLM into code for automated labeling! 🧪✨ 📍 Poster Session: Wednesday, 4:30 - 7:30 East Exhibit Hall A-C (#1903)✨ See below for Alchemist's details.
3.6K
Tzu-Heng (Brian) Huang
@zihengh1
Feb 9, 2025
Tons of model weights available, but what else can we do besides prediction? 🤔 📣 Introducing Grad-Mimic! A new data selection framework using well-trained model’s weights to find high-value samples for foundation models. Boost data curation & data efficiency!
3.1K
Tzu-Heng (Brian) Huang
@zihengh1
Dec 12, 2023
Can we treat model weights as market commodities? Just as manufacturers use purchased parts to cut production costs, ML could gain from trading well-trained parameters in market, instead of isolated development! We build a market to trade parameter sets in our #NeurIPS2023 paper!
3.3K
Tzu-Heng (Brian) Huang
@zihengh1
Oct 7, 2024
Replying to @zihengh1
🚀Alchemist is an automated annotation system that cuts costs by up to 500x while maintaining or improving labeling performance—and enables easily bringing SMEs into the loop! Our paper link: arxiv.org/abs/2407.11004 If you found this interesting, feel free to spread the word!
1.3K
Tzu-Heng (Brian) Huang
@zihengh1
Dec 12, 2023
Curious about buying parameters to accelerate model training or selling parameters to earn additional profits? 💵💵💵 Check out our #NeurIPS2023 paper or visit our poster (#2008) at 5:15 today! Let’s TRADE!
Tzu-Heng (Brian) Huang
@zihengh1
Dec 12, 2023
Can we treat model weights as market commodities? Just as manufacturers use purchased parts to cut production costs, ML could gain from trading well-trained parameters in market, instead of isolated development! We build a market to trade parameter sets in our #NeurIPS2023 paper!
1.6K
Tzu-Heng (Brian) Huang
@zihengh1
Dec 11, 2024
💸 Want to save your project expense on labeling? Check out Alchemist, a #NeurIPS2024 Spotlight! 🙋🏻 We will present our poster from 4:30 to 7:30 today in East Exhibit Hall A-C (#1903).
Tzu-Heng (Brian) Huang
@zihengh1
Dec 9, 2024
🚀 Excited for #NeurIPS2024 next week! We will present Alchemist, our 500x cheaper alternative to LLM-as-annotator, distilling LLM into code for automated labeling! 🧪✨ 📍 Poster Session: Wednesday, 4:30 - 7:30 East Exhibit Hall A-C (#1903)✨ See below for Alchemist's details.
935
Tzu-Heng (Brian) Huang
@zihengh1
Oct 2, 2023
Attending #ICCV2023 in #Paris this week! Excited to share our work on DataComp submission. Q: If you have large-scale and noisy web-crawled data, how do you curate training data efficiently? Answers are in the thread! [1/n]
3.5K
Tzu-Heng (Brian) Huang
@zihengh1
Oct 7, 2024
Replying to @zihengh1
Operating on non-text modalities can be more challenging. We combine local feature extraction with programs written on top of those features. Check out our paper for more!
521
Tzu-Heng (Brian) Huang
@zihengh1
Oct 2, 2023
Replying to @zihengh1
This approach yields the top-ranking and second-ranking position on the DataComp leaderboard in the small-scale filtering track! [3/n]
4.9K
Tzu-Heng (Brian) Huang
@zihengh1
Dec 12, 2023
Replying to @zihengh1
Chat with us in the poster session (#2008) on Tuesday from 5:15 ~ 7:15 at Great Hall & Hall B1+B2 (level 1). neurips.cc/virtual/2023/p… Joint work with @harit_v, and my advisor @fredsala!
96
Tzu-Heng (Brian) Huang
@zihengh1
Oct 7, 2024
Replying to @zihengh1
🙋🏻Still, prompting ChatGPT for your labels repeatedly? Try to generate your program code to save the project expenses! Joint work with Cathy Cao, @Vaish026, and my advisor @fredsala! Github repo: github.com/SprocketLab/Al… Preprint link: arxiv.org/abs/2407.11004
github.com
GitHub - SprocketLab/Alchemist
Contribute to SprocketLab/Alchemist development by creating an account on GitHub.
486
Tzu-Heng (Brian) Huang
@zihengh1
Oct 7, 2024
Replying to @zihengh1
LLM-based annotators offer efficiency over costly human labelers but come with challenges: ⚠️Cost: Labeling a moderate dataset can cost thousands. ⚠️Inflexibility: Small label schema changes require a full pipeline rerun. ⚠️Opaque: API access doesn’t allow any model inspection.
935