Lily Su

@excelsiorpred

I made GPT read contracts at a Fortune 100. Escaped to tinker with ML plumbing, one failed compile at a time. Founder: kernelpro.store

San Francisco, CA

Lily Su
@excelsiorpred
Dec 19, 2025
Been MIA due to leaky nose and leaky abstractions. Here are 3 things I learned battling nsys profiling on Windows wsl2. tdlr: just don't... 🧵
3
Lily Su
@excelsiorpred
Dec 19, 2025
Shifting from compute to memory now. Captured a memory snapshot during a forward/backward pass using torch.cuda.memory._record_memory_history() and dumped it to a pickle. Here's a quick screen recording of what it looks like in pytorch.org/memory_viz 🧵
00:00
3
Lily Su
@excelsiorpred
Nov 29, 2025
TIL: Never benchmark PyTorch with just time.time() Here’s why warmup + synchronize() actually matter: When you run output = model(input), the CPU dispatches the order to the GPU queue and immediately moves to the next line of code. The Fix...🧵
5
Lily Su
@excelsiorpred
Nov 28, 2025
I performed Architectural Ablations on a custom 10M param Llama-style stack (RoPE + SwiGLU). Experiment A (Activations): SwiGLU vs. SiLU. Experiment B (Normalization): PreNorm vs. PostNorm. Here is the breakdown of what matters and what doesn't. 🧵
2
Lily Su
@excelsiorpred
Nov 28, 2025
Running a head-to-head comparison today to see how Data Entropy affects convergence speed. TinyStories (Low Entropy)🟩🟩 34K steps | 557M Tokens | ~ 25 min OpenWebText (High Entropy)🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩 180K steps | 2.95B Tokens | ~ 2 hours
GIF