Pinned
Profiling CUDA in production inference is a different problem than profiling a dev run.
What actually works:
graphsignal.com/blog/cuda-prof…
Graphsignal
@GraphsignalAI
Inference Profiler
- "you only have like 15% of the chip being utilized." - @AnjneyMidha Getting more out of the GPU is exactly what we're working on at @GraphsignalAI, particularly with auto-flags: graphsignal.com/blog/auto-opti… (Our founder @dmitrimelikyan in the background at the @tbpn podcastAMP PBC Founder @AnjneyMidha says the massive amount of AI compute being wasted in the U.S. is a national security crisis: “Every dollar of long-term lease that an AI lab takes today, you lose about 30–40% of FLOPs in just bad scheduling. The nodes are just straight-up not
- New: auto-flags for vLLM, SGLang, other engines. graphsignal-run --auto-flags sets startup flags from your workload’s GPU profiles + telemetry (and recipes/docs on first launch). graphsignal.com/blog/auto-opti…
- We’re at @AMD Advancing AI this week. If you’re around and want to meetup / learn more about @GraphsignalAI, DM us.
- autodebug: an autonomous loop that deploys an inference service, benchmarks it, reads profiling telemetry, and redeploys with a better config. Then repeats. Uses @GraphsignalAI for inference profiling, @dstackai for GPU provisioning, Claude Code as the agent.
