Does your AI team still use Slurm? Or worse, do you manually ssh into nodes to launch workloads? 😢
I promise you, running a training or inference workload does not need to be that hard.
Trainy
@TrainyAI
Building open source tools for distributed training.
Joined June 2023
- Really respect the team behind Llama 3.1 for going into such depth in their technical report. They mention, "During a 54-day snapshot period of pre-training, we experienced a total of 466 job interruptions." This means they experienced an interruption less than every 3 hours.
- Excited to be releasing the latest feature of our Konduktor platform: the Node Health Controller. When running H100s optimally, GPUs and network switches have an abysmally high fault rate (~30%). This makes developing large scale models frustrating. trainy.ai/blog/konduktor…
- Does 100% GPU Util mean you're fully utilizing your GPUs? Probably... not. Streaming Multiprocesser (SM) efficiency expresses the fraction of SMs that are active in a given interval, and ends up being a much more informative metric.
- Mistral7B has beaten many previous SoTA models of larger or equal size across multiple benchmarks. Here we show how you can finetune from a Mistral 7B base using on your own data. trainy.ai/blog/fine-tune… #LLM #GenAI #MachineLearning
