vLLM

@vllm_project

A high-throughput and memory-efficient inference and serving engine for LLMs. Join slack.vllm.ai to discuss together with the community!

vllm.ai

Joined March 2024

Following

46.6K

Followers

Replies Articles Media

Pinned
vLLM
@vllm_project
Jul 9
Announcing the first-ever vLLM Conference — hosted by @inferact at Ray Summit, Aug 24–26 in San Francisco 🎉🌉 This is where we'll get into the work pushing open, high-performance inference forward, such as: 🗺️ Where the vLLM roadmap is headed ⚡ Getting the most out of
First vLLM Conference at Ray Summit
From vllm.ai
vLLM
@vllm_project
5h
Good to see @RedNote back with a new open model, and a much bigger one built for long-horizon agent work. dots3-note preview is a 280B MoE with 16B active at 512K context, it reads images, audio, and video, and it is trained to explore unfamiliar environments and update its own
dots studio
@dotsstudioai
7h
Introducing dots3-note preview — a small but mighty step toward long-horizon agency in real life. 🔹 280B MoE with 16B active parameters, a 512K context window, and multimodal understanding across text, vision, and audio 🔹 Introduces TEMPO, a new RL approach for long-horizon
vLLM
@vllm_project
Aug 12
🎉 Congrats to @Alibaba_Qwen on Qwen3.8-2.4T-A95B, one of the largest open-weight models released to date. 2.4T params, 95B active, 512 experts. Day-0 support in vLLM, verified on @nvidia and @AMD hardware. A ready-made 4-bit checkpoint per vendor, both out of the box:
Day 0 Support for Qwen3.8-2.4T-A95B on vLLM
From vllm.ai
vLLM
@vllm_project
Aug 12
vLLM's model loader and KV connector both have an @Azure Blob path now: weights in, KV out. @Microsoft and @NVIDIAAI shipped a recipe for each. Nothing serves until weights land in HBM. Dynamo ModelExpress plugs into the loader, up to 7.3x faster than the default on H100/A100.
vLLM
@vllm_project
Aug 11
We're co-hosting a @vllm_project x @nvidia Dynamo Meetup during vLLM Conference week in San Francisco ⚡ Tech talks on serving LLMs efficiently at scale + food, drinks, and the inference community all in one room 📅 Aug 24, 6–9pm PT Space is limited — RSVP:
vLLM x NVIDIA Dynamo Meetup · Luma
From luma.com