Log inSign up
Ettore Di Giacinto profile banner
user avatar

Ettore Di Giacinto

@mudler_it
dad, creator of LocalAI(localai.io) and Kairos (kairos.io) , ex @SUSE/@Rancher, ex-Gentoo Dev.
Italy
github.com/mudler
Joined January 2016
265
Following
3,672
Followers
RepliesMedia
  • Pinned
    user avatar
    Ettore Di Giacinto
    @mudler_it
    Aug 7
    vllm.cpp runs @MiniMax_AI 's MiniMax-H3 now. 33.1B, video and audio out of a single model, and I'm still a bit stunned that it works we reimplemented both VAEs from the checkpoint's remote python, so there's no torch anywhere in the process. And you drive the whole thing over
    Image
    00:00
  • user avatar
    Ettore Di Giacinto
    @mudler_it
    Aug 10
    RDNA4 is getting shape in vllm.cpp!
    user avatar
    Mike
    @anothervariable
    Aug 10
    Lots to chew on here and it's painfully slow. But it is working on vllm.cpp/rocm Gemma 4 26B MoE :). What you see in the screenshot is vllm.cpp with rocm and ported cuda kernels into rocm/hip RDNA4 serving Gemma 4 26B MoE. Prefill and decoding and responding to Hermes Agent
    Image
  • user avatar
    Ettore Di Giacinto
    @mudler_it
    Aug 8
    this is what I hoped would happen, and not this fast two days ago I shipped an AMD backend in vllm.cpp that had never been compiled by anyone, including me, because I have no AMD card. it went in as a skeleton to get things started, so everyone could build on top. now there's