Log inSign up
Image
user avatar
LocalAI
@LocalAI_API
OpenAI Open Source alternative. LocalAI is a community, drop-in replacement API compatible with OpenAI for local CPU/GPU inferencing
localai.io
Joined April 2023
39
Following
3,954
Followers
RepliesMedia
  • Pinned
    user avatar
    LocalAI
    @LocalAI_API
    16h
    LocalAI 4.8 is out. We just crossed 48.3k stars! - `local-ai chat` is a TUI agent harness now. - image to 3D, audio-driven talking avatars - voice cloning first-class in the UI - vLLM's serving architecture in C++20, no Python 386 PRs, 22 days, 25 contributors, 11 new👇
    Image
    00:00
  • user avatar
    LocalAI
    @LocalAI_API
    Jun 1
    Scaling LLMs across nodes? When a follow-up lands on a replica that never saw your chat, the whole prompt is recomputed and the KV cache wasted. LocalAI fixes this at the router: cache-aware routing across a mixed fleet of vLLM + SGLang + llama.cpp + ...
    Image
  • user avatar
    LocalAI
    @LocalAI_API
    Apr 2
    LocalAI 4.1.0 is out!
    Image
    user avatar
    Ettore Di Giacinto
    @mudler_it
    Apr 2
    Ok, notoriously I don't sleep that much. Time to share @LocalAI_API 4.1.0 (why not?) ! TLDR: - Distributed, hybrid clusters with production ready setup - Built-in auth, quota, user metrics - Fine-tuning and quantization from the UI 🔥Details below! 👇