LocalAI

@LocalAI_API

OpenAI Open Source alternative. LocalAI is a community, drop-in replacement API compatible with OpenAI for local CPU/GPU inferencing

localai.io

Joined April 2023

Following

4,098

Followers

Replies Reposts Media

Pinned
LocalAI
@LocalAI_API
Aug 5
LocalAI 4.8 is out. We just crossed 48.3k stars! - `local-ai chat` is a TUI agent harness now. - image to 3D, audio-driven talking avatars - voice cloning first-class in the UI - vLLM's serving architecture in C++20, no Python 386 PRs, 22 days, 25 contributors, 11 new👇
00:00
2
LocalAI
@LocalAI_API
Jun 1
Scaling LLMs across nodes? When a follow-up lands on a replica that never saw your chat, the whole prompt is recomputed and the KV cache wasted. LocalAI fixes this at the router: cache-aware routing across a mixed fleet of vLLM + SGLang + llama.cpp + ...
4
LocalAI
@LocalAI_API
May 31
Ettore Di Giacinto
@mudler_it
May 27
00:49
rf-detr.cpp: native C++/ggml (@ggml_org ) inference for @roboflow 's RF-DETR (my go-to for object detection and segmentation!) from the @LocalAI_API team. All 11 variants (5 detection + 6 segmentation), running at PyTorch speed (slightly, ~8% faster on CPU benchmarks), without
LocalAI
@LocalAI_API
Apr 2
LocalAI 4.1.0 is out!
Ettore Di Giacinto
@mudler_it
Apr 2
Ok, notoriously I don't sleep that much. Time to share @LocalAI_API 4.1.0 (why not?) ! TLDR: - Distributed, hybrid clusters with production ready setup - Built-in auth, quota, user metrics - Fine-tuning and quantization from the UI 🔥Details below! 👇
1
LocalAI
@LocalAI_API
Mar 21
New release is gonna be really exciting, here are some sneak peak
Richard Palethorpe
@jichiep
Mar 21
Share your GPUs with your friends! I took a few minutes to play with LocalAI's incoming user accounts feature that mudler added instead of sleeping.
00:00