LocalAI (@LocalAI

LocalAI

1,501 posts

LocalAI

@LocalAI_API

OpenAI Open Source alternative. LocalAI is a community, drop-in replacement API compatible with OpenAI for local CPU/GPU inferencing

Joined April 2023

Pinned
LocalAI
@LocalAI_API
Jan 23
LocalAI 3.9 and 3.10 are out! 🎉 Now we are a fully extensible OS for your AI applications. Highlights: 🤖 Support for Open Responses and Anthropic API 🗓️ Schedule background jobs (Cron/API) 🎥 New UI for Text-to-Video, Qwen TTS ⚡ Easy GPU: One docker image for everything👇
LocalAI
@LocalAI_API
Jun 1
Scaling LLMs across nodes? When a follow-up lands on a replica that never saw your chat, the whole prompt is recomputed and the KV cache wasted. LocalAI fixes this at the router: cache-aware routing across a mixed fleet of vLLM + SGLang + llama.cpp + ...
LocalAI
@LocalAI_API
May 31
00:49
Ettore Di Giacinto
@mudler_it
May 27
rf-detr.cpp: native C++/ggml (@ggml_org ) inference for @roboflow 's RF-DETR (my go-to for object detection and segmentation!) from the @LocalAI_API team. All 11 variants (5 detection + 6 segmentation), running at PyTorch speed (slightly, ~8% faster on CPU benchmarks), without
LocalAI
@LocalAI_API
Apr 2
LocalAI 4.1.0 is out!
Ettore Di Giacinto
@mudler_it
Apr 2
Ok, notoriously I don't sleep that much. Time to share @LocalAI_API 4.1.0 (why not?) ! TLDR: - Distributed, hybrid clusters with production ready setup - Built-in auth, quota, user metrics - Fine-tuning and quantization from the UI 🔥Details below! 👇
LocalAI
@LocalAI_API
Mar 21
New release is gonna be really exciting, here are some sneak peak
Richard Palethorpe
@jichiep
Mar 21
Share your GPUs with your friends! I took a few minutes to play with LocalAI's incoming user accounts feature that mudler added instead of sleeping.
00:00
See @LocalAI_API's full profile
Sign up
Log in

See @LocalAI_API's full profile

See @LocalAI_API's full profile