Website  •  Docs  •  Support  •  Discord

QVAC is an open-source, cross-platform ecosystem for building local-first, peer-to-peer AI applications and systems. With QVAC, you can run AI tasks like LLMs, speech, RAG, and more locally across Linux, macOS, Windows, Android, and iOS — or delegate inference to peers using its built-in P2P capabilities.

• Local-first: load AI models and perform inference on your own machine. No third-party APIs, SaaS, or cloud involved.
• P2P: build unstoppable internet systems — like BitTorrent, IPFS, and blockchain networks, but for AI.
• Cross-platform: consistent developer experience across hardware, operating systems, and JS runtime environments — write code once, run it everywhere.
• OpenAI-compatible API: integrate with the broader AI ecosystem.
• Open source: 100% free to use and modify — build on top, contribute back, be part of our community.

QVAC is composed of JavaScript libraries and tools that converge in the JS SDK. The SDK is the main entry point for using QVAC. It is type-safe and exposes all QVAC capabilities through a unified interface. It runs on Node.js, Bare runtime, and Expo.

Additionally, QVAC provides a CLI with tools and an HTTP server that exposes an OpenAI-compatible API. By implementing the OpenAI API format, QVAC can integrate with the broader AI ecosystem.

Install the @qvac/sdk npm package in your project. Then load models and run AI inference locally, or delegate inference to peers using the built-in P2P features.

1. Create the examples workspace:

mkdir qvac-examples
cd qvac-examples
npm init -y && npm pkg set type=module

2. Install the SDK:

npm install @qvac/sdk

3. Create the quickstart script:

import { loadModel, LLAMA_3_2_1B_INST_Q4_0, completion, unloadModel, } from "@qvac/sdk";
try {
    // Load a model into memory
    const modelId = await loadModel({
        modelSrc: LLAMA_3_2_1B_INST_Q4_0,
        modelType: "llm",
        onProgress: (progress) => {
            console.log(progress);
        },
    });
    // You can use the loaded model multiple times
    const history = [
        {
            role: "user",
            content: "Explain quantum computing in one sentence",
        },
    ];
    const result = completion({ modelId, history, stream: true });
    for await (const token of result.tokenStream) {
        process.stdout.write(token);
    }
    // Unload model to free up system resources
    await unloadModel({ modelId });
}
catch (error) {
    console.error("❌ Error:", error);
    process.exit(1);
}

4. Run the quickstart script:

node quickstart.js

• Completion: LLM inference for text generation and chat via qvac-fabric-llm.cpp.
• Text embeddings: vector embedding generation for semantic search, clustering, and retrieval, via qvac-fabric-llm.cpp.
• Translation: text-to-text neural machine translation (NMT), via qvac-fabric-llm.cpp and Bergamot.
• Transcription: automatic speech recognition (ASR) for speech-to-text via qvac-ext-lib-whisper.cpp or NVIDIA Parakeet, plus speaker diarization via NVIDIA Sortformer.
• Text-to-Speech: speech synthesis for text-to-speech (TTS) using the Chatterbox and Supertonic neural TTS models.
• OCR: optical character recognition (OCR) for extracting text from images, via the ONNX Runtime or GGML backends.
• Image generation: text-to-image generation via qvac-ext-stable-diffusion.cpp.
• Fine-tuning: adapting LLMs to domain-specific tasks via LoRA.
• Multimodal: LLM inference over text, images, and other media within a single conversation context.
• RAG: out-of-the-box retrieval-augmented generation workflow.

• Delegated inference: delegate inference to peers via the Holepunch stack, enabling resource sharing.
• Fetch models: download AI models from peers via the distributed model registry.
• Blind relays: connect peers across NATs/firewalls by routing traffic through relay nodes.

• Plugin system: build lean apps by including only required AI capabilities, and extend the SDK by plugging in custom capabilities.
• Logging: visibility into what's happening during loading, inference, and other operations.
• Download Lifecycle: pause and resume model downloads.
• Sharded models: download a model that is sharded into multiple parts.

Monorepo structure overview. All QVAC components live under /packages, including the SDK, libraries, and tooling. Not every component is published to npm.

Legend:

• Core: foundational building blocks shared across the ecosystem.
• Addon: capability packages — each QVAC capability is implemented by one or more addons.
• SDK: primary entry point for consumers.
• Tool: user-facing tools and services that support the ecosystem.

• For the standard development workflow used in this monorepo, see /docs/gitflow.md.
• For development specifics of each QVAC component, refer to the documentation in the respective subdirectory under /packages.
• For the QVAC architecture as a whole, see /docs/architecture.

Built something with QVAC? Add a badge or banner to your README, website, or app. It is a simple way to highlight your project, help others discover QVAC, and strengthen our community.

By using these badges and banners, you help foster the QVAC ecosystem!

Choose a banner or badge below and copy its Markdown snippet, or copy its image URL and use the hosted SVG asset directly.

Large format badges (240x60) for prominent placement in your README header.

Dark with monochrome glow

Dark with colorful flow

Dark with stars pattern

Light with colorful flow

Banner usage

[![Built with QVAC](https://raw.githubusercontent.com/tetherto/qvac/refs/heads/main/docs/branding/qvac-banner-dark-glow.svg)](https://github.com/tetherto/qvac)

Compact badges for use alongside other shields/badges in your README.

Compact

Variant	Dark bg	Light bg
Green logo
Monochrome

Inline

Variant	Dark bg	Light bg
Green logo
Monochrome

Badge usage

[![Built with QVAC](https://raw.githubusercontent.com/tetherto/qvac/refs/heads/main/docs/branding/qvac-badge-green-dark.svg)](https://github.com/tetherto/qvac)