QVAC: Local AI SDK for Cross-Platform Apps

What is QVAC?

QVAC is an open-source, cross-platform AI SDK from Tether for building local-first, peer-to-peer AI applications. It ships JavaScript/TypeScript (@qvac/sdk) and Python (tetherto-qvac-sdk) clients that load models and run inference on Linux, macOS, Windows, Android, or iOS, or delegate inference to peers over its P2P layer. It also runs as an HTTP server with an OpenAI-compatible API, acting as a local model provider.

Core Capabilities

• ✓Local-first, on-device AI inference: loads models and runs AI tasks on your own machine, with no third-party API, SaaS, or cloud dependency, per the README.
• ✓Peer-to-peer delegation: built-in P2P capability lets you hand inference off to peers, which the README compares to BitTorrent, IPFS, and blockchain networks applied to AI.
• ✓Dual-language SDKs sharing one worker: the JS/TS client (@qvac/sdk) and Python client (tetherto-qvac-sdk) drive the same underlying worker, so capabilities and behavior match across both.
• ✓Cross-platform reach: consistent developer experience on Linux, macOS, Windows, Android, and iOS, per the README.
• ✓OpenAI-compatible HTTP server: QVAC can run as a local model provider that plugs into any harness built for the OpenAI API, such as OpenCode or OpenClaw per the README.
• ✓Runtime flexibility on the JS/TS side: the client runs on Node.js, the Bare runtime, and Expo, not just a browser or server context.
• ✓Apache-2.0 licensed and open source, described in the README as free to use, modify, and build on.

Getting Started with QVAC

Once installed, the pattern is the same in both languages: load a model, run inference, then unload it. In JavaScript, loadModel takes a modelSrc, a modelType (e.g. "llm"), and an onProgress callback for download progress; completion takes the modelId plus a chat history array and returns a token stream you iterate with a for-await loop; unloadModel frees the resources when you're done. The Python client mirrors this with async load_model, completion, and unload_model calls inside a Client() context manager, since it's asyncio-native. Beyond the SDK, QVAC also runs as a standalone HTTP server, so you can point an existing OpenAI-API-compatible tool at it instead of writing SDK code at all.

Supported AI Tasks

• •Text generation, embeddings, and RAG — LLM inference and vector embeddings run via Fabric LLM, with an out-of-the-box RAG workflow for retrieval-augmented answers.
• •Speech and translation — automatic speech recognition through @qvac/asr-ggml (Whisper or NVIDIA Parakeet) with duplex streaming, text-to-speech through @qvac/tts-ggml (Chatterbox, Supertonic, Parler, or CosyVoice3), and neural machine translation via Fabric LLM and Bergamot.
• •Image, video, and music generation — text-to-image and image-to-image through a customized Diffusion backend, text-to-video and image-to-video through the same backend family, and music generation from text, lyrics, and musical controls via ACE-Step.
• •Multimodal and specialized inference — multimodal LLM conversations spanning text and images, OCR via ONNX Runtime or GGML backends, image classification, brain-computer interface transcription via a customized Whisper backend, and vision-language-action inference for robot control.
• •Fine-tuning — adapting LLMs to domain-specific tasks with LoRA.

Similar Local AI SDKs

Common Questions

The problem it solves

Most local-AI tooling picks one task — an LLM runner, or a TTS library, or an OCR tool — and developers building an app that needs more than one end up stitching together several separate projects, each with its own API shape and packaging quirks. QVAC's README frames its answer as a single SDK covering a wide spread of AI task types — from LLM chat and RAG to speech, image/video generation, translation, and OCR — behind one client API in JavaScript/TypeScript or Python, plus a built-in path to delegate inference to peers instead of a cloud vendor when local hardware falls short.

How to install / try

Installing QVAC depends on which client you pick. For JavaScript/TypeScript: npm install @qvac/sdk inside a Node.js project (the README's quickstart also runs npm init -y and npm pkg set type=module first, for ESM support). For Python: pip install "tetherto-qvac-sdk", then run python -m tetherto.qvac_sdk install-worker — a required step, because the Python client needs Node.js and npm on your machine, and install-worker shells out to npm install to set that up. Both clients pull the actual model weights (the README's quickstart example uses LLAMA_3_2_1B_INST_Q4_0) on first load, not at install time.

Who should try it — and who should skip

Reach for QVAC if you're building a JS/TS or Python app that needs local or peer-delegated inference across more than one AI task — chat plus RAG plus transcription, say — and you'd rather not integrate three separate libraries to get there. It's also worth a look if you just want a local, OpenAI-API-compatible endpoint to point an existing tool at, without writing SDK code. Skip it if you need a narrowly-focused, battle-tested library for a single task instead, such as a dedicated ASR or TTS project, or if undocumented hardware requirements and a small, early-stage repository are dealbreakers for your project.

Related repositories