yzma: Go Bindings for Local LLM Inference
/yzma
yzma is a Go library that calls llama.cpp in-process for local LLM and VLM inference, using purego and ffi instead of CGo. Reach for it if you're shipping a self-contained Go binary that needs on-device inference with GPU acceleration and no C toolchain in the build. Skip it if you'd rather run a model server over HTTP, or want more community support than yzma's 533 GitHub stars provide.
Go Bindings for llama.cpp
yzma is a Go library that binds directly to llama.cpp for local inference, letting a Go program load a GGUF model and generate text or process images without spawning a server process. It loads the llama.cpp shared library at runtime through purego and ffi, so there's no CGo and no C compiler step in `go build`. The tradeoff: yzma needs a matching llama.cpp binary alongside your Go code, not one self-contained dependency.
Core Capabilities
- ✓Calls llama.cpp in the same process instead of talking to a model server, cutting out the network hop of an HTTP-based setup.
- ✓Uses purego and the ffi package to call the llama.cpp shared library, so `go build` and `go run` work without a C compiler.
- ✓Loads GGUF-format models, the format llama.cpp itself uses, so anything convertible to GGUF from Hugging Face works.
- ✓Picks up whatever hardware acceleration is available on the machine — CUDA, Metal, or Vulkan — per the README's own list.
- ✓Tracks new llama.cpp releases, so features and fixes landing upstream become available to yzma without waiting on a separate reimplementation.
- ✓Ships a `yzma` CLI for downloading GGUF models straight from a Hugging Face URL, e.g. `yzma model get -u <url>`.
- ✓Handles both text-only language models and vision-language models (VLM) that take an image plus a prompt.
Practical Applications
- •Embedding a local chatbot inside a Go CLI tool or desktop app without shelling out to a separate inference server.
- •Running vision-language inference — the README's own example feeds a JPEG and a prompt to a Qwen2.5-VL model and gets a text description back.
- •Building for constrained or specialized hardware: the repo lists install notes for Raspberry Pi, NVIDIA Jetson Orin, and the Arduino UNO Q.
- •Cross-compiling a single Go binary for different OS/GPU targets using normal GOOS/GOARCH, without maintaining separate C build configs per platform.
Getting Started with yzma
Install instructions live in the repo's INSTALL.md, with separate sections for macOS, Linux, and Windows, plus dedicated notes for Raspberry Pi, NVIDIA Jetson Orin, and the Arduino UNO Q. The README describes two paths: run the `yzma` CLI tool to download prebuilt llama.cpp libraries for your platform, or let your application self-download them at install time — that self-download path can auto-detect CUDA and ROCm. Beyond pointing to INSTALL.md, the README doesn't spell out the exact `go get`/module command, so treat the precise package import path as not clearly documented here.
Writing Go AI Programs
A minimal program calls `llama.Load(libPath)` to load the shared library, `llama.Init()`, then `llama.ModelLoadFromFile()` and `llama.InitFromModel()` to get a model and context. From there you tokenize a prompt with `llama.Tokenize()`, build a batch with `llama.BatchGetOne()`, and pull tokens one at a time in a loop with a sampler chain, converting each token back to text with `llama.TokenToPiece()`. The repo's examples/ directory has three ready-to-run programs: a plain-text hello world, an interactive chat loop, and a VLM example that takes an image path and a prompt flag. Run any of them with a plain `go run ./examples/<name>/` once the model files are downloaded — there's no separate build step for the Go side.
Strengths
- ✓No CGo and no C compiler needed in your build — `go build` and `go run` behave normally, which matters in CI or cross-compilation setups.
- ✓Runs llama.cpp in-process, so there's no separate model server to deploy, monitor, or keep alive alongside your Go app.
- ✓Covers both text LLMs and vision-language models through the same API, per the README's VLM and chat examples.
- ✓Kept in sync with new llama.cpp releases, with tests run automatically against each one according to the README.
- ✓Comes with a `yzma` CLI for pulling GGUF models directly from a URL, so you're not hand-rolling a download script.
Current Limitations
- △License is not listed in the metadata provided here, so check the repo directly before depending on it for a commercial project.
- △Your Go binary and the llama.cpp shared library are two separate artifacts that have to match versions — the README's own compatibility table lists narrow build ranges (e.g. only b9541–b9548 works with yzma v1.16.0), so upgrading either side carelessly can break things.
- △The README states yzma covers 'over 96%' of llama.cpp functionality, meaning some llama.cpp capability is still out of reach — which slice that is isn't spelled out.
- △The community around it is small, so you're more likely to end up reading source or opening an issue than finding a ready answer online.
- △GPU acceleration depends on you having the right driver stack (CUDA, ROCm, Vulkan, or Metal) installed on the target machine — that's on you, not something yzma abstracts away.
Similar Go AI Libraries
Who is yzma For?
yzma is for Go developers who want local LLM or VLM inference inside their own application instead of calling out to a hosted API or standing up a separate model server. It fits well if you're already comfortable cross-compiling Go binaries and don't mind matching your llama.cpp library version to your yzma version by hand. It's a worse fit if you want a turnkey install with no version bookkeeping, or if your team doesn't already have someone comfortable debugging at the FFI/shared-library boundary when something goes wrong.
Common Questions
yzma loads models in the GGUF format, the same format llama.cpp itself uses, so anything already converted to GGUF — including models pulled from Hugging Face — works with yzma.
yzma does not require CGo. It calls the llama.cpp shared library through the purego and ffi packages instead, so `go build` and `go run` work without a C compiler in the toolchain.
yzma can use CUDA, Vulkan, HIP, ROCm, or SYCL on Linux, Metal on macOS, and CUDA, Vulkan, HIP, SYCL, or OpenCL on Windows, depending on what's installed on the machine, per the README's support table.
yzma runs on Linux, macOS, and Windows, with amd64 and arm64 CPU support on Linux and arm64 on macOS. The repo also documents install steps for Raspberry Pi, NVIDIA Jetson Orin, and the Arduino UNO Q.
yzma's tests run automatically against every new llama.cpp release, and the README keeps a compatibility table mapping llama.cpp build ranges to yzma versions — for example, builds b9541 through b9548 require yzma v1.16.0.
yzma supports multimodal inference through vision-language models: the repo's own example feeds a Qwen2.5-VL model both an image and a text prompt and returns a description of the picture, using yzma's mtmd package.
The problem it solves
Most ways to run a local LLM in a Go application mean shelling out to a separate server process (llama.cpp's own server binary, Ollama, LocalAI) and talking to it over HTTP, which adds a process to deploy and a network hop for every request. yzma solves the process-boundary part specifically: it binds llama.cpp into the same address space as your Go program via purego, so there's no server to start, monitor, or restart, and no CGo step forcing a C toolchain into your build pipeline.
Related repositories
Still deciding about yzma?
One click hands the question to an AI along with this page — see what it says about yzma.