ggml-org/llama.vscode

llama.vscode

Local LLM-assisted text completion, chat with AI and agentic coding extension for VS Code

Features

• Auto-suggest on input
• Accept a suggestion with Tab
• Accept the first line of a suggestion with Shift + Tab
• Accept the next word with Ctrl/Cmd + Right Arrow
• Toggle the suggestion manually by pressing Ctrl + L
• Control max text generation time
• Configure scope of context around the cursor
• Ring context with chunks from open and edited files and yanked text
• Supports very large contexts even on low-end hardware via smart context reuse
• Display performance stats
• Llama Agent for agentic coding
• Access to the llama-vscode agents from phone via a Telegram bot
• Deep links vscode://ggml-org.llama-vscode?view=agent&prompt=Hello opens VS Code (if not open), shows lama-vscode agent view and enters "Hello" as a prompt
• Kimi K3 - support for dynamically loaded tools (tools on demand) when used from moonshot.ai.
• Add/remove/export/import for models - completion, chat, embeddings and tools
• Model selection - for completion, chat, embeddings and tools
• Env (group of models) concept introduced. Selecting/Deselecting env selects/deselects all the models in it
• Add/remove/export/import for env
• Predefined models - both local and external
• Predefined envs for different use cases - only completion, chat + completion, chat + agent, loccal full package (with gpt-oss 20B), etc.
• MCP tools selection for the agent (from VS Code installed MCP Servers)
• Search and download models from Huggingface directly from llama-vscode

Installation

VS Code extension setup

Install the llama-vscode extension from the VS Code extension marketplace:

Note: also available at Open VSX

llama.cpp setup

• Now automatic on starting llama-vscode - from the official llama.cpp shell script or from brew (Mac, Linux) or Winget (Windows). The details below for the manual installation are left for reference.

Prerequisites:

• For macOS: Homebrew must be installed
• For Windows: Windows Package Manager (winget) is required

Show llama-vscode menu by clicking on llama-vscode in the status bar or Ctrl+Shift+M and select "Install/Upgrade llama.cpp". This will install llama.cpp automatically for Mac and Windows. For Linux get the latest binaries and add the bin folder to the path.

Once you have llama.cpp installed, you can select env for your needs from llama-vscode menu "Select/start env..."

Below are some details how to install llama.cpp manually (if you prefer it).

Mac OS

brew install llama.cpp

Windows

winget install llama.cpp

Any other OS

Either use the latest binaries or build llama.cpp from source. For more information how to run the llama.cpp server, please refer to the Wiki.

llama.cpp settings

Here are recommended settings, depending on the amount of VRAM that you have:

• More than 64GB VRAM:

  llama serve --fim-qwen-30b-default
  

• More than 16GB VRAM:

  llama serve --fim-qwen-7b-default
  

• Less than 16GB VRAM:

  llama serve --fim-qwen-3b-default
  

• Less than 8GB VRAM:

  llama serve --fim-qwen-1.5b-default
  

llama serve \
    -hf ggml-org/Qwen2.5-Coder-1.5B-Q8_0-GGUF \
    --port 8012 -ub 512 -b 512 --ctx-size 0 --cache-reuse 256

llama serve \
    -hf ggml-org/Qwen2.5-Coder-0.5B-Q8_0-GGUF \
    --port 8012 -ub 1024 -b 1024 --ctx-size 0 --cache-reuse 256

You can use any other FIM-compatible model that your system can handle. By default, the models downloaded with the -hf flag are stored in:

• Mac OS: ~/Library/Caches/llama.cpp/
• Linux: ~/.cache/llama.cpp
• Windows: LOCALAPPDATA

Recommended LLMs

The plugin requires FIM-compatible models: HF collection

Llama Agent

The extension includes Llama Agent

Features

• Llama Agent UI in Explorer view
• Works with local models - gpt-oss 20B is the best choice for now
• Could work with external models (for example from OpenRouter)
• MCP Support - could use the tools from the MCP Servers, which are installed and started in VS Code
• 9 internal tools available for use
• custom_tool - returns the content of a file or a web page
• custom_eval_tool - write your own tool in javascript (function with input and return value string)
• Attach the selection to the context
• Configure maximum loops for Llama Agent

Usage

1. Open Llama Agent with Ctrl+Shift+A or from llama-vscode menu "Show Llama Agent"
2. Select Env with an agent if you haven't done it before.
3. Write a query and attach files with the @ button if needed

More details(https://github.com/ggml-org/llama.vscode/wiki)

Examples

Speculative FIMs running locally on a M2 Studio:

https://github.com/user-attachments/assets/cab99b93-4712-40b4-9c8d-cf86e98d4482

Implementation details

The extension aims to be very simple and lightweight and at the same time to provide high-quality and performant local FIM completions, even on consumer-grade hardware.

• The initial implementation was done by Ivaylo Gardev @igardev using the llama.vim plugin as a reference
• Techincal description: https://github.com/ggerganov/llama.cpp/pull/9787

Other IDEs

• Vim/Neovim: https://github.com/ggml-org/llama.vim