LlamaFactory: LLM Fine-Tuning Framework for 100+ Models
/LlamaFactory
LlamaFactory turns LLM fine-tuning into a config-and-click workflow, wiring LoRA, QLoRA, PPO, DPO, and other methods behind one CLI and a Gradio Web UI instead of a hand-built training script. That breadth beats stitching together separate PEFT, TRL, and dataset-prep code yourself. The catch: heavy customization past its supported recipes means fighting the abstraction, not extending it.
What is LlamaFactory
LlamaFactory is an open-source Python framework for fine-tuning large language and vision-language models without hand-writing a training script. It wraps LoRA, QLoRA, full-parameter tuning, and preference methods like DPO, KTO, and ORPO behind one CLI and a Gradio Web UI called LlamaBoard. The README lists support for 100+ model families, including LLaMA, Qwen3, DeepSeek, Gemma, GLM, and multimodal models like LLaVA.
Key features and supported models
- ✓Fine-tunes 100+ model families end to end — LLaMA, Qwen3, DeepSeek, Gemma 3, GLM-4.5, Mixtral-MoE, and multimodal models like LLaVA, Qwen3-VL, and MiniCPM-o
- ✓Covers the full training stack: continuous pre-training, supervised fine-tuning, reward modeling, PPO, DPO, KTO, and ORPO
- ✓LoRA and QLoRA down to 2-bit, with quantization backends AQLM, AWQ, GPTQ, LLM.int8, HQQ, and EETQ
- ✓Bundles GaLore, BAdam, APOLLO, Adam-mini, Muon, DoRA, LongLoRA, LoRA+, LoftQ, and PiSSA as ready-to-flag training algorithms
- ✓Speeds training with FlashAttention-2, Unsloth, Liger Kernel, and KTransformers integrations
- ✓LlamaBoard Web UI runs the same jobs as the CLI with zero code
- ✓Serves fine-tuned models through an OpenAI-style API backed by a vLLM or SGLang worker
- ✓Logs experiments to TensorBoard, Wandb, MLflow, or SwanLab
Installing LlamaFactory
The README references a dedicated Installation section (and separate docs pages for Ascend NPU and AMD GPU setups) that isn't reproduced in the material available here, so the exact pip/git command isn't clearly documented in this brief. What is documented: zero-setup starts via a free Colab notebook, a free-trial PAI-DSW notebook, and free AMD GPU Cloud credits — all linked from the README — plus a Documentation site (marked work-in-progress) at llamafactory.readthedocs.io for the full local/cloud setup.
Fine-tuning with the CLI and Web UI
Local training starts from the CLI documented under the README's Getting Started/Quickstart section; the no-code route is `train_web.py`, which the project describes as an all-in-one Web UI for training, evaluation, and inference — this is what powers LlamaBoard. No custom loop needed. Cloud users can skip local setup entirely: the README links a free Colab notebook, a free-trial PAI-DSW notebook, and AMD GPU Cloud credits for fine-tuning without provisioning hardware. Serving a checkpoint afterward goes through the built-in OpenAI-style API, with vLLM or SGLang set via the `infer_backend` argument for faster concurrent inference; the project can also export a checkpoint straight to an Ollama modelfile.
Strengths
- ✓One framework covers pre-training, SFT, reward modeling, and DPO/KTO/ORPO preference tuning instead of stitching together separate libraries
- ✓LlamaBoard gives non-infra teammates a working Web UI without touching a config file
- ✓Quantization and speed tricks (QLoRA, FlashAttention-2, Unsloth, Liger Kernel) are already wired in as flags, not integrations you build yourself
- ✓Cloud on-ramps (Colab, PAI-DSW, AMD GPU Cloud) mean testing it costs nothing before committing hardware
- ✓The README cites production use by Amazon, NVIDIA, and Aliyun write-ups, so integration paths for major clouds are at least documented somewhere
Hardware requirements and limitations
- △Documentation is explicitly marked work-in-progress on its own docs site, so some workflows are easier to find by reading the changelog than a guide
- △The README itself warns that sites other than its own listed links are unauthorized third parties — worth knowing before searching for install guides or mirrors
- △Full-parameter fine-tuning of the larger model families still needs serious GPU memory; the README's own example for fitting a 70B model onto 2x24GB GPUs requires the FSDP+QLoRA combination, not a plain run
- △Broad method coverage (a dozen-plus tuning algorithms, multiple quantization backends) means more surface area to configure correctly than a narrower, single-method tool
Alternatives to LlamaFactory
Frequently asked questions
LlamaFactory's README lists support for 100+ model families, including LLaMA, Qwen3, DeepSeek, Gemma 3, GLM-4.5, Mixtral-MoE, and Phi, plus multimodal models like LLaVA, Qwen3-VL, and MiniCPM-o for image, video, and audio tasks.
LlamaFactory is free and open source, released under the Apache-2.0 license on GitHub — free to use, modify, and self-host for both personal and commercial fine-tuning projects.
Hardware needs scale with the fine-tuning method: full-parameter tuning of large models needs multiple high-memory GPUs, while LoRA and QLoRA cut that sharply — the README's own example uses FSDP+QLoRA to fit a 70B model onto just 2x24GB GPUs. LlamaFactory also supports Ascend NPU and AMD GPU hardware with dedicated setup docs.
Both are open-source frameworks for LLM fine-tuning, but they split configuration differently: Axolotl centers on a single YAML file per run, while LlamaFactory pairs a CLI with the LlamaBoard Web UI for teams that want a GUI option, not just config files. The choice usually comes down to whether you prefer editing YAML or clicking through a UI.
LlamaFactory integrates vLLM as one of its two inference backends — setting `infer_backend: vllm` in a config routes serving through vLLM for faster, concurrent inference of a fine-tuned checkpoint. SGLang is supported the same way as an alternative backend.
LlamaFactory supports LLaVA and other multimodal models directly, covering tasks like image understanding, visual grounding, video recognition, and audio understanding alongside text-only fine-tuning, with models such as Qwen2.5-VL, Qwen3-VL, and MiniCPM-o also on its supported list.
The problem it solves
Fine-tuning an LLM normally means picking a method (LoRA? QLoRA? full-parameter?), then hand-writing glue code between a model loader, a PEFT wrapper, a trainer, and a serving stack — and redoing that glue for every new architecture Meta, Qwen, or DeepSeek ships. LlamaFactory replaces that glue with one config surface: the same CLI/Web UI handles pre-training, SFT, reward modeling, and preference tuning across a wide model catalog, so a new release like Qwen3 or Gemma 3 becomes a template swap instead of a rewrite.
Best use cases
- •Full or LoRA/QLoRA fine-tuning of an open chat model on a single consumer GPU
- •Multimodal SFT for image-understanding or visual-grounding tasks with models like LLaVA or Qwen2.5-VL
- •Preference tuning (DPO/KTO/ORPO) a base model into a chat or safety-aligned variant
- •Standing up an OpenAI-compatible inference endpoint for a fine-tuned model via vLLM or SGLang
- •No-code experimentation through LlamaBoard for teams without a training-infra engineer on hand
Who should try it — and who should skip
Reach for LlamaFactory if you already know which model and method (LoRA, QLoRA, DPO...) you want and just need it wired up fast — a data scientist testing a fine-tune before writing production code, or a small team without a dedicated ML infra person, both get real mileage from LlamaBoard and the pre-built recipes. Skip it if you need a training loop you can rewrite line by line, or if your fine-tuning method isn't one of the ones it already supports — you'll spend more time working around the framework than the loop would have taken to write.
Related repositories
Is LlamaFactory worth your time?
ChatGPT, Claude and Perplexity can all read this page. Ask one of them what it makes of LlamaFactory.