RAG-Anything: Multimodal RAG for PDFs and Images
RAG-Anything is worth adopting once your documents stop being plain text — it fills the multimodal gap LightRAG does not cover alone. The setup cost is real: MinerU for parsing, LibreOffice for Office formats, and your own LLM, vision, and embedding functions wired in by hand. For PDFs mixing figures, tables, and equations, that cost buys one pipeline instead of stitching together separate OCR, table-extraction, and captioning tools.
What is RAG-Anything?
RAG-Anything is a Python framework that extends HKUDS's LightRAG into a multimodal retrieval pipeline. It parses PDFs, Office files, and images using MinerU, routes text, tables, equations, and figures through dedicated processors, and builds a shared knowledge graph so a single query can retrieve answers from mixed content rather than text alone.
Key features
- ✓Three parser backends — MinerU (default), Docling, or PaddleOCR — selectable via the `parser` config field, with an `auto`/`ocr`/`txt` parse_method.
- ✓Dedicated processors for images, tables, and equations, with the equation parser outputting native LaTeX.
- ✓Builds a multimodal knowledge graph on top of LightRAG, linking entities across modalities with belongs_to hierarchy chains and weighted relationship scores.
- ✓VLM-Enhanced Query mode feeds page images straight into your vision model alongside text context for image-heavy documents.
- ✓Direct Content List Insertion lets you skip RAG-Anything's own parser and feed in pre-parsed content from another pipeline instead.
- ✓Hybrid retrieval fuses vector similarity with graph traversal and weights results by modality relevance.
- ✓process_folder_complete runs a whole document folder through the pipeline in one call.
Installing RAG-Anything
pip install raganything gets you the base package; add extras with pip install 'raganything[all]' for every optional format, or narrower ones like [image] (needs Pillow, adds BMP/TIFF/GIF/WebP support) and [text] (needs ReportLab, adds TXT/MD). Installing from source instead uses uv: clone the repo, run uv sync (or uv sync --all-extras), then uv run python examples/raganything_example.py --help to confirm it works. RAG-Anything doesn't bundle MinerU. PDF parsing depends on it being installed separately — run mineru --version and RAGAnything().check_parser_installation() to confirm it's wired up correctly. Anything in .doc/.docx/.ppt/.pptx/.xls/.xlsx format needs a separate LibreOffice install (brew install --cask libreoffice on macOS, apt-get install libreoffice on Ubuntu/Debian) — RAG-Anything shells out to it rather than parsing Office formats natively.
Running an end-to-end multimodal RAG pipeline
A typical run wires four config pieces together: RAGAnythingConfig (working_dir, parser, parse_method, and per-modality enable_image/table/equation_processing flags), an llm_model_func, a vision_model_func, and an embedding_func. RAG-Anything doesn't call any LLM API directly, so you supply these yourself — usually with LightRAG's openai_complete_if_cache and openai_embed helpers pointed at an OpenAI-compatible endpoint. Call rag.process_document_complete(file_path=..., output_dir=..., parse_method='auto') to parse and index one file, then query it with rag.aquery('...', mode='hybrid') for plain-text answers, or rag.aquery_with_multimodal(query, multimodal_content=[{'type': 'equation', 'latex': ...}], mode='hybrid') when the question points at a specific image, table, or formula. process_folder_complete does the same across a whole directory. If you'd rather skip RAG-Anything's own document-processing loop, ImageModalProcessor and TableModalProcessor can be attached directly to a plain LightRAG instance instead.
Strengths
- ✓One pipeline covers parsing, modality-specific processing, graph construction, and retrieval instead of gluing together separate OCR, table, and caption tools.
- ✓Equation handling outputs native LaTeX, which most text-only RAG setups don't attempt.
- ✓Direct Content List Insertion lets you swap in a different upstream parser and still use RAG-Anything's graph and retrieval layers.
- ✓MIT license leaves no ambiguity about commercial use.
Limitations and external dependencies
- △PDF parsing depends on MinerU being installed and configured separately — it's not bundled, and check_parser_installation() exists because that step fails often enough to need its own diagnostic.
- △Office file support (.doc/.docx/.ppt/.pptx/.xls/.xlsx) requires a full LibreOffice install, a heavy dependency to pull into a Python project.
- △There's no default LLM, vision, or embedding provider — you write and wire those functions yourself before you can index anything.
- △It's built directly on LightRAG's storage and graph model, so adopting RAG-Anything means adopting LightRAG's architecture too, not bolting on a standalone add-on.
Multimodal RAG alternatives to consider
Frequently asked questions
RAG-Anything requires a separate LibreOffice installation to process Office formats like .doc, .docx, .ppt, .pptx, .xls, and .xlsx. The README lists install commands for macOS, Windows, Ubuntu/Debian, and CentOS/RHEL, and without LibreOffice installed RAG-Anything cannot parse these file types.
RAG-Anything can process documents without MinerU: the `parser` config field also accepts `docling` and `paddleocr` as alternative backends, so MinerU isn't the only parsing option available.
RAG-Anything doesn't ship a built-in model — you supply an llm_model_func, a vision_model_func, and an embedding_func yourself, typically using LightRAG's openai_complete_if_cache and openai_embed helpers pointed at an OpenAI-compatible API. Any model reachable through an OpenAI-compatible endpoint can be wired in.
RAG-Anything extracts entities from text, images, tables, and equations and links them with belongs_to relationship chains that preserve the original document hierarchy, then assigns weighted relevance scores to each relationship for retrieval ranking. This graph sits on top of LightRAG's storage layer.
RAG-Anything is released under the MIT licence, which permits commercial and production use without royalty obligations. Whether it's production-ready for a given team depends on tolerance for its setup requirements — MinerU, optional LibreOffice, and hand-wired model functions — which the README doesn't shortcut.
RAG-Anything is built directly on top of LightRAG, and the README notes that LightRAG now integrates RAG-Anything natively for multimodal RAG, so an existing LightRAG deployment can add RAG-Anything's parsing and modality processors rather than switching frameworks.
The problem it solves
Developers building RAG over documents that aren't pure text — PDFs with charts, scanned reports, slide decks — end up hand-rolling separate OCR, table-extraction, and image-captioning steps before they can even reach the retrieval part, and gluing those outputs into one index rarely works cleanly since LightRAG and most RAG frameworks only ingest text. RAG-Anything folds document parsing, per-modality processing, and graph-based indexing into a single pipeline built specifically to extend LightRAG to mixed content.
Best use cases
- •Turning a folder of research PDFs with charts and equations into a queryable knowledge base for a lab or team.
- •Indexing financial or technical reports where the numbers live in tables, not paragraphs.
- •Adding multimodal support to an existing LightRAG deployment without switching retrieval backends.
- •Building internal documentation search across mixed Office files, scanned PDFs, and screenshots.
Who should try it — and who should skip
Try RAG-Anything if you're already comfortable wiring your own LLM, vision, and embedding functions, and your documents genuinely mix text with figures, tables, or equations — a research team indexing papers, or a company processing scanned reports and slide decks alongside plain text. Skip it if your content is mostly plain text (LightRAG alone is simpler), or if you don't want to manage a MinerU-plus-LibreOffice toolchain on top of your regular Python dependencies.
Related repositories
Curious whether RAG-Anything is right for you?
Let ChatGPT, Claude, or Perplexity look into it — click below and see what AI actually says about RAG-Anything.
