LlamaIndex: RAG Framework for LLM Apps

What is LlamaIndex?

LlamaIndex is an open-source Python framework that its own README calls a "data framework" for building LLM applications on private or domain-specific data. It ships data connectors for sources like PDFs, APIs, and SQL, tools to structure that data into indices, and a retrieval/query layer that returns context-augmented answers instead of raw model guesses. A high-level API handles ingest-and-query in a handful of lines, with lower-level APIs available for more control.

Why LlamaIndex was developed

An LLM trained on public web data doesn't know your internal wiki, your product's PDFs, or the rows in your SQL database, and pasting that data straight into a prompt stops scaling past a few pages. The README frames the gap directly: augmenting a general-purpose LLM with your own private data so it answers from that data instead of guessing from pretraining. LlamaIndex is built to be the ingestion-to-retrieval plumbing for that specific gap — connectors that pull data in, indices that structure it, a query layer that feeds relevant chunks back into the prompt — rather than a general agent-building framework or a model host.

Key capabilities of LlamaIndex

• ✓Data connectors that ingest APIs, PDFs, docs, SQL, and other formats, per the README's own list of supported source types.
• ✓Multiple index and graph structures for organizing ingested data so an LLM can query it.
• ✓A retrieval/query engine that takes an LLM prompt and returns retrieved context plus a knowledge-augmented answer, not a raw completion.
• ✓Two install paths: the `llama-index` starter package bundles core plus a selection of integrations, or `llama-index-core` plus individual integration packages — over 300 are listed on LlamaHub — for the LLM, embedding, and vector store providers you pick yourself.
• ✓Namespaced imports that show what you're using at a glance: `llama_index.core.xxx` is the core package, `llama_index.xxx.yyy` without `core` is an integration package.
• ✓A layered API — a high-level path that ingests and queries a directory of files in about 5 lines of code, and lower-level APIs to swap out data connectors, indices, retrievers, query engines, or reranking modules.
• ✓Disk persistence via `StorageContext` — save an index to `./storage` and reload it later instead of rebuilding it from scratch every run.
• ✓LlamaParse, a separate platform for agentic OCR and document parsing across 130+ formats, plus structured Extract, Index, Split, and document-agent tools — usable standalone or wired into the LlamaIndex framework.

Getting started with LlamaIndex

There are two paths, both via pip. For most projects, `pip install llama-index` gets you the starter package — core LlamaIndex plus a preselected set of integrations, no extra configuration needed. If you want to pick your own LLM, embedding, and vector store providers, install `llama-index-core` and add only the integration packages you need — the README's own example is `pip install llama-index-core llama-index-llms-openai llama-index-llms-ollama llama-index-embeddings-huggingface`, chosen from over 300 packages listed on LlamaHub. For an OpenAI-backed setup you then set the `OPENAI_API_KEY` environment variable before writing any LlamaIndex code. Both paths are plain Python packages; no separate CLI installer or system dependency is documented in the README.

Building your first LLM application

The core loop is four calls: load a folder of files with `SimpleDirectoryReader`, build a `VectorStoreIndex` from the documents, turn the index into a `query_engine`, then call `.query()` with a question — the README's own example runs this against OpenAI by default. Swap `Settings.llm` to `Ollama` and `Settings.embed_model` to a HuggingFace embedding model to run the same pipeline against a locally hosted model instead, which is the alternate example the README shows. By default an index lives in memory only; call `index.storage_context.persist()` to write it to `./storage` on disk, and reload it later with `load_index_from_storage()` against a rebuilt `StorageContext` instead of re-indexing your documents every run.

Other LLM development frameworks

Common questions about LlamaIndex

Best use cases

• •Q&A over an internal knowledge base — point a data connector at your docs, PDFs, or wiki export and query it through an LLM.
• •Retrieval-augmented search where answers need to cite a specific SQL table or API response instead of the model's training data.
• •Swapping in a self-hosted LLM instead of OpenAI — the README's own example runs the same index against Ollama with a local Llama 3.1 model.
• •Document-heavy extraction pipelines where LlamaParse handles OCR and parsing before the framework indexes the output.
• •Prototyping a RAG chatbot fast with the 5-line high-level API, then dropping to the lower-level retriever/query-engine APIs for more control.

Who should try it — and who should skip

Try LlamaIndex if you're building anything that needs an LLM to answer questions grounded in a specific data source — internal docs, a SQL database, a folder of PDFs — and you'd rather use existing data connectors and index structures than write the ingest-chunk-embed-retrieve loop by hand. Skip it if you're shipping a simple chat wrapper around one LLM provider with no external data to retrieve, since the package-selection overhead buys you nothing there, or if you need document OCR beyond basic file loading, since that's LlamaParse territory and a separate product.

Related repositories