pdf-inspector: Rust PDF Parser for Text & Markdown
/pdf-inspector
pdf-inspector is a Rust library that classifies PDFs as text-based, scanned, image-based, or mixed, then extracts and converts the text-based ones to Markdown without touching OCR. Reach for it when you're building a document pipeline that needs to skip OCR costs for PDFs that already contain real text. Skip it if your PDFs are mostly scans — pdf-inspector detects that case correctly but doesn't extract from it.
Introducing pdf-inspector
pdf-inspector is a Rust library, built by Firecrawl, that tells text-based PDFs apart from scanned or image-based ones and turns the text-based pages into structured Markdown — headings, lists, tables, and links included. It ships official bindings for Python, Node.js, and browser WebAssembly, plus two CLI tools, pdf2md and detect-pdf, for one-off jobs from the terminal.
Key capabilities for PDF analysis
- ✓Smart classification in ~10-50ms: samples content streams to label a PDF as TextBased, Scanned, ImageBased, or Mixed, with a 0.0-1.0 confidence score and per-page OCR routing.
- ✓Position-aware text extraction that captures font info and X/Y coordinates for every text item.
- ✓Markdown conversion that reconstructs headings (H1-H4 from font-size ratios), bullet/numbered/letter lists, monospace-detected code blocks, tables, and bold/italic runs.
- ✓Dual-mode table detection: rectangle-based parsing of PDF drawing operators plus heuristic detection from text alignment, covering financial tables and continuation tables split across pages.
- ✓CID font handling via ToUnicode CMap decoding for Type0/Identity-H fonts, across UTF-16BE, UTF-8, and Latin-1 encodings.
- ✓Multi-column layout detection that reconstructs newspaper-style reading order and supports RTL text.
- ✓Encoding-issue detection that flags broken font encodings so a caller knows to fall back to OCR instead of returning garbled text.
- ✓A single document load shared between detection and extraction, so classifying and then extracting the same file doesn't re-parse it.
Practical applications and smart routing
- •Pre-filtering a document ingestion pipeline: classify every incoming PDF first, extract text-based ones locally, and route only the scanned or image-based ones to an OCR service.
- •Converting reports, research papers, invoices, and legal documents into clean Markdown for search indexing or for feeding into an LLM's context window.
- •Flagging pages with pages_needing_ocr so a caller can send only specific pages of a Mixed PDF to OCR instead of the whole document.
- •Running the same detection and conversion logic client-side in a browser via the WASM build, for tools that can't ship every PDF to a backend.
Installation across platforms
Python: pip install maturin, then maturin develop --release to build the PyO3 bindings locally (the Quick Start doesn't mention a prebuilt wheel). Node.js: npm install @firecrawl/pdf-inspector for the napi-rs bindings. Browser: npm install @firecrawl/pdf-inspector-wasm for the WebAssembly build. Rust: cargo add pdf-inspector, or add pdf-inspector = "0.1" to Cargo.toml manually, pulling from crates.io. CLI: cargo install pdf-inspector installs the pdf2md and detect-pdf binaries; from a source checkout, run them via cargo run --bin pdf2md -- document.pdf instead.
Code examples for integration
In Python, pdf_inspector.process_pdf("document.pdf") returns an object with pdf_type (one of "text_based", "scanned", "image_based", "mixed") and a markdown string that's None when there's nothing to extract. In Node.js, processPdf(readFileSync('document.pdf')) from @firecrawl/pdf-inspector returns the equivalent pdfType (PascalCase: "TextBased", "Scanned", "ImageBased", "Mixed") and markdown. The WASM build mirrors the Node API but takes a Uint8Array you fetch yourself: processPdf(pdf) after await init(). In Rust, pdf_inspector::process_pdf("document.pdf")? returns a result.pdf_type enum and an Option<String> in result.markdown. The pdf2md CLI wraps the same logic for shell use — pdf2md document.pdf --json pipes structured output, --select-pages 1,3,5-10 limits it to specific pages, and --compact collapses long dot leaders and similar padding for token-efficient output.
Strengths
- ✓The benchmark's own numbers put pdf-inspector at 0.875 overall on the 200-PDF opendataloader-bench corpus with a 0.470s median run, versus pymupdf4llm's 0.735/17.117s and markitdown's 0.589/16.165s — a real gap, not a marginal one, given all five parsers were scored the same way with OCR disabled.
- ✓The classify-then-route numbers in the README — about 20ms to classify a PDF, about 150ms to extract a TextBased one locally, versus 2-10s for an OCR service — make it straightforward to estimate what a pipeline would save by adding this step.
- ✓Pure Rust with a single dependency on lopdf and no ML model weights: nothing to license, nothing to run on a GPU, and the WASM build ships the identical parser to a browser.
- ✓MIT license, so there's no copyleft obligation to work around when embedding it in a closed-source pipeline.
Understanding pdf-inspector's scope
- △No OCR of its own — pdf-inspector detects scanned and image-based pages and lists which ones need OCR, but extracting text from them is left to a separate service.
- △The Python bindings, per the Quick Start, are built locally with maturin develop --release rather than installed from a prebuilt wheel.
- △The benchmark README compares pdf-inspector against four other engines on one 200-PDF corpus with OCR disabled; it says nothing about how classification confidence holds up on PDFs from unusual or malformed tooling outside that set.
- △Markdown conversion depends on font-size ratios and font-name patterns (Bold, Italic, Courier, etc.) to infer structure, so a PDF that doesn't encode those signals cleanly will produce flatter Markdown.
Comparing with other PDF parsers
Frequently asked questions
pdf-inspector sorts PDFs into four categories — TextBased, Scanned, ImageBased, or Mixed — by sampling text and image operators in the content streams, and returns a 0.0-1.0 confidence score with the result.
pdf-inspector does not run OCR itself. It flags scanned or image-based pages, including a pages_needing_ocr list for Mixed PDFs, so a caller can route just those pages to a separate OCR service.
pdf-inspector is written in Rust and ships official bindings for Python (via PyO3/maturin), Node.js (via napi-rs), and browser WebAssembly, plus a Rust crate on crates.io and two CLI tools, pdf2md and detect-pdf.
pdf-inspector rebuilds headings from font-size ratios, detects bullet/numbered/letter lists and monospace code blocks, formats tables from both drawn rectangles and aligned text, and turns bold/italic font patterns and URLs into Markdown syntax.
pdf-inspector is released under the MIT license, so it's safe to embed in a closed-source or commercial pipeline without copyleft obligations.
The README's benchmark ran pdf-inspector across a 200-PDF corpus with a 0.470s median run time, and its own use-case example estimates roughly 20ms to classify and about 150ms to extract a text-based PDF locally — figures that support high-volume use, though they come from the project's own benchmark rather than an independent one.
The problem it solves
Most document pipelines treat every incoming PDF the same way: run it through OCR, even when the PDF already has selectable text embedded from a word processor or a LaTeX export. The README puts that waste at roughly 54% of PDFs — the majority — that don't need OCR at all, yet get sent through it anyway because there's no fast, local, dependency-light step to tell text-based and scanned files apart before they reach an OCR service.
Who should try it — and who should skip
Reach for pdf-inspector if you're building an ingestion pipeline in Rust, Python, or Node.js that receives a mix of real PDFs and scans, and you want to skip OCR costs for the ones that already have text — the classification step alone, at roughly 10-50ms per document, pays for itself once volume is high enough. Skip it if every PDF you handle is scanned or image-based from the start: pdf-inspector will correctly tell you that, but it hands the actual OCR work to something else, so you'd be adding a step that never finds anything to extract.
Related repositories
Want a second opinion on pdf-inspector?
Ask an AI that can read this page — one click and you get its take on pdf-inspector.