MCP Local RAG

Search private documents from an MCP client or the terminal without sending them to an embedding API.

mcp-local-rag indexes PDF, DOCX, Markdown, and text files on your machine. Search combines semantic similarity with keyword matching, so queries can match both intent and exact technical terms such as API names, class names, and error codes.

Features

• Runs locally: Document parsing, embeddings, storage, and search run on your machine. After the initial model download, text ingestion and search work offline.

• Hybrid search: Semantic retrieval finds related concepts, while keyword matching boosts exact technical terms.

• Configurable embeddings: Choose a Hugging Face embedding model that fits the language and domain of your documents.

• Semantic chunking: Documents are split at topic boundaries instead of fixed character counts. Markdown code blocks stay intact.

• MCP and CLI: Use the same index from an AI coding tool or directly from the terminal.

No API key, Docker, Python, or external database is required.

Quick Start

Requirements

• Node.js 22 or later

• Internet access on first use to download the npm package and embedding model

• A directory containing the documents you want to search

Set BASE_DIR to that directory. It is also the security boundary for file operations. Replace /absolute/path/to/your/documents below with the directory's absolute path.

mcp-local-rag uses the standard MCP protocol over a local stdio server, so it works with AI coding tools and other MCP hosts that support local MCP servers.

Use one of the examples below, or register npx -y mcp-local-rag and set BASE_DIR using your client's MCP configuration format.

For Claude Code: Run this command:

claude mcp add local-rag --scope user --env BASE_DIR=/absolute/path/to/your/documents -- npx -y mcp-local-rag

For Codex: Add to ~/.codex/config.toml:

[mcp_servers.local-rag]
command = "npx"
args = ["-y", "mcp-local-rag"]

[mcp_servers.local-rag.env]
BASE_DIR = "/absolute/path/to/your/documents"

For OpenCode: Add to ~/.config/opencode/opencode.json (or opencode.jsonc):

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "local-rag": {
      "type": "local",
      "command": ["npx", "-y", "mcp-local-rag"],
      "environment": {
        "BASE_DIR": "/absolute/path/to/your/documents"
      }
    }
  }
}

For Cursor: Add to ~/.cursor/mcp.json:

{
  "mcpServers": {
    "local-rag": {
      "command": "npx",
      "args": ["-y", "mcp-local-rag"],
      "env": {
        "BASE_DIR": "/absolute/path/to/your/documents"
      }
    }
  }
}

Restart the client, then ask it to build the index:

Sync all documents in the configured root and wait until it finishes.

The first sync downloads the default embedding model (about 90 MB) and may take 1–2 minutes before ingestion starts. Later runs use the local cache.

Once the sync completes:

What does the API documentation say about authentication?

CLI Quick Start

To use the CLI without an MCP client:

npx mcp-local-rag ingest ./docs/
npx mcp-local-rag query "authentication API"

The CLI uses the current directory as its document root by default. Run both commands from the same directory so they use the same default index, or set BASE_DIR and DB_PATH explicitly.

Why This Exists

Some document sets cannot be sent to a hosted embedding service because of confidentiality or organizational policy. Keeping the index local makes them searchable without adding a per-query API cost.

Semantic search alone can miss exact identifiers that matter in technical documentation. Keyword reranking keeps those terms visible without giving up natural-language retrieval.

Supported Content

Input How to ingest

PDF, DOCX, TXT, Markdown File ingestion or directory sync

HTML already fetched by the client ingest_data; cleaned with Readability and converted to Markdown

Plain text or Markdown held in memory ingest_data with a stable source identifier

HTML fetching is not built into the server. An MCP client can fetch a page and pass its HTML to ingest_data.

Excel, PowerPoint, standalone images, and source-code file extensions are not supported by file ingestion. PDFs can optionally use a local vision model to describe figures, but this is not OCR or image search.

MCP Tools

Tool Purpose

sync_start Reconcile the index with all configured roots or one path

sync_status Poll a running sync job

ingest_file Ingest or replace one file

ingest_data Ingest text, Markdown, or HTML already held by the client

query_documents Search with semantic matching and keyword boost

read_chunk_neighbors Read surrounding chunks from a search result

list_files Show supported files and their ingestion state

delete_file Delete an indexed file or an ingest_data item

status Show index and search status

Syncing a Document Root

sync_start ingests new and changed files, skips byte-identical files, and removes index entries for files that no longer exist:

Sync everything under the configured document roots and wait for completion.

The tool returns a jobId immediately. Clients should poll sync_status until its state becomes succeeded or failed. Sync does not generate visual captions. Set STORE_IMAGES=true in the MCP server environment to store supported PDF and DOCX images for new or changed files selected by sync; unchanged files remain skipped.

Only one sync job is retained by the server process. A newer job replaces a finished record, and restarting the server discards it.

Ingesting One File

ingest_file accepts PDF, DOCX, TXT, and Markdown. MCP file paths must be absolute and must stay inside a configured document root:

Ingest the document at /Users/me/docs/api-spec.pdf.

Re-ingesting the same path replaces its existing chunks.

Searching and Reading More Context

What does the API documentation say about authentication?
Find the documented behavior of ERR_CONNECTION_REFUSED.

Results contain the text, source path, title, chunk index, relevance score, and any images stored on that chunk. MCP returns each image as an image content block paired with its result identity; CLI query includes an images array of { imageIndex, mimeType, data } on every result. Pass the chunkIndex and either filePath or source from a result to read_chunk_neighbors when the answer needs more context:

Read the surrounding chunks for that authentication result.

Both query_documents and list_files accept an optional absolute scope path prefix, or a list of prefixes. A prefix matches the exact path and its descendants.

Ingesting HTML

Use ingest_data after the MCP client fetches a page:

Fetch https://example.com/docs and ingest the HTML.

The server extracts the main article, converts it to Markdown, and stores it under the supplied source identifier. Reusing the same source updates the existing content.

Respect the source site's terms and copyright when indexing external content.

PDF Visual Captions and Stored Images

Visual mode adds a generated caption for figure-heavy PDF pages. It is opt-in and does not load a vision model during normal ingestion.

Ingest /Users/me/docs/research-paper.pdf with visual: true.

npx mcp-local-rag ingest ./docs/research-paper.pdf --visual

Image storage is independent of visual captions. Set STORE_IMAGES=true for the MCP server, or pass --images to CLI ingestion and sync:

npx mcp-local-rag ingest ./docs/research-paper.pdf --images
npx mcp-local-rag sync ./docs/ --images

PDF storage uses detected figure/table regions. DOCX storage includes only PNG/JPEG images that the existing Mammoth conversion emits as <img>; charts, SmartArt, and shapes are not separately rendered. Stored images follow their surrounding text into the final semantic chunk and do not alter ranking, scores, or result count.

visual / --visual STORE_IMAGES / --images PDF behavior

false false Text only; no visual captions or returned images.

true false Generated captions become searchable text; no images are stored or returned.

true true Generated captions become searchable text, and images from matched chunks are returned inline.

false true Images are attached to nearby retained PDF text and returned inline for matched chunks; the VLM is not imported, loaded, or run.

Profile Model cache Use case

fast (default) about 250 MB Lightweight visual indexing

quality about 2.9 GB Figures containing labels, annotations, or other in-image text

Select the larger model with visualQuality: "quality" over MCP or --visual-quality quality over CLI. Measured CPU inference was about twice as slow as fast, though results depend on hardware and model updates.

Caption