百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
P

pdf-brain

> 编程语言
开源

使用 libsql w/vectors 进行向量搜索的本地 PDF 知识库

371 stars0 点赞1 次浏览
访问官网GitHub

工具介绍

使用 libsql w/vectors 进行向量搜索的本地 PDF 知识库

pdf-brain

Local PDF & Markdown knowledge base with semantic search and AI-powered enrichment.

Works with PDFs AND Markdown files - Index your research papers, books, notes, docs, and any .md files in one unified, searchable knowledge base.

┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  PDF / MD   │────▶│   Ollama    │────▶│   Ollama    │────▶│   libSQL    │
│  (extract)  │     │    (LLM)    │     │ (embeddings)│     │  (vectors)  │
└─────────────┘     └─────────────┘     └─────────────┘     └─────────────┘
      │                   │                   │                   │
   pdf-parse         llama3.2:3b        mxbai-embed          HNSW index
   + markdown        enrichment          1024 dims           cosine sim

Features

  • PDF + Markdown - Index .pdf and .md files with the same workflow
  • Local-first - Everything runs on your machine, no API costs
  • AI enrichment - LLM extracts titles, summaries, tags, and concepts
  • SKOS taxonomy - Organize documents with hierarchical concepts
  • Vector search - Semantic search via Ollama embeddings
  • Hybrid search - Combine vector similarity with full-text search
  • MCP server - Use with Claude, Cursor, and other AI assistants

Quick Start

Note: pdf-brain is agent-first and emits a single JSON envelope to stdout by default.
Use --format text for human-readable output (and TUI/progress rendering), or inspect the machine contract via pdf-brain capabilities.

bash
# 1. Install (standalone binary, no runtime needed)
curl -fsSL https://raw.githubusercontent.com/joelhooks/pdf-brain/main/scripts/install.sh | bash

# 2. Install Ollama (macOS)
brew install ollama

# 3. Pull required models
ollama pull mxbai-embed-large   # embeddings (required)
ollama pull llama3.2:3b         # enrichment (optional but recommended)

# 4. Start Ollama
ollama serve

# 5. Initialize (creates DB + seeds starter taxonomy)
pdf-brain init

# 6. Add your first document
pdf-brain add ~/Documents/paper.pdf --enrich

Installation

Prerequisites

Ollama is required for embeddings. The LLM model is optional but recommended for enrichment.

bash
# macOS
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows
# Download from https://ollama.com/download

Models

bash
# Required: Embedding model (1024 dimensions)
ollama pull mxbai-embed-large

# Recommended: Local LLM for enrichment
ollama pull llama3.2:3b

# Start Ollama server
ollama serve

Install pdf-brain

bash
# Standalone binary (no runtime needed)
curl -fsSL https://raw.githubusercontent.com/joelhooks/pdf-brain/main/scripts/install.sh | bash

# or via npm
npm install -g pdf-brain

CLI Reference

Agent Output (Default)

pdf-brain is optimized for agentic workflows: stdout is machine-readable by default.

  • --format json|ndjson|text (default: json)
  • --pretty pretty-print JSON
  • --quiet (alias: --no-hints) omit nextActions
  • --log-level silent|error|info|debug (logs go to stderr)

Discover the full command/tool contract (including JSON Schemas) at runtime:

bash
pdf-brain capabilities

Basic Commands

bash
# Check Ollama status
pdf-brain check

# Show library stats
pdf-brain stats

# Initialize library (creates DB, seeds taxonomy)
pdf-brain init

Adding Documents

bash
# Add a PDF
pdf-brain add /path/to/document.pdf

# Add a Markdown file
pdf-brain add /path/to/notes.md

# Add from URL (PDF or MD)
pdf-brain add https://example.com/paper.pdf
pdf-brain add https://raw.githubusercontent.com/user/repo/main/README.md

# Add with manual tags
pdf-brain add document.pdf --tags "ai,agents,research"

# Add with AI enrichment (extracts title, summary, concepts)
pdf-brain add document.pdf --enrich
pdf-brain add notes.md --enrich

Searching

bash
# Semantic search (uses embeddings)
pdf-brain search "context engineering patterns"

# Full-text search only (faster, no embeddings)
pdf-brain search "context engineering" --fts

# Hybrid search (combines both)
pdf-brain search "machine learning" --hybrid

# Limit results
pdf-brain search "query" --limit 5

# Expand context around matches
pdf-brain search "query" --expand 500

Managing Documents

bash
# List all documents
pdf-brain list

# List by tag
pdf-brain list --tag ai

# Get document details
pdf-brain read "document-title"

# Remove a document
pdf-brain remove "document-title"

# Update tags
pdf-brain tag "document-title" "new,tags,here"

Taxonomy Commands

The taxonomy system uses SKOS (Simple Knowledge Organization System) for hierarchical concept organization.

bash
# List all concepts
pdf-brain taxonomy list

# Show concept tree
pdf-brain taxonomy tree

# Show subtree from a concept
pdf-brain taxonomy tree programming

# Search concepts
pdf-brain taxonomy search "machine learning"

# Add a new concept
pdf-brain taxonomy add ai/transformers --label "Transformers" --broader ai-ml

# Assign concept to document
pdf-brain taxonomy assign "doc-id" "programming/typescript"

# Seed taxonomy from JSON file
pdf-brain taxonomy seed --file data/taxonomy.json

Bulk Ingest

Recursively ingest directories containing PDFs and/or Markdown files:

…

Supported formats:

  • .pdf - Research papers, books, documents
  • .md - Notes, documentation, Obsidian vaults, READMEs

Enrichment

When you add documents with --enrich, the LLM extracts:

Field Description
title Clean, properly formatted title
author Author name(s) if detectable
summary 2-3 sentence summary
documentType book, paper, tutorial, guide, article, etc.
category Primary category
tags 5-10 descriptive tags
concepts Matched concepts from your taxonomy
proposedConcepts New concepts the LLM suggests adding

LLM Providers

Enrichment supports multiple providers via the config system:

bash
# Check current config
pdf-brain config show

# Use local Ollama (default)
pdf-brain config set enrichment.provider ollama
pdf-brain config set enrichment.model llama3.2:3b

# Use AI Gateway (Anthropic, OpenAI, etc.)
pdf-brain config set enrichment.provider gateway
pdf-brain config set enrichment.model anthropic/claude-haiku-4-5
export AI_GATEWAY_API_KEY=your-key

# Provider priority: config > CLI flag > auto-detect
pdf-brain add paper.pdf --enrich              # uses config
pdf-brain add paper.pdf --enrich --provider ollama  # override

Enrichment Fallback

If LLM enrichment fails (API error, rate limit, malformed response), pdf-brain automatically falls back to heuristic-based enrichment:

  • Title: Cleaned from filename
  • Tags: Extracted from path, filename, and content keywords
  • Category: Inferred from directory structure

The actual error is logged so you can debug provider issues.

Taxonomy

The taxonomy is a hierarchical concept system for organizing documents. It ships with a starter taxonomy covering:

  • Programming - TypeScript, React, Next.js, Testing, Architecture, DevOps, AI/ML
  • Education - Instructional Design, Learning Science, Course Creation, Assessment
  • Business - Marketing, Copywriting, Bootstrapping, Product, Sales
  • Design - UX, Visual Design, Systems Thinking, Information Architecture
  • Meta - Productivity, Note-taking, Knowledge Management, Writing

Growing Your Taxonomy

When enriching documents, the LLM may propose new concepts. These are saved for review:

bash
# See proposed concepts from enrichment
pdf-brain taxonomy proposed

# Accept a specific concept
pdf-brain taxonomy accept ai/rag --broader ai-ml

# Accept all proposed concepts
pdf-brain taxonomy accept --all

# Reject a concept
pdf-brain taxonomy reject ai/rag

# Clear all proposals
pdf-brain taxonomy clear-proposed

# Manually add a concept
pdf-brain taxonomy add ai/rag --label "RAG" --broader ai-ml

# Or edit data/taxonomy.json and re-seed
pdf-brain taxonomy seed --file data/taxonomy.json

Custom Taxonomy

Create your own taxonomy.json:

json
{
  "concepts": [
    { "id": "cooking", "prefLabel": "Cooking" },
    { "id": "cooking/baking", "prefLabel": "Baking" },
    { "id": "cooking/grilling", "prefLabel": "Grilling" }
  ],
  "hierarchy": [
    { "conceptId": "cooking/baking", "broaderId": "cooking" },
    { "conceptId": "cooking/grilling", "broaderId": "cooking" }
  ]
}
bash
pdf-brain taxonomy seed --file my-taxonomy.json

Configuration

Config File

pdf-brain stores configuration in $PDF_LIBRARY_PATH/config.json:

bash
# Show all config
pdf-brain config show

# Get a specific value
pdf-brain config get enrichment.provider

# Set a value
pdf-brain config set enrichment.model anthropic/claude-haiku-4-5

Config Options

json
{
  "ollama": {
    "host": "http://localhost:11434"
  },
  "embedding": {
    "provider": "ollama",
    "model": "mxbai-embed-large"
  },
  "enrichment": {
    "provider": "gateway",
    "model": "anthropic/claude-haiku-4-5"
  },
  "judge": {
    "provider": "gateway",
    "model": "anthropic/claude-haiku-4-5"
  }
}
Setting Default Description
ollama.host http://localhost:11434 Ollama API endpoint
embedding.provider ollama Embedding provider (ollama only)
embedding.model mxbai-embed-large Embedding model (1024 dims)
enrichment.provider ollama LLM provider: ollama or gateway
enrichment.model llama3.2:3b Model for document enrichment
judge.provider ollama Provider for concept deduplication
judge.model llama3.2:3b Model for judging duplicate concepts

Environment Variables

Variable Default Description
PDF_LIBRARY_PATH ~/Documents/.pdf-library Library storage location
OLLAMA_HOST http://localhost:11434 Ollama API endpoint
AI_GATEWAY_API_KEY - API key for AI Gateway
PDF_BRAIN_LOG_LEVEL silent stderr logging verbosity
PDF_BRAIN_QUERY_EMBED_CACHE_SIZE 256 Query embedding LRU cache size (0 disables)

AI Gateway

For cloud LLM providers (Anthropic, OpenAI, etc.), use the AI Gateway:

bash
# Set your API key
export AI_GATEWAY_API_KEY=your-key

# Configure to use gateway
pdf-brain config set enrichment.provider gateway
pdf-brain config set enrichment.model anthropic/claude-haiku-4-5

# Other supported models:
# - anthropic/claude-sonnet-4-20250514
# - openai/gpt-4o-mini
# - openai/gpt-4o

Storage

~/Documents/.pdf-library/
├── library.db          # libSQL database (vectors, FTS, metadata, taxonomy)
├── library.db-shm      # Shared memory (WAL mode)
├── library.db-wal      # Write-ahead log
└── downloads/          # PDFs downloaded from URLs

Database Size

The database can get large due to vector index overhead. For ~500k chunks:

Component Size Notes
Text content ~180MB Actual chunk text
Embeddings ~1.9GB 500k × 1024 dims × 4 bytes
Vector index ~48GB HNSW neighbor graphs (~10

Issues· 0 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

TypeScript

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月17日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言