CodeWiki

CodeWiki: Evaluating AI's Ability to Generate Holistic Documentation for Large-Scale Codebases

AI-Powered Repository Documentation Generation • Multi-Language Support • Architecture-Aware Analysis

Generate holistic, structured documentation for large-scale codebases • Cross-module interactions • Visual artifacts and diagrams

Quick Start • CLI Commands • Output Structure • Repo Docs • Paper

CodeWiki documents itself — browse the generated documentation for this repository at CodeWiki docs.

--- ## Quick Start ### 1. Install CodeWiki ```bash # Install from source pip install git+https://github.com/FSoft-AI4Code/CodeWiki.git # Verify installation codewiki --version ``` ### 2. Configure Your Environment CodeWiki supports multiple LLM providers: **OpenAI-compatible**, **Atlas Cloud**, **Anthropic**, **AWS Bedrock**, **Azure OpenAI**, plus subscription mode via **Claude Code** and **Codex** CLIs (no API key required). ``` … ``` **About Atlas Cloud.** [Atlas Cloud](https://www.atlascloud.ai) is a full-modal AI inference platform that exposes LLM, image, and video models (300+) behind a single OpenAI-compatible API, so it works with CodeWiki out of the box. Browse model IDs at the [models endpoint](https://api.atlascloud.ai/v1/models) and pick a strong coding model for `--main-model` / `--cluster-model`; their [coding plan](https://www.atlascloud.ai/console/coding-plan) offers budget-friendly API access. **Subscription mode** routes every LLM call through the local `claude` / `codex` CLI binary (via the [`caw`](https://github.com/zzjas/caw) library), so you can run CodeWiki on a Claude Pro/Max or Codex subscription instead of paying per-token API usage. Claude Code's built-in `Write`/`Edit`/`Bash` tools are disabled inside CodeWiki's agent loop so documentation writes still go through CodeWiki's Mermaid-validating editor. > **Note on model names.** In subscription mode the model string is forwarded directly to `claude --model` / `codex --model`, so use the bare CLI model name (e.g. `gpt-5.4`, `claude-sonnet-4-6`) — **not** the litellm-style `openai/…` or `anthropic/…` prefix used by `openai-compatible`. If you previously ran with `openai-compatible`, re-run `config set` for **both** `--main-model` and `--cluster-model` to clear any stale prefixes; `config set` only updates the keys you pass. ### 3. Generate Documentation ```bash # Navigate to your project cd /path/to/your/project # Generate documentation codewiki generate # Generate with HTML viewer for GitHub Pages codewiki generate --github-pages --create-branch ``` **That's it!** Your documentation will be generated in `./docs/` with comprehensive repository-level analysis. ### Usage Example --- ## What is CodeWiki? CodeWiki is an open-source framework for **automated repository-level documentation** across ten programming languages. It generates holistic, architecture-aware documentation that captures not only individual functions but also their cross-file, cross-module, and system-level interactions. ### Key Innovations | Innovation | Description | Impact | |------------|-------------|--------| | **Hierarchical Decomposition** | Dynamic programming-inspired strategy that preserves architectural context | Handles codebases of arbitrary size (86K-1.4M LOC tested) | | **Recursive Agentic System** | Adaptive multi-agent processing with dynamic delegation capabilities | Maintains quality while scaling to repository-level scope | | **Multi-Modal Synthesis** | Generates textual documentation, architecture diagrams, data flows, and sequence diagrams | Comprehensive understanding from multiple perspectives | ### Supported Languages ** Python** • **☕ Java** • ** JavaScript** • ** TypeScript** • **⚙️ C** • ** C++** • ** C#** • ** Kotlin** • ** PHP** • ** Ruby** • ** Scala** --- ## CLI Commands ### Configuration Management ```bash # Set up your API configuration codewiki config set \ --api-key \ --base-url \ --main-model \ --cluster-model \ --fallback-model # Configure max token settings codewiki config set --max-tokens 32768 --max-token-per-module 36369 --max-token-per-leaf-module 16000 # Configure max depth for hierarchical decomposition codewiki config set --max-depth 3 # Show current configuration codewiki config show # Validate your configuration codewiki config validate ``` ### Documentation Generation ``` … ``` ### Customization Options CodeWiki supports customization for language-specific projects and documentation styles: ```bash # C# project: only analyze .cs files, exclude test directories codewiki generate --include "*.cs" --exclude "Tests,Specs,*.test.cs" # Focus on specific modules with architecture-style docs codewiki generate --focus "src/core,src/api" --doc-type architecture # Add custom instructions for the AI agent codewiki generate --instructions "Focus on public APIs and include usage examples" # Analyze files ignored by Git (Git ignore filtering is enabled by default) codewiki generate --no-gitignore ``` #### Pattern Behavior (Important!) - **`--include`**: When specified, **ONLY** these patterns are used (replaces defaults completely) - Example: `--include "*.cs"` will analyze ONLY `.cs` files - If omitted, all supported file types are analyzed - Supports glob patterns: `*.py`, `src/**/*.ts`, `*.{js,jsx}` - **`--exclude`**: When specified, patterns are **MERGED** with default ignore patterns - Example: `--exclude "Tests,Specs"` will exclude these directories AND still exclude `.git`, `__pycache__`, `node_modules`, etc. - Default patterns include: `.git`, `node_modules`, `__pycache__`, `*.pyc`, `bin/`, `dist/`, and many more - Supports multiple formats: - Exact names: `Tests`, `.env`, `config.local` - Glob patterns: `*.test.js`, `*_test.py`, `*.min.*` - Directory patterns: `build/`, `dist/`, `coverage/` - **`--use-gitignore/--no-gitignore`**: Git ignore rules are applied by default - Root and nested `.gitignore` files are respected before call-graph analysis - Tracked files remain included, matching Git behavior - Built-in and explicit `--exclude` patterns still apply when Git includes a path - Use `codewiki config set --no-gitignore` to persistently disable this behavior #### Setting Persistent Defaults Save your preferred settings as defaults: ```bash # Set include patterns for C# projects codewiki config agent --include "*.cs" # Exclude test projects by default (merged with default excludes) codewiki config agent --exclude "Tests,Specs,*.test.cs" # Set focus modules codewiki config agent --focus "src/core,src/api" # Set default documentation type codewiki config agent --doc-type architecture # View current agent settings codewiki config agent # Clear all agent settings codewiki config agent --clear ``` | Option | Description | Behavior | Example | |--------|-------------|----------|---------| | `--include` | File patterns to include | **Replaces** defaults | `*.cs`, `*.py`, `src/**/*.ts` | | `--exclude` | Patterns to exclude | **Merges** with defaults | `Tests,Specs`, `*.test.js`, `build/` | | `--use-gitignore/--no-gitignore` | Apply Git ignore rules | Enabled by default | `--no-gitignore` | | `--focus` | Modules to document in detail | Standalone option | `src/core,src/api` | | `--doc-type` | Documentation style | Standalone option | `api`, `architecture`, `user-guide`, `developer` | | `--instructions` | Custom agent instructions | Standalone option | Free-form text | ### Token Settings CodeWiki allows you to configure maximum token limits for LLM calls. This is useful for: - Adapting to different model context windows - Controlling costs by limiting response sizes - Optimizing for faster response times ```bash # Set max tokens for LLM responses (default: 32768) codewiki config set --max-tokens 16384 # Set max tokens for module clustering (default: 36369) codewiki config set --max-token-per-module 40000 # Set max tokens for leaf modules (default: 16000) codewiki config set --max-token-per-leaf-module 20000 # Set max depth for hierarchical decomposition (default: 2) codewiki config set --max-depth 3 # Override at runtime for a single generation codewiki generate --max-tokens 16384 --max-token-per-module 40000 --max-depth 3 ``` | Option | Description | Default | |--------|-------------|---------| | `--max-tokens` | Maximum output tokens for LLM response | 32768 | | `--max-token-per-module` | Input tokens threshold for module clustering | 36369 | | `--max-token-per-leaf-module` | Input tokens threshold for leaf modules | 16000 | | `--max-depth` | Maximum depth for hierarchical decomposition | 2 | ### Configuration Storage - **API keys**: Securely stored in system keychain (macOS Keychain, Windows Credential Manager, Linux Secret Service). Falls back to `~/.codewiki/credentials.json` in headless/container environments. Set `CODEWIKI_NO_KEYRING=1` to force file-based storage. - **Settings & Agent Instructions**: `~/.codewiki/config.json` --- ## Documentation Output Generated documentation includes both **textual descriptions** and **visual artifacts** for comprehensive understanding. ### Textual Documentation - Repository overview with architecture guide - Module-level documentation with API references - Usage examples and implementation patterns - Cross-module interaction analysis ### Visual Artifacts - System architecture diagrams (Mermaid) - Data flow visualizations - Dependency graphs and module relationships - Sequence diagrams for complex interactions ### Output Structure ``` ./docs/ ├── overview.md # Repository overview (start here!) ├── module1.md # Module documentation ├── module2.md # Additional modules... ├── module_tree.json # Hierarchical module structure ├── first_module_tree.json # Initial clustering result ├── metadata.json # Generation metadata └── index.html # Interactive viewer (with --github-pages) ``` > **See it in action:** This repository's own docs are checked in under [`./docs/`](./docs/) — open [`./docs/index.html`](./docs/index.html) in a browser for the interactive viewer, or start from [`./docs/overview.md`](./docs/overview.md). --- ## Experimental Results CodeWiki has been evaluated on **CodeWikiBench**, the first benchmark specifically designed for repository-level documentation quality assessment. ### Performance by Language Category | Language Category | CodeWiki (Sonnet-4) | DeepWiki | Improvement | |-------------------|---------------------|----------|-------------| | High-Level (Python, JS, TS) | **79.14%** | 68.67% | **+10.47%** | | Managed (C#, Java) | **68.84%** | 64.80% | **+4.04%** | | Systems (C, C++) | 53.24% | 56.39% | -3.15% | | **Overall Average** | **68.79%** | **64.06%** | **+4.73%** | ### Results on Representative Repositories | Repository | Language | LOC | CodeWiki-Sonnet-4 | DeepWiki | Improvement | |------------|----------|-----|-------------------|----------|-------------| | All-Hands-AI--OpenHands | Python | 229K | **82.45%** | 73.04% | **+9.41%** | | puppeteer--puppeteer | TypeScript | 136K | **83.00%** | 64.46% | **+18