[Feature Request]: ZIM/Kiwix archives as a knowledge source (Book, Mastery Path, Immersive Reading, Knowledge Center)

Feature Request Description

Add native ZIM/Kiwix support as a first-class knowledge source, so a user's offline ZIM archives (Wikipedia, Wikisource, Project Gutenberg, docs.python.org, Khan Academy, etc. served via kiwix-serve) can feed the same learning surfaces a regular Knowledge Base does today — not just live web search, but as source material for Book generation, Mastery Path, Immersive Reading/Learning, and the Knowledge Center.

I have a working prototype (local, not yet a PR) that adds a kiwix provider to the web_search tool's provider layer (deeptutor/services/search/providers/, alongside searxng), following the exact pattern DeepTutor already uses for self-hosted, base-url-only providers:

• Auto-discovers every loaded book per instance from Kiwix's /catalog/search (OPDS 1.2) endpoint — /catalog/v2/entries has been observed to silently omit loaded books.
• Per-book search + a short-lived failure cache, since a book's _ftindex catalog tag has been observed unreliable in both directions (tagged searchable but errors; tagged not searchable but returns real hits).
• Supports multiple comma-separated Kiwix instances in one base_url field.

That gets Kiwix content into ad-hoc chat via web_search, but it stops there — it's a live-search tool, not a knowledge source. It doesn't help:

• Book: generating an interactive Book grounded in a ZIM archive (e.g. a full offline encyclopedia or textbook mirror) the way it can from uploaded documents.
• Mastery Path / Guided Learning: building a learning path over ZIM content.
• Immersive Reading/Learning: reading a ZIM article with the same structured-article experience as other ingested sources.
• Knowledge Center: deeptutor kb create only takes files/directories today; a multi-GB ZIM archive can't be pointed at directly, and dumping every article to disk first (e.g. via zimdump) to feed the KB pipeline is a heavyweight, redundant, one-time-stale copy of data that already exists on disk in a well-indexed format.

Related Module

Knowledge Base Management (also affects Book, Guided Learning)

Use Case

Self-hosters running kiwix-serve over a directory of ZIM files already have a large, curated, offline knowledge corpus (encyclopedias, reference sites, docs mirrors, books) that today can only be reached from DeepTutor via a bolted-on web-search-style tool. Being able to register a ZIM archive (or a whole Kiwix instance) as a Knowledge Base source — ideally reading it directly (e.g. via libzim's Python bindings) rather than requiring a separate export/ingestion step — would let that same content ground Book generation, Mastery Path, Immersive Reading and Knowledge Center search, the same way uploaded PDFs/docs do now, without duplicating gigabytes of data into a second on-disk copy just to make it visible to the RAG pipeline.

Additional Context

Happy to share the prototype kiwix.py provider (web_search-tool-level) as a starting point/reference for the HTML/catalog parsing quirks it had to work around, if useful groundwork for a deeper KB-level integration. The core open question for a KB-level integration is probably: read ZIM content directly via libzim at query/ingest time (live, no duplication) vs. a one-time zimdump-style extraction into DeepTutor's existing document-ingestion pipeline (simpler, but static and duplicative).