Unified multi-source build: PDF content silently dropped from final output, SKILL.md becomes AI-hallucinated filler with no grounded content
Bug
skill-seekers create <config.json> with a unified multi-source config (39 sources: 19 PDF + 20 EPUB) completes with exit code 0 and no errors, but the final output/<name>/references/pdf/ directory is empty except for an index.md that claims "Reference from 26 PDF document(s)" — zero actual PDF content files are written. EPUB sources partially survive but only as raw _data.json, not the readable .md reference files that single-source builds produce.
The final SKILL.md (level-1 AI enhancement) ends up as generic, non-grounded boilerplate with zero citations to any source book, because the enhancement step has almost no real reference content to read.
Per-source extraction itself works correctly — .skillseeker-cache/<name>/sources/<name>_pdf_N_<title>/references/<title>.md contains full, correct extracted text for every PDF (confirmed: 31k+ lines for one ACSM title). The data loss happens specifically in the unified-build assembly step.
Root cause (found in unified_skill_builder.py)
_load_source_skill_mds() (around line 147) only reads each PDF source's SKILL.md into a combined skill_mds["pdf"] string for text-synthesis into the final SKILL.md body:
pdf_sources = []
for pdf_dir in sources_dir.glob(f"{self.name}_pdf_*"):
pdf_skill_path = pdf_dir / "SKILL.md"
...
pdf_sources.append(content)
...
skill_mds["pdf"] = "\n\n---\n\n".join(pdf_sources)There is no corresponding step that copies each PDF source's references/*.md files into the unified output's references/pdf/ directory — only the (short, summary-level) per-source SKILL.md text gets folded in, and even that gets trimmed down to just "Chapter Overview" / "Key Concepts" sections by _synthesize_docs_pdf / _synthesize_github_pdf / _synthesize_docs_github_pdf. The actual reference material never makes it to disk.
Repro
- Build a unified config with several PDF
type: "pdf"sources, e.g.:
{
"name": "test",
"merge_mode": "rule-based",
"sources": [
{"type": "pdf", "path": "/path/to/book1.pdf"},
{"type": "pdf", "path": "/path/to/book2.pdf"}
]
}skill-seekers create test_config.json --enhance-level 1 --output output/test- Inspect
output/test/references/pdf/— onlyindex.mdexists, no per-book content.mdfiles, despite.skillseeker-cache/test/sources/test_pdf_*/references/*.mdcontaining full extracted text.
Environment
- skill-seekers 3.9.1 (pip)
- Windows 11, Python 3.12.10
- Repro used 19 PDF + 20 EPUB sources; PDF loss was total, EPUB loss was partial (only some sources'
_data.jsonsurvived, no readable.md)
Expected
Unified build should copy each source's full references/*.md content into the final output/<name>/references/<type>/ tree (as single-source builds already do), not just fold a summarized SKILL.md excerpt into the merged doc.
Workaround used
Reused the .skillseeker-cache/<name>/sources/*/references/*.md files directly (extraction succeeds and is cached even though final assembly drops it) and hand-assembled a master references/ tree + SKILL.md.
Source: yusufkaraaslan/Skill_Seekers