Feature: Fact extraction pipeline (L0 conversation → L1 distilled facts)

title: "Feature: Fact extraction pipeline (L0 conversation → L1 distilled facts)" labels: ["enhancement", "memory", "pipeline"]

Problem

memU stores raw conversation chunks as retrieval segments. When the agent later retrieves context, it gets back conversational fragments — complete with filler words, asides, and formatting artifacts — rather than clean, declarative facts.

Real example from production use:

A segment retrieved for "homelab SSH setup" returns:

sshpass -p 'x10u8ant4r' ssh [email protected]    # VM 100 (VoyagerBuntu)

This works for exact-match lookups, but for reasoning tasks the agent needs context like "the user prefers SSH in order root → skylab → admin" — a synthesized fact that doesn't appear verbatim in any single chat message.

The core tension: Chat transcripts are excellent sources of truth but poor retrieval targets. They're verbose, context-dependent, and spread related facts across multiple messages. The agent already pays a token cost to receive segments; those tokens should deliver maximum information density.

Proposed solution

Add an optional fact extraction step in the bridging pipeline, between prepare and commit:

Current:  prepare → [agent decides on skills] → commit
Proposed: prepare → [agent decides on skills + extracts facts] → commit

The prepare step already hands the agent its own session transcript and asks it to evaluate skills. This proposal extends that prompt to also extract declarative facts from the conversation:

Output format (new recall_file track: "memory", with a kind: "fact" discriminator):

---
name: homelab-infrastructure
description: Homelab infrastructure reference
kind: fact
---
## SSH Access
- SSH user order: root → skylab → admin (established 2026-08-06)
- Password scheme: x10u8ant4r (services), x10u8ant4rLy (PVE/PBS)
- Fish shell on .45/.67/.18/.38 — use Python one-liners or bash -c

## Firewall
- pfSense at 192.160.1.1 — NEVER modify while user is remote
- Pi-hole at 192.160.1.100 (raspberrypi) — Docker container 'pihole'

These fact files would:

1. Be stored alongside existing conversation-derived segments
2. Be embedded and retrieved through the same pipeline
3. Have higher information density (fewer tokens for the same knowledge)
4. Be idempotent — the agent would merge new facts with existing fact files rather than creating duplicates

Key design decisions

Why agent-driven, not a separate LLM call: memU's philosophy (per the README) is "the judgment and synthesis stay inside the agent. MemoryService makes no LLM or chat calls." This proposal respects that — the agent itself does the extraction during the existing prepare step. No new API keys, no new models, no new services.

Why a separate fact file, not replacement of raw segments: Raw conversation segments remain valuable for:

• Verbatim recall ("what exactly did the user say?")
• Timestamped sourcing ("when was this decided?")
• Fallback when fact extraction misses something

Fact files complement raw segments; they don't replace them.

Why kind: fact on the existing memory track: The memory track already supports this pattern — it stores both raw transcripts and manually authored reference docs (like homelab-infrastructure.md). A kind discriminator lets retrieval treat facts and raw chats differently (e.g., facts get a retrieval score boost, or are returned in a separate facts layer alongside segments/files/resources).

Implementation notes

• Scope: Extend the prepare prompt template (in memu/hosts/bridging/prompt.py) to include fact extraction instructions
• Storage: Fact files are recall_files on the memory track with kind: fact in metadata
• Retrieval: Facts are embedded and indexed like any other segment — no retrieval code changes needed in v1
• Backward compatibility: Fully optional. Existing users who don't want this see no change.
• Config surface: MEMU_FACT_EXTRACTION=enabled (default: disabled)

Success criteria

• Fact files are 3-5× more token-efficient than raw chat segments for the same knowledge
• Related facts are consolidated (no duplication of SSH preferences across 3 segments)
• Facts are updated/merged rather than duplicated when new information arrives
• Retrieval quality improves measurably: queries return synthesized facts, not just conversational matches