[feature] private infinite memory with on-demand AI access

Vision

Give every person effectively infinite external memory: a private, durable record of what they have seen, said, heard, and created over a lifetime, searchable from any device and usable by the AI they choose.

The archive should become more useful over decades without requiring the user to surrender ownership of the data or its master keys to Screenpipe, a storage provider, or a model provider.

A separate encrypted, device-owned memory system demonstrates the useful architectural split: untrusted infrastructure can retain ciphertext indefinitely, while an explicitly trusted node holds keys, interprets permissions, searches the archive, and releases only authorized results.

This issue defines the constraints before choosing an implementation. Possible architectures and tradeoffs are listed in the first comment.

cc @EzraEllette

Product and user constraints

• The primary customer is nontechnical. Setup cannot require understanding DNS, certificates, VPNs, object stores, enclaves, keys, or server maintenance.
• Capture must continue locally without internet access. Connectivity affects synchronization and remote queries, not recording.
• A remote query requires only one authorized query node to be online; every capture device must not need to be online simultaneously.
• ChatGPT on iOS and other hosted AI clients must be able to query from outside the home network through a standards-compatible public HTTPS interface.
• A sleeping laptop cannot provide an "always available" experience. The product must clearly distinguish private-device availability from always-on availability.
• The Screenpipe app should provide the richest private experience. Third-party AI clients should receive narrow capabilities, not the full archive.
• Enrollment, recovery, revocation, storage health, and availability must be understandable from ordinary product UI.
• Defaults must be safe. Advanced users may choose their own server, network, storage provider, or keys.
• The experience must work across macOS, Windows, Linux, iOS, and Android within each platform's background-execution and secure-storage limits.
• Storage, bandwidth, battery use, indexing progress, and degraded/offline states must be visible without exposing sensitive content.

Privacy and trust constraints

• If we claim that Screenpipe cannot decrypt user data, Screenpipe-controlled infrastructure must never receive plaintext, master keys, recovery material, or an unrestricted decryption capability.
• Storage and relay services may see unavoidable metadata such as account, IP address, object sizes, timing, and access patterns. Product claims must state this boundary.
• Model providers see the plaintext excerpts returned to them. They must not receive the master key, full vault, unrestricted index, or unrelated records.
• Search must release the minimum matching excerpts needed for the answer, with user-visible provenance back to the original record.
• Every TLS termination point is part of the trusted computing base. Ordinary hosted clients validate WebPKI certificates, not hardware-attestation documents.
• If Screenpipe controls the public hostname, DNS, certificate issuance, or TLS proxy, it may be technically capable of substituting a valid endpoint and observing plaintext. The privacy claim must account for that.
• Hardware enclaves reduce host access but do not by themselves solve TLS ownership, code authorization, key-release governance, rollback, recovery, or malicious-successor deployments.
• Publishing code or waiting through a delay is not user consent to a successor release. A hostile but publicly visible release must not automatically inherit every user's keys.
• Current KMS policy is not proof that historical grants never existed. Security evidence must cover key creation, every release, historical access, and revocation.
• The key hierarchy must be explicit: owner authority, device identity, data-encryption keys, index keys, grant-signing authority, recovery material, and storage credentials are separate roles.
• A storage credential must never imply device identity, decryption, grant issuance, recovery, or plaintext access.
• Native clients should bind credentials to hardware-backed, non-exportable keys where available. Hosted clients must be treated as bearer clients.
• Compromise of one device, connector, query node, tenant, or key epoch must have a bounded blast radius.
• Logs, telemetry, crash reports, notifications, and support tooling must not contain captured content, tokens, keys, raw queries, or returned excerpts by default.
• Claims must be precise and testable: "encrypted at rest" is different from "Screenpipe cannot decrypt," and both are different from "the model provider never sees plaintext."

Authorization and connector constraints

• Hosted AI access must use narrow, explicit, expiring, revocable grants.
• Grants must bind subject, audience, tools, data scope, time range, result limits, and rate limits.
• OAuth must validate issuer, audience/resource, expiry, scope, redirect URI, and PKCE correctly.
• A read/search grant must not imply write, send, delete, secret use, or owner authority.
• Just-in-time approval should support one exact query without silently widening the standing grant.
• Revocation must fail closed immediately at the serving node and across pagination, retries, and cached sessions.
• Every permitted read needs a content-free audit receipt that the user can inspect.
• Replay, copied bearer tokens, changed query arguments, stale approvals, and cross-tenant identifiers must fail closed.
• Public endpoints require rate limits, abuse controls, resource budgets, and denial-of-service isolation that do not depend on trusted forwarding headers.

Availability and networking constraints

• Connectivity, storage, query execution, and key custody are separate concerns.
• An embedded mesh/VPN or reverse tunnel can make a node reachable, but it does not provide encrypted storage, indexing, permissions, recovery, or compute.
• Private-device mode works only while an authorized device and its query service are online.
• Always-available mode requires an always-on user-owned node or an explicitly trusted managed executor.
• Relay failure must not corrupt history or prevent local capture.
• Query-node failure should have explicit failover semantics without copying unrestricted keys to every server.
• Network resumptions, duplicate delivery, partial uploads, clock skew, and intermittent mobile connectivity must be expected.
• Public endpoint discovery and certificate rotation must not create an invisible new decryption party.

Twenty-four-month and lifetime storage constraints

• Twenty-four months of raw capture can be hundreds of gigabytes or multiple terabytes per user. The design must extend to decades without requiring all raw media on every device.
• User devices need a configurable byte budget, not only a time-based retention setting.
• Eviction is cache management, never deletion of signed history or the only recoverable ciphertext.
• Raw video/audio, searchable text/metadata, derived summaries, and indexes have different retention, privacy, and recomputation costs.
• Large encrypted media should be eligible for keyless object storage while devices keep metadata, indexes, recent hot data, and explicitly pinned records.
• Object storage is untrusted: it can delete, withhold, reorder, corrupt, roll back, or suspend access.
• Deduplication, compression, chunking, and packing must not leak plaintext equality across users or break authenticated retrieval.
• Upload must be resumable and content-addressed. Downloaded bytes remain untrusted until signatures, sizes, hashes, and authorization are verified.
• Users need export, deletion, account closure, legal-retention, and provider-migration paths that do not depend on Screenpipe retaining their master key.
• At least one independent recovery source and a tested restore path are required before local eviction is described as safe.

Search and indexing constraints

• A lifetime archive cannot be served by scanning every record or loading the entire decrypted history into a bounded in-memory snapshot.
• Indexing must be incremental, resumable, encrypted at rest, and partitionable by user, space, time, and key epoch.
• The system needs exact text/OCR/transcript search, semantic retrieval, temporal filtering, and direct links to the original evidence.
• Broad questions may use hierarchical summaries, but answers must remain traceable to original records rather than summary-only assertions.
• Query planning should search compact indexes first and retrieve/decrypt only selected raw chunks.
• Index format changes, model changes, and embedding migrations must not require re-uploading plaintext or losing history.
• Derived indexes and summaries must inherit source permissions and revocation.
• Deleted, superseded, or newly restricted records must stop appearing in search results and cached model context.
• Search quality, latency, and cost must be measured against realistic 24-month datasets, including cold starts and low-connectivity devices.

Integrity, recovery, and lifecycle constraints

• History needs authenticated authorship and tamper-evident ordering, not only encrypted objects.
• Rollback and equivocation must be detectable across devices or through an independent witness.
• Recovery must restore keys, signed heads, metadata, indexes or their rebuild inputs, and every referenced file—not merely list bucket objects.
• Recovery drills must use a fresh destination and must never overwrite a working archive.
• Key rotation, lost-device removal, compromised-device response, ownership transfer, and inheritance need explicit behavior.
• Removing a user's access must not destroy shared data owned by another user, and shared access must not create universal decryption keys.
• Every migration and cleanup operation needs a recoverable, previewable boundary.

Scale and operations constraints

• Tenants must be cryptographically and operationally isolated.
• Per-user CPU, memory, storage, bandwidth, query, and model costs need quotas and backpressure.
• Background ingestion and indexing must not starve interactive queries.
• Operational health should be observable using content-free metrics.
• Support staff need a diagnosis path that does not require asking for the user's archive or keys.
• Updates must support staged rollout, rollback detection, compatibility windows, and explicit key-release policy.
• The design should permit self-hosted, user-owned, and managed deployments without fragmenting the data model.
• Unit economics must cover multi-terabyte retention, egress, indexing, backups, and occasional re-indexing over many years.

Minimum acceptance evidence

• An allowed canary record is returned through the real hosted-client path.
• A withheld canary and records outside the grant scope are absent.
• Revocation makes the next request fail, including paginated and cached requests.
• The audit receipt identifies the requester, grant, operation, scope, and outcome without captured plaintext.
• No Screenpipe service, relay, storage provider, or model provider receives a master key or unrestricted archive.
• Offline devices continue capturing and synchronize safely later.
• Device-offline behavior matches the selected availability mode and is clear to the user.
• A constrained device can evict local ciphertext and later recover the exact file.
• A fresh recovery drill restores and verifies the archive independently.
• A realistic 24-month benchmark demonstrates bounded local storage, incremental indexing, acceptable query latency, and measurable operating cost.
• The stated privacy claim is independently verifiable against DNS, TLS, deployment, key release, runtime, storage, and recovery behavior.