Watching Relationship Changes More Ergonomically

Problem Statement

The Watch API cannot be used out of the box as a replyable CDC stream for watching for relationship tuple changes today.

As a developer, I want to be able to start watching for changes at the beginning of "time" and periodically store the cursor to mark the last seen update. I should be able to do so regardless of the number of tuples in the underlying database at any point. The snapshot cursor today doesn't provide this ability, and the default behavior of the Watch API starts watching from the HEAD revision which leaves gaps of updates that may have not been seen. This is made worse by GC'ed revisions that were stored in a cursor in a previous watch that are no longer valid snapshots.

Since the relationship table implements an append-only log of inserts/deletes with a revision (xid) marker, we'd like to be able to start watching for changes as of the earliest snapshot revision. This will allow us to consume the "current state" of the relationship table AS OF any particular snapshot and reconcile the effective set of tuples in our downstream service.

Solution Brainstorm

I propose at least one of the following two changes to SpiceDB:

1. Modify the Watch API such that, given a zedtoken/cursor that references an old snapshot which has since been deleted, then the Watch will pick up with the min revision at that point in time. We can add a new option to the Watch API whereby if no cursor is provided and the option is set to start from min_revision then the Watch starts at the min revision. The default option will behave in a way that starts at the head revision (which will be backward compatible with what is there today).

2. Add, at minimum, a new API called MinSnapshotRevision which returns a ZedToken that encodes the min snapshot revision currently in the underlying relationship database. We can use the ZedToken returned from this to start our Watch using the existing Watch API. If at any point we get an error from the Watch API indicating that the cursor we have provided no longer exists (because it was GCed), then we can call this to restart the watch at that snapshot.

Of these two options I would lean towards the ergonomics of option 1, because that removes the burden from the caller. Although option 1 would suffice for the need, I believe SpiceDB should expose APIs like that described in option 2 in one of the services to get the min and head revision zedtoken of the SpiceDB instance at any point in time AND implement something like option 1 above.