Adding to snapshot additional files (PostgreSQL backup)

I am investigating how to combine PostgreSQL backup with kopia.

In investigated different options, including its high-level pg_basebackup tool and low-level backup API.

The issue with directly copying database files is that files might not have consistency (some state might be in wal). The issue with using pg_basebackup is that you are effectively making a whole copy (or an increment to it) of the database somewhere to your machine. So double the storage just so that kopia can then do another copy/backup somewhere else. This double copying also means you are doing additional SSD wear on the machine.

I know about kopia --stdin-file but that might work with pg_dump and even that just in single-file mode (-Fc) and not multi-file mode (-Fd) which allows partial restoration. Also, it means that I have two snapshots, one for files, another for pg_dump output.

pg_basebackup supports output to a tar stdout, but kopia is lacking something like --stdin-tar. And even with that the best way to run pg_basebackup is with --wal-method=stream but which outputs two tar files and is not compatible with transfer over stdin.

So I looked at the low-level API pg_basebackup uses to see if I could do something more integrated with kopia. And what I came up with is:

SELECT pg_backup_start('kopia');
-- kopia snapshots /pgsql/data
SELECT * FROM pg_backup_stop();   -- returns lsn, labelfile, spcmapfile

Then, the second of these fields should be written to a file named backup_label in the root directory of the backup. And additionally, wal between start and stop should also be written to the backup (this is what --wal-method=stream does in pg_basebackup.

The ordering constraint is external and unavoidable. backup_label does not exist until pg_backup_stop returns, which is necessarily after the copy. The payload is tiny and known in advance by shape. backup_label, optionally tablespace_map, plus a bounded WAL range.

So, the issue is that it looks like I would need from kopia some mode which would start a snapshot, backup over all regular files (including /pgsql/data), end, then I would call pg_backup_stop. Make few more files (labelfile + wal), and call kopia to backup them, too, and finish the snapshot. Is this possible?

That would then be amazing: postgresql data would be stored on the server only once, kopia would deduplicate postgresql data chunks, kopia would also normally copy over postgresql data like any other file, while wal would be available so that final data is consistent. One would not need to use PostgreSQL --incremental I think because kopia would do deduplication. Which would simplify backup organization.