Engine: report data size and reuse on ingest and export spans

Context

The engine moves data for the user in two directions:

1. Ingest: image pulls, git fetches, HTTP fetches, client uploads (filesync)
2. Export: registry pushes, client exports

The engine skips data it already has: known image layers, unchanged files, layers a registry already holds.

Dagger Cloud meters CPU and memory. It must also meter data movement, and it must see how much data the engine did not move because it already had it.

What the engine reports today

Progress log records, scope dagger.io/progress, one stream per span and item. Attributes: dagger.io/progress.item, .current, .total, .unit. The latest current per span and item is the amount moved.

Also available on all versions:

• Metric dagger.io/metrics.filesync.written_bytes: file bytes written into the engine by filesync.
• Span attribute http.request.header.content-length on HTTP PUT spans: bytes uploaded to registries.

For registry pushes, the progress records and the HTTP PUT content lengths give the same total.

Problem

The records report the data that moved. They do not report the data the user asked for. Metering needs both.

1. Reuse is invisible. A layer the engine already has opens no content writer, so it emits no record. A layer the registry already has returns "already exists", so the push emits no record. A pull that reuses every layer looks the same as no pull.
2. Filesync, client export, and HTTP records have no content digest. Cloud cannot tell whether the same content moved before.
3. Git reports objects, not bytes.
4. There is no attribute for the operation or the direction. Cloud must parse the span name.
5. A total requires the latest record of every stream. Busy orgs produce millions of progress records per day. These records exist to drive progress bars, not to meter.
6. v0.21.9 reports registry pulls and pushes only. v0.21.8 and earlier report nothing.

Filesync also has a specific ambiguity. A large upload has two causes that look the same: many files changed, or the client connected to an engine that had no earlier snapshot.

Solution

When each ingest and export operation completes, also on failure, set these attributes on its span:

dagger.io/data.direction          "ingest" or "export"
dagger.io/data.logical_bytes      size of the content the user asked for
dagger.io/data.transferred_bytes  bytes that crossed the engine boundary
dagger.io/data.content_digest     digest of the content
dagger.io/data.base_digest        filesync only: digest of the snapshot the diff was computed against, empty if none

logical_bytes minus transferred_bytes is the reused part. Use compressed sizes where the content is compressed. transferred_bytes equals the sum of the progress records for the span. Keep the progress records for the UI.

base_digest separates the two causes of a large filesync upload:

The second and fourth rows are uploads the scheduler could have avoided by routing the client to the engine that had the snapshot.

Out of scope: traffic that containers start themselves, and any change to the cache or sync protocols. This is reporting only.