[BUG REPORT] S3: filer metadata can reference a tombstoned chunk — masked by readDeleted=true, becomes silent data loss after vacuum

### Describe the bug

An S3 object's filer entry can point at a chunk that has been deleted
(tombstoned) in the volume index. Because the filer read path unconditionally
appends `readDeleted=true`, the object stays fully readable (HEAD/GET 200 with
the original size and Last-Modified), so the inconsistency is invisible to
clients. Once the volume is vacuumed, the tombstoned needle is physically
reclaimed and the object suddenly returns 404 — silent, delayed data loss.

Observed on one object, shortly after an S3 `PUT` returned 400 while the object
was nevertheless created (the filer entry's mtime matches the failing PUT).
Both objects referenced below are still readable at the time of writing.

### Environment

- SeaweedFS 4.47, volume server built with `-tags=5BytesOffset` (large-disk
  build; `.idx` records are 17 bytes: needleId(8) + offset(5) + size(4))
- 3-replica volume (`ReplicaPlacement:020`), S3 API
- Writer: rclone v1.69.1

### Evidence

#### 1. `volume.fsck -findMissingChunksInFiler` reports the chunks as missing on all 3 replicas (reproducible)

```
$ volume.fsck -collection mybucket-20260915 -volumeId 166145293 -findMissingChunksInFiler
total 25 directories, 14707013 files
/buckets/mybucket-20260915/18/object-a.eml
/buckets/mybucket-20260915/18/object-b.eml
/buckets/mybucket-20260915/18/object-a.eml
/buckets/mybucket-20260915/18/object-b.eml
/buckets/mybucket-20260915/18/object-a.eml
/buckets/mybucket-20260915/18/object-b.eml
```

Each path is printed once per replica (the check is per-replica and does not
deduplicate), i.e. the chunk is missing on all three.

#### 2. Reading through the filer succeeds; reading the same fid directly from every replica 404s

```
# via the filer / S3 path -> 200 with the full object
$ curl -I "http://10.0.0.20:15672/buckets/mybucket-20260915/18/object-a.eml"
HTTP/1.1 200 OK
Content-Length: 109036
Last-Modified: Tue, 15 Sep 2026 10:38:03 GMT
Server: SeaweedFS 8000GB 4.47

# the fid from fs.meta.cat, read directly from each replica -> 404
$ curl -I "http://10.0.0.11:8013/166145293,0643b039c91ef08cc9"
HTTP/1.1 404 Not Found
Server: SeaweedFS Volume 8000GB 4.47
$ curl -I "http://10.0.0.12:8023/166145293,0643b039c91ef08cc9"
HTTP/1.1 404 Not Found
Server: SeaweedFS Volume 8000GB 4.47
$ curl -I "http://10.0.0.13:8027/166145293,0643b039c91ef08cc9"
HTTP/1.1 404 Not Found
Server: SeaweedFS Volume 8000GB 4.47
```

#### 3. The `.idx` shows the needle written and then tombstoned (later record wins)

```
$ stat -c '%s' mybucket-20260915_166145293.idx
7551536          # 7551536 / 17 = 444208 -> 5BytesOffset build

$ python3 idx_dump.py mybucket-20260915_166145293.idx 643b039c9
275037  643b039c9  off=16974881328  size=109048
275039  643b039c9  off=16974996632  size=-1  DEL
```

The later record is `TombstoneFileSize` (-1). `doLoading`
(`weed/storage/needle_map_memory.go:37-58`) applies records in file order, so the
needle ends up absent from the in-memory map and every normal read returns 404.
This matches the fsck output — the fsck report is correct, not a false positive.

#### 4. `readDeleted=true` shows the data is physically still present

```
$ curl -I "http://10.0.0.13:8027/166145293,0643b039c91ef08cc9?readDeleted=true"
HTTP/1.1 200 OK
Content-Length: 109036
Last-Modified: Tue, 15 Sep 2026 10:38:03 GMT
Server: SeaweedFS Volume 8000GB 4.47
```

#### 5. S3 gateway access log around the write

```
10.0.0.60 - - [15/Sep/2026:18:37:59 +0800] http s3.example.internal:8333 404 "HEAD /mybucket-20260915/18/object-a.eml HTTP/1.1" 647 0.002 238 0 - upstream=127.0.0.1:3333 "-" - "rclone/v1.69.1"
10.0.0.60 - - [15/Sep/2026:18:38:03 +0800] http s3.example.internal:8333 400 "PUT /mybucket-20260915/18/object-a.eml?x-id=PutObject HTTP/1.1" 109848 0.049 561 321 - upstream=127.0.0.1:3333 "-" - "rclone/v1.69.1"
10.0.0.60 - - [15/Sep/2026:18:38:07 +0800] http s3.example.internal:8333 200 "HEAD /mybucket-20260915/18/object-a.eml HTTP/1.1" 647 0.002 453 0 - upstream=127.0.0.1:3333 "-" - "rclone/v1.69.1"
```

The filer entry's mtime is `1789468683` = `2026-09-15 18:38:03 +0800` — the same
second as the PUT that returned 400. The PUT's request length (109848 bytes,
including headers) is the whole object (~109036 bytes), i.e. the server received
the complete object and still answered 400.

### Root cause analysis

1. **Filer reads always append `readDeleted=true`.**
   `weed/filer/stream.go:527` (`ChunkStreamReader.fetchChunkToBuffer`) and
   `weed/filer/filechunk_manifest.go:421` build every chunk URL with
   `AppendQueryParameter(urlString, "readDeleted", "true")`; the volume server
   honors it (`weed/server/volume_server_handlers_read.go:180`,
   `weed/storage/volume_read.go:36,133`). As long as the volume has not been
   vacuumed, a metadata entry that points at a tombstoned needle is served
   normally, so the inconsistency has no client-visible symptom.

2. **The write path is not atomic and its rollback is best-effort.**
   `putToFiler` uploads chunks, verifies checksums, then creates the filer entry
   (`weed/s3api/s3api_object_handlers_put.go`). Chunks are deleted only when
   `entryCreated == false` (:993-1003). The post-create rollback calls `rmObject`
   and only logs a failure (:940-948). If that rollback only partially succeeds
   (data deleted, metadata left behind), the dangling reference persists with no
   retry or compensation.

3. **Nothing reconciles the two views.** There is no background check between
   filer metadata and volume needle maps; `volume.fsck -findMissingChunksInFiler`
   detects the state but is manual, and `volume.check.disk` only compares
   replicas against each other.

`readDeleted=true` is presumably deliberate, to tolerate the in-flight window
between tombstoning a needle and removing the metadata entry. The problem is
that the resulting state is unbounded and unobservable.

### Impact

Silent, delayed data loss. The object remains listable and readable until
`volume.vacuum` (or any compaction) reclaims the tombstoned needle; after that
every read returns 404, with no warning beforehand. A client that received a 400
for its PUT can still observe a "successful" object afterwards.

### Suggested fixes

- Make the write/rollback path transactional, or add an explicit compensation
  step with retries; do not swallow rollback errors.
- Make the `readDeleted` fallback observable: log/emit a metric when a chunk is
  served from a deleted needle, since that indicates filer metadata and the
  volume index disagree. Optionally bound how long it is tolerated.
- Add a background consistency check between filer metadata and volume
  tombstones with alerting, or surface it in `volume.fsck` as a distinct,
  actionable category.

### How to inspect `.idx` locally (5BytesOffset builds)

```python
import sys
d = open(sys.argv[1], 'rb').read()
N = 17  # needleId(8) + offset(5) + size(4); plain builds use 16 with offset at bytes[8:12]
assert len(d) % N == 0, f"size={len(d)} not a multiple of {N}"
for i in range(0, len(d), N):
    r = d[i:i+N]
    nid = int.from_bytes(r[0:8], 'big')
    off = (r[11] | r[10]<<8 | r[9]<<16 | r[8]<<24 | r[12]<<32) * 8
    size = int.from_bytes(r[13:17], 'big', signed=True)
    print(f"{i//N}\t{nid:x}\toff={off}\tsize={size}\t{'DEL' if size < 0 else ''}")
```