velero-plugin-for-aws: multipart uploads force a CRC32 trailer regardless of checksumAlgorithm="" - every backup >5 MiB fails on S3-compatible backends (EOF)
Multipart uploads force a CRC32 trailer regardless of checksumAlgorithm="" / RequestChecksumCalculation — every backup >5 MiB fails on S3-compatible backends (EOF)
What steps did you take and what happened
velero v1.18.2, velero-plugin-for-aws v1.14.2, BackupStorageLocation against OCI Object Storage's S3 Compatibility API:
config:
region: us-ashburn-1
s3Url: https://<ns>.compat.objectstorage.us-ashburn-1.oraclecloud.com
s3ForcePathStyle: "true"
checksumAlgorithm: ""Server env: AWS_REQUEST_CHECKSUM_CALCULATION=when_required, AWS_RESPONSE_CHECKSUM_VALIDATION=when_required.
Every scheduled backup (all namespaces, ~2500-3400 items) ends Failed, failureReason: rpc error: code = Unknown desc = EOF, right after "Initial backup processing complete". In the bucket only backups/<name>/<name>-logs.gz remains: velero-backup.json was uploaded and then deleted by PutBackup's error path because the CONTENTS upload failed. A one-namespace backup (~250 KiB tarball) completes on the same BSL.
What did you expect to happen
With checksumAlgorithm: "" (and/or the SDK's when_required opt-out) no checksum is sent on any upload, single-part or multipart, and the backup completes.
Root cause
object_store.go builds the uploader as manager.NewUploader(client). The transfer manager (feature/s3/manager v1.22.18) has its own Uploader.RequestChecksumCalculation, defaulting to WhenSupported, and it never consults the client's option or the env var:
// feature/s3/manager/upload.go
RequestChecksumCalculation: aws.RequestChecksumCalculationWhenSupported,
...
default:
if u.cfg.RequestChecksumCalculation != aws.RequestChecksumCalculationWhenRequired {
u.in.ChecksumAlgorithm = types.ChecksumAlgorithmCrc32
}So objects above the 5 MiB part size go multipart with a forced CRC32 trailer (aws-chunked). OCI answers UploadPart with 501 NotImplemented: AWS chunked encoding not supported; the plugin returns the error and velero's streaming client sees the closed stream as a bare EOF. #313 (release-1.14, v1.14.3-rc) sets RequestChecksumCalculation on the S3 client only, so it fixes single-part PutObject and leaves multipart broken.
Reproduction with this repo's pinned SDK versions (aws-sdk-go-v2 v1.41.12, feature/s3/manager v1.22.18, service/s3 v1.101.0), AWS_REQUEST_CHECKSUM_CALCULATION=when_required exported, non-seekable body like the gRPC stream:
client RequestChecksumCalculation=WhenRequired (from env)
manager.NewUploader(client) 3 MiB -> OK
manager.NewUploader(client) 6 MiB -> upload multipart failed ... UploadPart, https response error StatusCode: 501 ... api error NotImplemented: AWS chunked encoding not supported.
NewUploader(client, u.RequestChecksumCalculation=WhenRequired) 6 MiB -> OKaws-cli confirms the backend behavior: s3api upload-part --checksum-algorithm CRC32 -> 501; the same part without a checksum -> 200.
Fix
Make the uploader inherit the client's mode (PR attached):
o.s3Uploader = manager.NewUploader(client, func(u *manager.Uploader) {
u.RequestChecksumCalculation = client.Options().RequestChecksumCalculation
})Environment
- Velero version: v1.18.2 (server image docker.io/velero/velero:v1.18.2)
- velero-plugin-for-aws: v1.14.2 (also present on main and release-1.14 as of 2026-09-17)
- Kubernetes: v1.36.1 (OKE)
- Object storage: OCI Object Storage, Amazon S3 Compatibility API, path-style, HTTPS
Related: #9951 (single-part half, fixed by velero-plugin-for-aws#313).
Source: velero-io/velero