Add content-idempotent path uploads for S3

Summary

Add an opt-in upload_file_idempotent() method for S3 path uploads. The method compares the local file's full-object SHA-256 with the current object's checksum, skips identical bytes, and uses conditional writes when creation or replacement is needed. Existing upload_file() behavior would remain unchanged.

Proposed API

Expose upload_file_idempotent() on:

• S3 Client
• Bucket
• Object
• boto3.s3.transfer.S3Transfer

Return False when identical content is skipped and True after a conditional upload succeeds. This proposal is intentionally path-only; it does not add a file-object variant.

Proposed behavior

• Hash the complete local file as Base64-encoded SHA-256.
• Call HeadObject(ChecksumMode='ENABLED').
• If the key is missing, upload with IfNoneMatch='*'.
• If the full-object SHA-256 matches, skip the transfer.
• If it differs, upload with IfMatch set to the ETag observed by HeadObject.
• Raise a documented exception when the existing object has no comparable full-object SHA-256 or exposes only a composite checksum.
• Reserve checksum, IfMatch, and IfNoneMatch upload arguments because the method owns them.
• Compare body bytes only. Metadata, ACL, storage class, and encryption argument changes do not trigger an upload when bytes match.
• Invoke progress callbacks only for a real transfer.
• Surface 409/412 conditional-write races through the existing upload failure behavior without hidden retries.

Costs and permissions

Each call adds a complete local read for hashing and one HeadObject request. Callers need permission to read object metadata and may need KMS permissions when requesting checksums for encrypted objects. Existing objects must already expose a comparable full-object SHA-256.

The checksum comparison is a snapshot decision. Conditional writes prevent stale creation or replacement, but do not lock the object against later unrelated writers.

Related work

• boto/boto3#4366 adds key/ETag conditional arguments but does not compare content.
• boto/s3transfer#371 adds IfMatch and IfNoneMatch routing for managed uploads but does not provide content comparison.

Development status

A local implementation and tests are available for discussion. The dependency work is based on, rather than independently reimplementing, s3transfer PR #371.

AI tools assisted with implementation and test iteration; human review is pending before the draft is promoted for merge.