Support additional Amazon S3 checksum algorithms for @uppy/aws-s3-multipart and @uppy/companion
Amazon S3’s PutObject function now lets you specify the use of any one of four widely used checksum algorithms (SHA-1, SHA-256, CRC-32, and CRC-32C) when you upload each of your objects to S3, in addition to MD5. (More details on Amazon's blog post from February 2022)
It would be great if Uppy could support this for S3 Multipart uploads, letting us choose from other algorithms than just MD5, depending on our needs (speed, security, etc.).
A gist of how this works has been created by @hguillermo (comment) in issue https://github.com/transloadit/uppy/issues/3391
What I believe needs to be changed in the codebase is:
Companion
- The params in function createMultipartUpload in uppy/companion/src/server/controllers/s3.js would have to allow for ChecksumAlgorithm to be set ( values: "CRC32"|"CRC32C"|"SHA1"|"SHA256"|string; )
- The functions handling the signing of the parts in s3.js (signPartUpload and batchSignPartsUpload) would have add support for the keys ChecksumCRC32, ChecksumCRC32C, ChecksumSHA1, ChecksumSHA256 in the object passed to s3Client. The checksum values would then be sent in as a parameter from the Uppy Client.
Uppy Client
On the client side the browser would have to calculate a the checksum (CRC32, CRC32C, SHA1 or SHA256) of each individual chunk (just like this suggestion for calculateMD5), pass it to the companion, and once a signed url is received (from Companion or a custom backend) I believe you also have do add the chunk's checksum value as a header sent along the S3 PUT ("x-amz-checksum-crc32", "x-amz-checksum-crc32c", etc)
- https://docs.amazonaws.cn/en_us/AmazonS3/latest/API/API_PutObject.html
- https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html
- https://aws.amazon.com/blogs/aws/new-additional-checksum-algorithms-for-amazon-s3/
Another requirement for this to be possible is that individual headers can be sent along each chunk, which is already addressed in the case for being able to verify the integrity using the "Content-MD5" header. See issue https://github.com/transloadit/uppy/issues/3881 by @kevin-west-10x
Source: transloadit/uppy