MinIO server rejects uploads with aws-chunked encoding when payload >16 MiB
Starting with service/s3 v1.73.0, AWS Go SDK calculates checksums by default and includes them in every request. The checksum is provided either as a separate header when using HTTP, or in a trailer header block using aws-chunked encoding (STREAMING-UNSIGNED-PAYLOAD-TRAILER) when using HTTPS. More detailed explanation of this behavior is available here: aws-sdk-go-v2/issues/1667#issuecomment-1113754596
Current Behavior
When HTTPS is used with MinIO, uploads larger than 16 MiB fail due to an error:
StatusCode: 400, api error BadRequest: chunk too big: choose chunk size <= 16MiB
This leads to two limitations:
- Single object uploads are capped at 16 MiB
- Multipart uploads are capped at ~160 GiB (16 MiB × 10,000 parts)
Expected Behavior
Ideally, MinIO would honor the same default limits when aws-chunked encoding is used, so there is no need to disable SDK's default checksum/integrity protections when uploading over HTTPS.
Steps to Reproduce
module minio-test
go 1.24
require (
github.com/aws/aws-sdk-go-v2 v1.39.0
github.com/aws/aws-sdk-go-v2/config v1.31.8
github.com/aws/aws-sdk-go-v2/credentials v1.18.12
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.42
github.com/aws/aws-sdk-go-v2/service/s3 v1.73.0
)
-----------------
package main
import (
"bytes"
"context"
"crypto/tls"
"fmt"
"net/http"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/credentials"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
func uploadTest(endpoint, bucket, key string, useTLS bool) string {
accessKey := "accessKey"
secretKey := "secretKey"
dataSize := 20 * 1024 * 1024 // 20MB data
data := bytes.Repeat([]byte("0123456789ABCDEF"), dataSize/16)
client := &http.Client{}
if useTLS {
client.Transport = &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, //nolint:gosec
}
}
cfg, err := config.LoadDefaultConfig(context.TODO(),
config.WithCredentialsProvider(
credentials.NewStaticCredentialsProvider(accessKey, secretKey, ""),
),
config.WithHTTPClient(client),
)
s3Client := s3.NewFromConfig(cfg, func(o *s3.Options) {
o.UsePathStyle = true
o.BaseEndpoint = aws.String(endpoint)
})
_, err = s3Client.PutObject(context.TODO(), &s3.PutObjectInput{
Bucket: aws.String(bucket),
Key: aws.String(key),
Body: bytes.NewReader(data),
})
if err != nil {
return fmt.Sprintf("FAILED: %v", err)
}
return "SUCCESS"
}
func main() {
fmt.Println("HTTP :", uploadTest("http://localhost:9100", "test", "test-http", false))
fmt.Println("HTTPS:", uploadTest("https://localhost:9000", "test", "test-https", true))
}
Context
Single SDK is used to interact with all S3-compatible storage systems including Amazon S3 itself
Your Environment
MinIO server version: RELEASE.2025-09-07T16-13-09Z AWS SDK for Go v2: github.com/aws/aws-sdk-go-v2/service/s3 v1.73.0+
Source: minio/minio