#21611·minio

MinIO server rejects uploads with aws-chunked encoding when payload >16 MiB

Author: sandraromanchenkoCreated Sep 28, 2025Updated Nov 21, 2025
Labelscommunitytriage

Starting with service/s3 v1.73.0, AWS Go SDK calculates checksums by default and includes them in every request. The checksum is provided either as a separate header when using HTTP, or in a trailer header block using aws-chunked encoding (STREAMING-UNSIGNED-PAYLOAD-TRAILER) when using HTTPS. More detailed explanation of this behavior is available here: aws-sdk-go-v2/issues/1667#issuecomment-1113754596

Current Behavior

When HTTPS is used with MinIO, uploads larger than 16 MiB fail due to an error: StatusCode: 400, api error BadRequest: chunk too big: choose chunk size <= 16MiB

This leads to two limitations:

  • Single object uploads are capped at 16 MiB
  • Multipart uploads are capped at ~160 GiB (16 MiB × 10,000 parts)

Expected Behavior

Ideally, MinIO would honor the same default limits when aws-chunked encoding is used, so there is no need to disable SDK's default checksum/integrity protections when uploading over HTTPS.

Steps to Reproduce

module minio-test

go 1.24

require (
        github.com/aws/aws-sdk-go-v2 v1.39.0
        github.com/aws/aws-sdk-go-v2/config v1.31.8
        github.com/aws/aws-sdk-go-v2/credentials v1.18.12
        github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.42
        github.com/aws/aws-sdk-go-v2/service/s3 v1.73.0
)
-----------------
package main

import (
	"bytes"
	"context"
	"crypto/tls"
	"fmt"
	"net/http"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/credentials"
	"github.com/aws/aws-sdk-go-v2/service/s3"
)

func uploadTest(endpoint, bucket, key string, useTLS bool) string {
	accessKey := "accessKey"
	secretKey := "secretKey"
	dataSize := 20 * 1024 * 1024 // 20MB data
	data := bytes.Repeat([]byte("0123456789ABCDEF"), dataSize/16)
	client := &http.Client{}
	if useTLS {
		client.Transport = &http.Transport{
			TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, //nolint:gosec
		}
	}
	cfg, err := config.LoadDefaultConfig(context.TODO(),
		config.WithCredentialsProvider(
			credentials.NewStaticCredentialsProvider(accessKey, secretKey, ""),
		),
		config.WithHTTPClient(client),
	)
	s3Client := s3.NewFromConfig(cfg, func(o *s3.Options) {
		o.UsePathStyle = true
		o.BaseEndpoint = aws.String(endpoint)
	})
	_, err = s3Client.PutObject(context.TODO(), &s3.PutObjectInput{
		Bucket: aws.String(bucket),
		Key:    aws.String(key),
		Body:   bytes.NewReader(data),
	})
	if err != nil {
		return fmt.Sprintf("FAILED: %v", err)
	}
	return "SUCCESS"
}

func main() {
	fmt.Println("HTTP :", uploadTest("http://localhost:9100", "test", "test-http", false))
	fmt.Println("HTTPS:", uploadTest("https://localhost:9000", "test", "test-https", true))
}

Context

Single SDK is used to interact with all S3-compatible storage systems including Amazon S3 itself

Your Environment

MinIO server version: RELEASE.2025-09-07T16-13-09Z AWS SDK for Go v2: github.com/aws/aws-sdk-go-v2/service/s3 v1.73.0+