`SlowDown`: bucket creation concurrency limit reached; retry later during concurrent E2E bucket creation

Supported storage confirmation

• I have reproduced this issue with all RustFS data directories on supported storage (local disks, SAN/Fibre Channel, or JBOD; XFS recommended), without NFS, SMB/Samba/CIFS, or other remote or shared filesystem mounts, including storage backing Docker volumes and Kubernetes PersistentVolumes.

Describe the bug

I’m using RustFS as an S3-compatible object storage backend for automated end-to-end tests.

Each E2E test/test worker initializes its own storage client and creates an isolated bucket to avoid test-data interference. When tests run in parallel, multiple CreateBucket requests are sent to RustFS at nearly the same time.

After a certain level of concurrency, RustFS starts rejecting bucket creation requests with:

SlowDown: bucket creation concurrency limit reached; retry later

This causes random E2E failures because multiple tests attempt to create isolated buckets concurrently.

Steps to reproduce

1. Start RustFS.
2. Use an S3-compatible client, for example the AWS SDK for JavaScript.
3. Send many concurrent CreateBucket requests.

Example reproduction:

import { S3Client, CreateBucketCommand } from '@aws-sdk/client-s3';

const client = new S3Client({
  endpoint: 'http://localhost:9000',
  forcePathStyle: true,
  credentials: {
    accessKeyId: 'test',
    secretAccessKey: 'test',
  },
});
// sending a lot of concurrent create bucket requests 
await Promise.all(
  Array.from({ length: 50 }, async (_, i) => {
    await client.send(
      new CreateBucketCommand({
        Bucket: `test-bucket-${Date.now()}-${i}`,
      }),
    );
  }),
);

Actual Behavior

Some bucket creation requests fail with:

SlowDown: bucket creation concurrency limit reached; retry later

Expected behavior

In a testing environment, I would expect one of the following:

1. The bucket creation concurrency limit should be configurable.
2. RustFS should queue or handle a higher number of concurrent bucket creation requests.
3. RustFS should return a more machine-friendly throttling response, such as HTTP 503 with a Retry-After header.
4. Documentation should describe the recommended retry/backoff strategy for concurrent bucket creation.

RustFS version

RUSTFS v1.0.0-rc.6

Deployment method

Docker / Docker Compose

Server operating system and architecture

cachy os

Possible Improvement

It would be very useful to have a server-side setting such as:

RUSTFS_MAX_BUCKET_CREATION_CONCURRENCY=...

or a similar configuration option, especially for test/CI environments where many short-lived buckets are created dynamically.

Alternatively, RustFS could expose this limit in configuration and/or include a Retry-After header so clients can implement deterministic retry logic.

Additional Context

This issue mainly appears in automated test environments where tests run in parallel and each test needs an isolated bucket.

In production, bucket creation is usually rare, but in E2E test suites, dynamic bucket creation can be frequent and concurrent.