[Bug] v2.2.17 uploads fail with bundled RustFS because pre-upload HeadObject returns 403
Client Type
Web (Desktop Browser)
Operating System
Other Linux
Deployment Platform
Self hosting Docker
Version
v2.2.17 or newer
Browser
Chrome
What happened?
File uploads worked before upgrading LobeHub. After upgrading to v2.2.17, every upload fails in upload.createS3PreSignedUrl:
Error in tRPC handler (lambda) on path: upload.createS3PreSignedUrl
Error [TRPCError]: UnknownError
[cause]: Unknown: UnknownError
'$fault': 'client'
'$metadata': {
httpStatusCode: 403,
requestId: '...',
attempts: 1
}The failure happens before the browser receives a presigned URL, so this is not a browser CORS or PUT-upload failure.
Configuration
S3_BUCKET=lobe S3_INTERNAL_ENDPOINT=http://rustfs:9000 S3_REGION=us-east-1 S3_ENDPOINT=https://s3.example.com S3_ENABLE_PATH_STYLE=1 S3_SET_ACL=0
LobeHub's S3_ACCESS_KEY_ID is the same as RustFS's root RUSTFS_ACCESS_KEY. The secret values were compared by SHA-256 and match.
The running RustFS container and pulled rustfs/rustfs:latest image IDs also match.
Regression
The behavior appears to have been introduced by commit a20880a1252803230b065e4ad77d29e2a710d570 / PR #19119.
The new upload-reservation flow calls:
params.storage.getFileMetadata(params.pathname)
before issuing the presigned URL. This performs HeadObject against a new, nonexistent pathname and expects a 404. RustFS returns 403, which is rethrown and becomes UnknownError.
In v2.2.16, createS3PreSignedUrl directly generated the presigned PUT URL and uploads worked.
How to reproduce it?
Deploy LobeHub v2.2.17 or newer using the official Docker Compose configuration with its bundled RustFS service.
Configure the documented RustFS/S3 settings:
S3_BUCKET=lobe S3_INTERNAL_ENDPOINT=http://rustfs:9000 S3_REGION=us-east-1 S3_ENABLE_PATH_STYLE=1 S3_SET_ACL=0
Use the RustFS root credentials for S3_ACCESS_KEY_ID and S3_SECRET_ACCESS_KEY.
Confirm that the lobe bucket exists and both containers are healthy.
Sign in to LobeHub and attempt to upload any small file or image.
Observe that the upload fails before it begins. The LobeHub container logs show:
Error in tRPC handler (lambda) on path: upload.createS3PreSignedUrl Error [TRPCError]: UnknownError ... httpStatusCode: 403
Downgrade only LobeHub to v2.2.16 without changing RustFS, its data, credentials, bucket, or S3 configuration.
Repeat the same upload and observe that it succeeds.
The regression occurs because v2.2.17 performs a server-side HeadObject request for the newly generated, nonexistent pathname before creating the presigned upload URL. RustFS returns 403 for that request, and LobeHub aborts the upload instead of generating the URL.
What it should be?
The official Docker Compose storage backend should work with the upload flow shipped by LobeHub.
The existence check should use an operation compatible with the bundled RustFS version, or otherwise handle the provider behavior safely. Treating all 403 responses as “not found” would not be safe.
An integration regression test using the official RustFS Compose service would help prevent this.
Additional Information
Related upstream RustFS report
️ Willing to Submit a PR?
No, but I am happy to help test the fix
✅ Validations
- Read the docs.
- Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
- Make sure this is a LobeChat issue and not a third-party library or provider issue.
- Check that this is a concrete bug. For Q&A, please use GitHub Discussions or join our Discord Server.
Source: lobehub/lobehub