[Bug] S3 cache list ignores objects after the first page
Expected Behavior
k8sgpt cache list should return every object in a configured S3 remote cache. k8sgpt cache purge --all should use the complete listing so that all remote cache objects are deleted.
Current Behavior
S3Cache.List calls ListObjectsV2 once and only processes that response. It does not handle IsTruncated or send the returned NextContinuationToken, so an S3 bucket with more than one response page is silently truncated to the first page (up to S3's default maximum of 1,000 keys).
This makes k8sgpt cache list omit objects without reporting an error. It also makes k8sgpt cache purge --all leave objects behind because it deletes only the objects returned by that incomplete listing.
Steps to Reproduce
- Configure an S3 or S3-compatible remote cache containing more objects than a single
ListObjectsV2response can return, or use the deterministic two-page response described below. - Run
k8sgpt cache listand inspect the displayed object names. - Run
k8sgpt cache purge --alland list the bucket again. - Observe that objects after the first response page are omitted from
cache listand are not deleted bypurge --all.
The behavior is reproduced without a live S3 account using the repository's AWS SDK client and an httptest server. The first XML response contains first, IsTruncated=true, and NextContinuationToken=next-token; the second response contains second. The baseline at origin/main (05247a8) returns only first after one HTTP request, instead of both objects after two requests.
Environment
- k8sgpt version: main (
05247a851ba9292ca57e5070f1d0c4d3986b8d4c) - AWS SDK:
github.com/aws/aws-sdk-gov1.55.7 - S3 implementation: Amazon S3 or an S3-compatible endpoint
- AI Backend/Provider: n/a; the defect is in cache object listing
- OS/Platform: macOS / local
httptestserver
Additional Context
Amazon S3 documents that ListObjectsV2 returns at most 1,000 keys by default. When more keys remain, it sets IsTruncated=true and supplies NextContinuationToken, which must be sent as ContinuationToken on the next request.
References:
- https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html
- https://docs.aws.amazon.com/sdk-for-go/api/service/s3/
The current implementation is in pkg/cache/s3_based.go. The focused fix should make S3Cache.List consume all SDK pages and propagate listing errors. It should not change S3 object storage, loading, deletion, cache-key behavior, or the GCS, Azure, or Interplex implementations.
The local regression test is TestS3CacheListPaginates. It verifies both returned objects and that the continuation request is made. The fix passes the focused cache test, all pkg/cache tests, go test ./... -count=1, go vet ./..., gofmt, and git diff --check.
A focused PR can be submitted after maintainer confirmation.
Source: k8sgpt-ai/k8sgpt