Git LFS crashes when Batch API responses include duplicate objects
Describe the bug
The Git LFS client crashes when a Git LFS Batch API response includes duplicate elements for the same object.
To Reproduce
The following shell integration script reproduces the bug when the lfstest-gitserver utility is modified to return duplicate object elements in its JSON responses to Batch API requests. The necessary patch is shown below the reproduction script.
#!/usr/bin/env bash
. "$(dirname "$0")/testlib.sh"
begin_test "batch transfer with duplicate OID"
(
set -e
reponame="batch-transfer-duplicate-oid"
setup_remote_repo "$reponame"
clone_repo "$reponame" "$reponame"
git lfs track "*.bin"
# This content announces to the server that it should respond to
# to all Batch API requests with JSON that includes duplicate
# elements in its "objects" array for each object.
contents="send-duplicate-oid"
contents_oid="$(calc_oid "$contents")"
printf "%s" "$contents" >test.bin
git add .gitattributes test.bin
git commit -m "initial commit"
git push origin main
rm -rf .git/lfs/objects
git lfs pull
assert_local_object "$contents_oid" "${#contents}"
)
end_test--- a/t/cmd/lfstest-gitserver.go
+++ b/t/cmd/lfstest-gitserver.go
@@ -68,7 +68,8 @@ var (
"storage-download-retry-range", "storage-download-retry-range-rejected", "storage-download-retry-no-invalid-range",
"storage-download-encoding-gzip", "storage-download-encoding-zstd", "storage-download-encoding-zstd-retry-range",
"storage-download-encoding-zstd-1", "storage-download-encoding-zstd-2", "storage-download-encoding-zstd-3",
- "send-verify-action", "send-deprecated-links", "redirect-storage-upload", "batch-hash-algo-empty", "batch-hash-algo-invalid",
+ "send-verify-action", "send-deprecated-links", "send-duplicate-oid",
+ "redirect-storage-upload", "batch-hash-algo-empty", "batch-hash-algo-invalid",
"auth-bearer", "auth-multistage",
}
@@ -588,6 +589,10 @@ func lfsBatchHandler(w http.ResponseWriter, r *http.Request, id, repo string) {
}
res = append(res, o)
+
+ if handler == "send-duplicate-oid" {
+ res = append(res, o)
+ }
}
ores := batchResp{HashAlgorithm: hashAlgo, Transfer: transferChoice, Objects: res}The script fails during the git push origin main command and logs the following Go stack backtrace:
panic: sync: negative WaitGroup counter
goroutine 97 [running]:
sync.(*WaitGroup).Add(0x856c1f2f200, 0xffffffffffffffff)
sync/waitgroup.go:118 +0x264
sync.(*WaitGroup).Done(...)
sync/waitgroup.go:156
github.com/git-lfs/git-lfs/v3/tq.(*abortableWaitGroup).Done(0x856c1fce900?)
github.com/git-lfs/git-lfs/v3/tq/transfer_queue.go:168 +0xbc
github.com/git-lfs/git-lfs/v3/tq.(*TransferQueue).handleTransferResult(0x856c1ff0700, {0x856c2120ba0, {0x0, 0x0}}, 0x0?)
github.com/git-lfs/git-lfs/v3/tq/transfer_queue.go:885 +0x650
github.com/git-lfs/git-lfs/v3/tq.(*TransferQueue).addToAdapter.func1()
github.com/git-lfs/git-lfs/v3/tq/transfer_queue.go:745 +0x108
created by github.com/git-lfs/git-lfs/v3/tq.(*TransferQueue).addToAdapter in goroutine 68
github.com/git-lfs/git-lfs/v3/tq/transfer_queue.go:731 +0x160Expected behaviour
The client should report an error instead of crashing when a Batch API response includes duplicate elements for the same object.
System environment
The examples above were performed on macOS, but should behave the same way on any platform.
Output of git lfs env
git-lfs/3.8.0 (GitHub; darwin arm64; go 1.27.0)
git version 2.55.0Additional context
Note that if the lfstest-gitserver utility is instead modified to only insert a duplicate object in its response to Batch API download requests, per the alternate patch below, then the reproduction script fails during the git lfs pull command and not during the git push origin main command.
The stack backtrace is identical in both cases, however.
Alternate Patch for Test Server Utility--- a/t/cmd/lfstest-gitserver.go
+++ b/t/cmd/lfstest-gitserver.go
@@ -68,7 +68,8 @@ var (
"storage-download-retry-range", "storage-download-retry-range-rejected", "storage-download-retry-no-invalid-range",
"storage-download-encoding-gzip", "storage-download-encoding-zstd", "storage-download-encoding-zstd-retry-range",
"storage-download-encoding-zstd-1", "storage-download-encoding-zstd-2", "storage-download-encoding-zstd-3",
- "send-verify-action", "send-deprecated-links", "redirect-storage-upload", "batch-hash-algo-empty", "batch-hash-algo-invalid",
+ "send-verify-action", "send-deprecated-links", "send-duplicate-oid",
+ "redirect-storage-upload", "batch-hash-algo-empty", "batch-hash-algo-invalid",
"auth-bearer", "auth-multistage",
}
@@ -588,6 +589,10 @@ func lfsBatchHandler(w http.ResponseWriter, r *http.Request, id, repo string) {
}
res = append(res, o)
+
+ if handler == "send-duplicate-oid" && action == "download" {
+ res = append(res, o)
+ }
}
ores := batchResp{HashAlgorithm: hashAlgo, Transfer: transferChoice, Objects: res}/cc @Yeaseen as reporter
Source: git-lfs/git-lfs