git lfs checkout triggers refresh of full index (and racy behavior)

Describe the issue

When doing a lfs checkout, git-lfs starts a subprocess with git update-index -q --refresh --stdin. (See link). It then passes the path of each smudged file to that process, so that the stat() contents for that file can be updated in the index.

I'm questioning whether the --refresh parameter is necessary or desirable here. It seems to trigger a full scan of the entire repository, not just the smudged files.

Looking at the behavior of the --refresh switch in git update-index, we see that it basically just calls a refresh_callback(), and this happens as the options are parsed. (Link). refresh_callback() then scans the entire repository to update the stat data in the index.

So what git update-index -q --refresh --stdin does is:

• Step 1: launch refresh(), which scans the entire repository to update the index for every file
• Step 2: then read paths from stdin line by line, updating the index for those files.

It seems like Step 1 is unnecessary here. (But I might well be missing something.)

I encountered this because of issue #6157 (on Windows, if another process holds a to-be-smudged file open, the delete-then-recreate flow will caused an "access denied" error). In that issue, they mentioned that Windows Defender (or some other antivirus scan) could be the "other process".

But I was seeing the error even with such antivirus disabled. With more tracing, I was able to see that it was actually git-lfs's own git update-index subprocess that was holding files open, if timing worked just right. What happens is that the

• git update-index -q --refresh --stdin was still in "Step 1" (refreshing index for whole repository), and happened to be opening a file that git-lfs was smudging
• git-lfs tried to smudge that file at the same time, causing the error.

I know that that issue has been fixed (in pull request #6221, essentially by just retrying the deletion with delay). But it still seems like it would be cleaner if --refresh was removed, if only to improve speed, or to make it less likely that all retries will fail.

Output of git lfs env

This is the version where the race condition is actually causing lfs checkout to fail. That seems fixed (because of retries), but I think it's still worth looking into removing --refresh.

git-lfs/3.7.1 (GitHub; windows amd64; go 1.25.1; git b84b3384)
git version 2.54.0.windows.1

Endpoint=<suppressed>
LocalWorkingDir=C:\buildagent\work\77ad1ff61fb0fd7f
LocalGitDir=C:\buildagent\work\77ad1ff61fb0fd7f\.git
LocalGitStorageDir=C:\buildagent\work\77ad1ff61fb0fd7f\.git
LocalMediaDir=C:\buildAgent\system\git\git-D7A34037.git\lfs\objects
LocalReferenceDirs=
TempDir=C:\buildAgent\system\git\git-D7A34037.git\lfs\tmp
ConcurrentTransfers=8
TusTransfers=false
BasicTransfersOnly=false
SkipDownloadErrors=false
FetchRecentAlways=false
FetchRecentRefsDays=7
FetchRecentCommitsDays=0
FetchRecentRefsIncludeRemotes=true
PruneOffsetDays=3
PruneVerifyRemoteAlways=false
PruneVerifyUnreachableAlways=false
PruneRemoteName=origin
LfsStorageDir=C:\buildAgent\system\git\git-D7A34037.git\lfs
AccessDownload=none
AccessUpload=none
DownloadTransfers=basic,lfs-standalone-file,ssh
UploadTransfers=basic,lfs-standalone-file,ssh
GIT_EXEC_PATH=C:/Program Files/Git/mingw64/libexec/git-core
git config filter.lfs.process = "git-lfs filter-process"
git config filter.lfs.smudge = "git-lfs smudge -- %f"
git config filter.lfs.clean = "git-lfs clean -- %f"