perf: detectRule() calls r.Regex.FindAllStringIndex twice per match cycle, doubling regex work on every scan
Summary
Hey! I was going through the codebase and noticed something in detect/detect.go
that I'd love to help fix if you're open to it.
The detectRule() function runs the rule's regex against the current fragment
twice per invocation — once on line 442 to check if any matches exist (early
exit), and again on line 452 to actually iterate over those matches. Both calls
receive identical arguments and produce identical results. The first result is
just thrown away.
Location
detect/detect.go lines 442 and 452
The Code
// Line 442 — first call, result only used for the early-exit check
matches := r.Regex.FindAllStringIndex(currentRaw, -1)
if len(matches) == 0 {
return findings
}
// ... some setup in between ...
// Line 452 — second call, same regex, same input, same result
for _, matchIndex := range r.Regex.FindAllStringIndex(currentRaw, -1) {I also noticed the maintainer left a // TODO profile this comment on line 447
right between these two calls, which made me think this area was already on the
radar for performance work.
Why It Matters
- Every fragment that passes the Aho-Corasick prefilter and contains at least one match pays 2× the regex cost unnecessarily
- Regex execution is the dominant cost of a gitleaks scan
- With up to 40 concurrent goroutines (the semaphore default), this wasted work compounds across all parallel scans
- On large repos or files with many matches this is measurable wall-clock time
The Fix
Reuse the result from the first call instead of running the regex again:
// Capture once
matches := r.Regex.FindAllStringIndex(currentRaw, -1)
if len(matches) == 0 {
return findings
}
// Reuse — no second regex call needed
for _, matchIndex := range matches {This is a 1-line change with zero behavioral impact — the second call was already producing the same result as the first.
What I'll Include in the PR
- The fix in
detect/detect.go - A benchmark test (
BenchmarkDetectRule_WithMatches) to prove and guard the improvement going forward
Happy to open a PR for this right away if it looks good to you. Just let me know if you'd prefer a different approach or if there's something about this area of the code I might be missing!
cc @zricethezav
Source: gitleaks/gitleaks