leasing: LockWriteOps range branch races with lease-cache mutation
Bug report criteria
- This bug report is not security related, security issues should be disclosed privately via [email protected].
- This is not a support request or question, support requests or questions should be raised in the etcd discussion forums.
- I have read the etcd bug reporting guidelines.
- Existing open issues and the etcd frequently asked questions have been checked and this is not a duplicate.
What happened?
leaseCache.LockWriteOps handles single-key writes through lc.Lock, which
holds lc.mu. Its range branch instead iterates lc.entries without holding
that mutex:
for k := range lc.entries {
...
if wc, _ := lc.Lock(k); wc != nil {
ret = append(ret, wc)
}
}
Other cache operations such as Add and Evict mutate lc.entries while
holding lc.mu. The race detector reports the unsynchronized map iteration
against those writes. Outside the race detector, concurrent map iteration and
mutation can also terminate the process.
What did you expect to happen?
The range branch should synchronize access to lc.entries just like the other
cache operations. Running the focused test with -race should not report a data
race.
How can we reproduce it (as minimally and precisely as possible)?
Add this complete unit test as
client/v3/leasing/cache_lock_write_ops_test.go and run:
cd client/v3
go test -race ./leasing -run '^TestLeaseCacheLockWriteOpsRangeSynchronizesEntries$' -count=1
// Copyright 2026 The etcd Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package leasing
import (
"context"
"fmt"
"testing"
"time"
"go.etcd.io/etcd/api/v3/etcdserverpb"
"go.etcd.io/etcd/api/v3/mvccpb"
clientv3 "go.etcd.io/etcd/client/v3"
)
func TestLeaseCacheLockWriteOpsRangeSynchronizesEntries(t *testing.T) {
lc := leaseCache{
entries: map[string]*leaseKey{
"k/initial": leaseKeyForLockWriteOpsRange("k/initial", 1),
},
revokes: make(map[string]time.Time),
header: &etcdserverpb.ResponseHeader{Revision: 1},
}
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
donec := make(chan struct{}, 2)
go func() {
defer func() { donec <- struct{}{} }()
for ctx.Err() == nil {
wcs := lc.LockWriteOps([]clientv3.Op{
clientv3.OpDelete("k/", clientv3.WithRange("k0")),
})
closeAll(wcs)
}
}()
go func() {
defer func() { donec <- struct{}{} }()
for i := 0; ctx.Err() == nil; i++ {
key := fmt.Sprintf("k/%d", i)
lc.Add(key, getResponseForLockWriteOpsRange(key, int64(i+2)), clientv3.OpGet(key))
if i%2 == 0 {
lc.Evict(key)
}
}
}()
<-donec
<-donec
}
func leaseKeyForLockWriteOpsRange(key string, rev int64) *leaseKey {
return &leaseKey{
response: getResponseForLockWriteOpsRange(key, rev),
rev: rev,
}
}
func getResponseForLockWriteOpsRange(key string, rev int64) *clientv3.GetResponse {
return &clientv3.GetResponse{
Header: &etcdserverpb.ResponseHeader{Revision: rev},
Kvs: []*mvccpb.KeyValue{
{
Key: []byte(key),
Value: []byte("value"),
CreateRevision: rev,
ModRevision: rev,
Version: 1,
},
},
Count: 1,
}
}
Anything else we need to know?
The unsynchronized iteration is in the current LockWriteOps range
branch.
The existing leaseCache.LockRange already performs the range scan and replaces
the matching wait channels while holding lc.mu; the range branch can reuse or
factor that synchronization. A fix should avoid taking lc.mu and then calling
lc.Lock, because lc.Lock acquires the same mutex.
Etcd version (please run commands below)
Confirmed on:
$ git rev-parse HEAD
12caed621b38312cc1084113415bf135c59e6457
$ go version
go version go1.26.5 linux/amd64
the same unsynchronized range loop is present at v3.6.13
(b0f9ef190952e6e66a778513097a02ee41220727)
Etcd configuration (command line flags or environment variables)
No server configuration is needed; this is a client-side unit test.
Etcd debug information
Not applicable.
Relevant log output
WARNING: DATA RACE
Read at ... by goroutine ...:
go.etcd.io/etcd/client/v3/leasing.(*leaseCache).LockWriteOps()
client/v3/leasing/cache.go:96
Previous write at ... by goroutine ...:
go.etcd.io/etcd/client/v3/leasing.(*leaseCache).Add()
client/v3/leasing/cache.go:133
FAIL go.etcd.io/etcd/client/v3/leasing
Source: etcd-io/etcd