encode_ordinary_batch — 在 32 核心的机箱上出现可重现的多秒钟尾部卡顿 (o200k_base, num_threads=8)
Running `encode_ordinary_batch` with its default `num_threads=8` on a 32-core Sapphire Rapids box, I'm seeing large tail spikes on the slowest run of every 10-run batch. Same input, same build, the worst run is 1.1× to 7.6× slower than the median depending on corpus. Multilingual text is the worst offender: two back-to-back runs produced worst-of-10 times of 9.3s and 14.6s, against medians of 5.0s and 4.8s. I read the performance comment in `src/lib.rs` around line 232 that documents the `fancy_regex`/`regex` scratch-buffer contention and the thread-local clone workaround. That's clearly the right fix for the contention it describes — but the tail spikes I'm seeing are on top of that workaround, so something else is going on.
内容来源: openai/tiktoken