Cranelift: Regression in memory bound loop under signals_based_traps(false) by ~8-18%
We embed wasmtime in a high-throughput HTTP proxy and bisected a throughput regression to #14230 (b5b8f49a04, "Make LastStores a proper lattice"). A memory-bound guest loop runs 8 to 18% slower depending on engine config, and the stock config is unaffected, which is likely why upstream benchmarks did not catch it. Full data, the bisect, and a byte-reproducible source repro are below.
.clif Test Case
This is a runtime performance regression (~8-18% on a memory-bound wasm loop), not a miscompile or crash, so there is no reduced .clif that "fails". The regression is a lost redundant-load elimination across the whole hot function, and clif-util bugpoint doesn't apply to perf deltas.
The hot function is a plain bubble sort (Rust source inline below, wasm build is byte-reproducible with the pinned toolchain). Its CLIF can be dumped from either revision with:
wasmtime compile --emit-clif <dir> \
-O signals-based-traps=n -O memory-reservation=0x800000 \
-O memory-may-move=y bench_workload.wasmHappy to attach the pre/post-#14230 CLIF or disassembly for the sort function if that's useful for triage.
Steps to Reproduce
Build the guest component (byte-deterministic with the pinned toolchain - full source inline below: WIT world + ~170-line
lib.rs+Cargo.toml/Cargo.lock):rustup toolchain install 1.94.1 --profile minimal rustup target add wasm32-wasip2 --toolchain 1.94.1 cargo +1.94.1 build --target wasm32-wasip2 --release md5sum target/wasm32-wasip2/release/bench_workload.wasm # cebda31ff7808793ae03be52e211cc80 - the exact artifact all numbers # below were measured on(
.cargo/config.tomlsets-C link-arg=-zstack-size=131072, and the standard WASI 0.2.0 WIT packages are vendored underwit/deps/: cli, clocks, filesystem, io, random, and sockets - e.g. from the WebAssembly/wasi-cli 0.2.0 release.)Build the A/B runner (single binary linking both wasmtime revisions via cargo package renaming - source in the appendix below): wt_old = rev bc2f967927, wt_new = rev 774dec1c5e.
Run it against the component, pinned, on a quiet machine:
taskset -c 8 cargo run --release -- bench_workload.wasmThe runner instantiates the component with all imports stubbed as traps (
Linker::define_unknown_imports_as_traps- the benched path calls no imports) and timesinvoke("bench_sort_large", [])- a bubble sort of 7680 descendingi32s in linear memory - over 3 alternating rounds per config.
Expected Results
Comparable per-call latency between bc2f967927 and 774dec1c5e for the same wasm and the same Config, on all configs.
Actual Results
Per-call latency regresses when memory accesses compile to explicit bounds-check control flow. The stock config is unaffected (medians of 21 calls):
| config | old (bc2f967927) |
new (774dec1c5e) |
delta |
|---|---|---|---|
stock Config |
24.91 ms | 25.04 ms | +0.5% |
signals_based_traps(false) |
33.64 ms | 36.32 ms | +8.0% |
signals_based_traps(false) + memory_reservation(8<<20) + memory_may_move(true) |
36.60 ms | 43.21 ms | +18.1% |
git bisect run over the 91 commits (bc2f967927..774dec1c5e) on the 18% config was perfectly bimodal (every good rev within ±1% of old, every bad rev within ±1% of new, no ambiguous steps):
First bad commit: b5b8f49a04 - "Make LastStores a proper lattice" (#14230).
We see the same delta end-to-end through our production host (HTTP-benchmarked p50 36.9 -> 43.3 ms for the same guest call).
We understand #14230 fixes real soundness/determinism problems (order-dependent forwarding, illegal forwarding across control-flow joins) and are not asking for a revert - we're reporting the cost in case some precision is recoverable for the explicit-bounds-check pattern. We embed wasmtime in a host that owns its signal handlers, so signals_based_traps(false) is not optional for us.
Versions and Environment
Cranelift version or commit: regression window bc2f967927 (good) -> 774dec1c5e (bad), first bad commit b5b8f49a04 (#14230, cranelift-codegen 0.137.0-dev / wasmtime 50.0.0-dev line). Both built from source, default-features = false, features = ["cranelift", "runtime", "std", "parallel-compilation", "component-model"].
Operating system: Rocky Linux 9.3 (primary numbers), also reproduced (noisier) on Ubuntu 22.04.
Architecture: x86_64 (Intel Xeon Gold 6330, taskset-pinned). Host rustc 1.97.1, guest toolchain pinned to rustc 1.94.1 as above.
Extra Info
The magnitude tracks the exact emitted code shape - both the producing toolchain and the component glue move it (same host, same config, same workload source):
- rustc 1.94.1 build: +18.1%
- identical source rebuilt with rustc 1.97.1: +8.1%
- glue-trimmed variant world (wasi imports only), rustc 1.97.1: +4.3%
- import-free world, rustc 1.97.1: -1.9%
- bare core module (
wasm32-unknown-unknown): ±1-2%
So a minimized repro must preserve the pipeline and toolchain (hence the pinned build), and the exposed population is existing deployed binaries, which embedders cannot recompile away. Our working hypothesis is that the extra branches/blocks from explicit bounds checks interact with the more conservative LastStores meet, but we haven't isolated the exact CLIF - happy to help narrow it.
Appendix: A/B runner source
[package]
name = "laststores-perf-repro"
version = "0.1.0"
edition = "2021"
[dependencies]
wt_old = { package = "wasmtime", git = "https://github.com/bytecodealliance/wasmtime", rev = "bc2f967927", default-features = false, features = ["cranelift", "runtime", "std", "parallel-compilation", "component-model"] }
wt_new = { package = "wasmtime", git = "https://github.com/bytecodealliance/wasmtime", rev = "774dec1c5e", default-features = false, features = ["cranelift", "runtime", "std", "parallel-compilation", "component-model"] }use std::time::Instant;
macro_rules! bench {
($wt:ident, $slow_cfg:expr, $bytes:expr) => {{
use $wt::component::{Component, Linker, Val};
let mut c = $wt::Config::new();
c.wasm_component_model(true);
if $slow_cfg {
c.signals_based_traps(false);
c.memory_reservation(8 << 20);
c.memory_may_move(true);
}
let engine = $wt::Engine::new(&c).unwrap();
let component = Component::new(&engine, $bytes).unwrap();
let mut linker: Linker<()> = Linker::new(&engine);
linker.define_unknown_imports_as_traps(&component).unwrap();
let mut store = $wt::Store::new(&engine, ());
let inst = linker.instantiate(&mut store, &component).unwrap();
let f = inst.get_func(&mut store, "invoke").unwrap();
let args = [Val::String("bench_sort_large".into()), Val::List(vec![])];
let mut ms = Vec::new();
for i in 0..9 {
let mut results = [Val::Bool(false)];
let t = Instant::now();
f.call(&mut store, &args, &mut results).unwrap();
if i >= 2 { ms.push(t.elapsed().as_secs_f64() * 1e3); }
assert!(matches!(&results[0], Val::Result(Ok(_))));
}
ms
}};
}
fn median(v: &mut Vec<f64>) -> f64 {
v.sort_by(|a, b| a.partial_cmp(b).unwrap());
v[v.len() / 2]
}
fn main() {
let bytes = std::fs::read(std::env::args().nth(1).expect("path to component")).unwrap();
for (label, slow) in [("stock", false), ("no-signals+8M-reservation", true)] {
// Alternate old/new rounds so CPU frequency drift hits both sides;
// run pinned (taskset) on a quiet machine.
let (mut old_ms, mut new_ms) = (Vec::new(), Vec::new());
for _ in 0..3 {
old_ms.extend(bench!(wt_old, slow, &bytes));
new_ms.extend(bench!(wt_new, slow, &bytes));
}
let (old, new) = (median(&mut old_ms), median(&mut new_ms));
println!("{label:>26}: old {old:7.2} ms new {new:7.2} ms ({:+.1}%)", (new / old - 1.0) * 100.0);
}
}Appendix: guest source
Cargo.lock
Details# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "anyhow"
version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "bench-workload"
version = "0.1.0"
dependencies = [
"wit-bindgen",
]
[[package]]
name = "bitflags"
version = "2.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4388bee8683e3d04af747c73422af53102d2bd24d9eadb6cbc100baef4b43f8"
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "foldhash"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "futures"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b147ee9d1f6d097cef9ce628cd2ee62288d963e16fb287bd9286455b241382d"
dependencies = [
"futures-channel",
"futures-core",
"futures-executor",
"futures-io",
"futures-sink",
"futures-task",
"futures-util",
]
[[package]]
name = "futures-channel"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d"
dependencies = [
"futures-core",
"futures-sink",
]
[[package]]
name = "futures-core"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d"
[[package]]
name = "futures-executor"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "baf29c38818342a3b26b5b923639e7b1f4a61fc5e76102d4b1981c6dc7a7579d"
dependencies = [
"futures-core",
"futures-task",
"futures-util",
]
[[package]]
name = "futures-io"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cecba35d7ad927e23624b22ad55235f2239cfa44fd10428eecbeba6d6a717718"
[[package]]
name = "futures-macro"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e835b70203e41293343137df5c0664546da5745f82ec9b84d40be8336958447b"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "futures-sink"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c39754e157331b013978ec91992bde1ac089843443c49cbc7f46150b0fad0893"
[[package]]
name = "futures-task"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393"
[[package]]
name = "futures-util"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6"
dependencies = [
"futures-channel",
"futures-core",
"futures-io",
"futures-macro",
"futures-sink",
"futures-task",
"memchr",
"pin-project-lite",
"slab",
]
[[package]]
name = "hashbrown"
version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"foldhash",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "id-arena"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "indexmap"
version = "2.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9"
dependencies = [
"equivalent",
"hashbrown 0.17.1",
"serde",
"serde_core",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "leb128fmt"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
[[package]]
name = "log"
version = "0.4.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "953f07c43838f8e6f9758cab68bf5bed85465e7587ebe0b823f1bcd81978ad3a"
[[package]]
name = "memchr"
version = "2.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b947ae49db0d222b1dbc6b113ce7248a3fc3a6ca21b696717bfc000ba4484d8"
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "pin-project-lite"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "semver"
version = "1.0.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd"
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.150"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8014e44b4736ed0538adeecded0fce2a272f22dc9578a7eb6b2d9993c74cfb9"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "slab"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unicode-xid"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
[[package]]
name = "wasm-encoder"
version = "0.239.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5be00faa2b4950c76fe618c409d2c3ea5a3c9422013e079482d78544bb2d184c"
dependencies = [
"leb128fmt",
"wasmparser",
]
[[package]]
name = "wasm-metadata"
version = "0.239.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20b3ec880a9ac69ccd92fbdbcf46ee833071cf09f82bb005b2327c7ae6025ae2"
dependencies = [
"anyhow",
"indexmap",
"wasm-encoder",
"wasmparser",
]
[[package]]
name = "wasmparser"
version = "0.239.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c9d90bb93e764f6beabf1d02028c70a2156a6583e63ac4218dd07ef733368b0"
dependencies = [
"bitflags",
"hashbrown 0.15.5",
"indexmap",
"semver",
]
[[package]]
name = "wit-bindgen"
version = "0.46.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f17a85883d4e6d00e8a97c586de764dabcc06133f7f1d55dce5cdc070ad7fe59"
dependencies = [
"bitflags",
"futures",
"once_cell",
"wit-bindgen-rust-macro",
]
[[package]]
name = "wit-bindgen-core"
version = "0.46.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cabd629f94da277abc739c71353397046401518efb2c707669f805205f0b9890"
dependencies = [
"anyhow",
"heck",
"wit-parser",
]
[[package]]
name = "wit-bindgen-rust"
version = "0.46.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a4232e841089fa5f3c4fc732a92e1c74e1a3958db3b12f1de5934da2027f1f4"
dependencies = [
"anyhow",
"heck",
"indexmap",
"prettyplease",
"syn",
"wasm-metadata",
"wit-bindgen-core",
"wit-component",
]
[[package]]
name = "wit-bindgen-rust-macro"
version = "0.46.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e0d4698c2913d8d9c2b220d116409c3f51a7aa8d7765151b886918367179ee9"
dependencies = [
"anyhow",
"prettyplease",
"proc-macro2",
"quote",
"syn",
"wit-bindgen-core",
"wit-bindgen-rust",
]
[[package]]
name = "wit-component"
version = "0.239.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88a866b19dba2c94d706ec58c92a4c62ab63e482b4c935d2a085ac94caecb136"
dependencies = [
"anyhow",
"bitflags",
"indexmap",
"log",
"serde",
"serde_derive",
"serde_json",
"wasm-encoder",
"wasm-metadata",
"wasmparser",
"wit-parser",
]
[[package]]
name = "wit-parser"
version = "0.239.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "55c92c939d667b7bf0c6bf2d1f67196529758f99a2a45a3355cc56964fd5315d"
dependencies = [
"anyhow",
"id-arena",
"indexmap",
"log",
"semver",
"serde",
"serde_derive",
"serde_json",
"unicode-xid",
"wasmparser",
]
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"Cargo.toml
[package]
name = "bench-workload"
version = "0.1.0"
edition = "2021"
[dependencies]
wit-bindgen = "0.46"
[lib]
crate-type = ["cdylib"]
path = "src/lib.rs"
[profile.release]
opt-level = 3.cargo/config.toml
[target.wasm32-wasip2]
rustflags = ["-C", "link-arg=-zstack-size=131072"]lib.rs
//! bench_workload - five small compute kernels behind a component-model
//! dispatcher (`invoke(name, params)`). The benchmark calls
//! "bench_sort_large": a bubble sort of 7680 descending i32s. Each kernel
//! returns its computed value (never a constant) so the work cannot be
//! optimized away; `std::hint::black_box` is additional insurance.
wit_bindgen::generate!({
world: "funcs",
path: "wit",
//Source: bytecodealliance/wasmtime