HTTP/3 very slow
I was looking at using oha to run some load tests with HTTP/3, but I found the request rate seems to be very low for the same URL using http2 vs http3.
On M1 Macbook Pro, after cargo install --features http3 oha
❯ oha --version
oha 1.12.1
❯ oha --http-version 2 https://www.google.com --output-format json --no-tui -z 5s
{
"summary": {
"successRate": 1.0,
"total": 5.006699625,
"slowest": 1.1751909170000001,
"fastest": 0.244194458,
"average": 0.4514453835295238,
"requestsPerSec": 114.84611481960033,
"totalData": 1886846,
"sizePerRequest": 3593,
"sizePerSec": 376864.2301963541
},
"latencyPercentiles": {
"p10": 0.300358958,
"p25": 0.336630375,
"p50": 0.411378375,
"p75": 0.511191792,
"p90": 0.676943,
"p95": 0.79763875,
"p99": 1.01387975,
"p99.9": 1.1751909170000001,
"p99.99": 1.1751909170000001
},
"statusCodeDistribution": {
"200": 193,
"302": 332
},
}
❯ oha --http-version 3 https://www.google.com --output-format json --no-tui -z 5s
{
"summary": {
"successRate": 1.0,
"total": 35.060763417,
"slowest": 1.198709834,
"fastest": 0.24654925,
"average": 0.42182671901604263,
"requestsPerSec": 17.42688807807713,
"totalData": 1748157,
"sizePerRequest": 3116,
"sizePerSec": 49860.779675789
},
"latencyPercentiles": {
"p10": 0.2904625,
"p25": 0.330885209,
"p50": 0.388051417,
"p75": 0.493884125,
"p90": 0.549086292,
"p95": 0.682034583,
"p99": 0.944012625,
"p99.9": 1.198709834,
"p99.99": 1.198709834
},
"statusCodeDistribution": {
"200": 163,
"302": 398
},
}Naturally, google.com is not a good URL to run a load test against, but the status code distributions seem similar for both so I don't think that is itself an issue, and I see similar behavior to an internal test I am running using Envoy. I would expect HTTP2 vs 3 to perform similarly to each other, but here we see 10x less rate for HTTP/3. The latencies, while looking higher on HTTP/3, don't seem dramatically different, so this makes me think there may be some hanging outside of request processing? Note the total duration, which is 35s instead of the requested 5s.
I understand HTTP/3 support here is new and experimental so not a huge priority to fix this, but wanted to make sure it's been shared. Perhaps it's just missing cleanup logic per request.
Source: hatoo/oha