Replay using kafka

Hi,

We used Goreplay to capture and replay traffic using KAFKA for our system, which gets a throughput of 1.6 lakh RPM (equivalent to 2700 RPS).

We successfully captured data over a 2-hour run, pushing 2 crore messages during that period. However, when we attempt to replay the captured traffic, we observe a throughput of only ~300 RPS, significantly lower than the captured rate of 2700 RPS.

./gor --input-kafka-host broker-ip --input-kafka-topic topic-name --input-kafka-offset 0 --output-http prod_url --stats --output-http-stats --verbose 1 [DEBUG][elapsed 472.957µs]: output_http:latest,mean,max,count,count/second,gcount [DEBUG][elapsed 68.516µs]: output_http:0,0,0,0,0,5 2025/01/04 23:30:13 [PPID 14882 and PID 15550] Version:2.0.0 [DEBUG][elapsed 5.000066334s]: output_http:0,0,50,1427,285,99 [DEBUG][elapsed 5.000299959s]: output_http:33,0,53,1390,278,12 [DEBUG][elapsed 1.387815704s]: output_http:0,1,55,1404,280,112 [DEBUG][elapsed 5.000939106s]: output_http:50,0,50,1085,217,118 [DEBUG][elapsed 5.000809986s]: output_http:13,1,43,1043,208,42 [DEBUG][elapsed 5.000092741s]: output_http:16,1,60,1404,280,136