Skip to content

Add reproducible C/Rust benchmarks and measured HTTP/2 NODELAY findings - #3

Open
Ray0907 wants to merge 7 commits into
mainfrom
bench/http2-e2e
Open

Ray0907 wants to merge 7 commits into
mainfrom
bench/http2-e2e

Conversation

@Ray0907

@Ray0907 Ray0907 commented Sep 19, 2026

Copy link
Copy Markdown
Owner

Summary

Add a local-only independent Go load generator/origin, a bounded Linux benchmark harness, a read-only Actions workflow, and the completed measured report at docs/benchmarks/2026-09-19.md.

This PR changes benchmark tooling and documentation only. Runtime source and main are unchanged. A TCP_NODELAY experiment is built only in an isolated checkout and recorded as a separate binary/diff, not silently applied to the baseline.

Completed verification

Benchmark run: https://github.com/Ray0907/tinyproxy-rust/actions/runs/35413653419

Runtime baseline: d500d2510475daafa8269e09caea6d2043c26aa5; C Tinyproxy 1.11.3, release checksum verified. Benchmark harness in that run: 93353a2dcdec2ecf0d9dae5e373611b99a06e6e3.

  • Both baseline and experimental Rust executables passed 57 regression tests and verified-TLS curl interoperability; experiment also passed Clippy with warnings denied.
  • 35 timed workloads × 3 shuffled rounds = 105 observations, 1,557,891 validated measured completions, zero measured or warmup request errors.
  • Six held-tunnel configurations completed with zero setup errors; FDs returned to pre-load counts after clients closed.
  • Artifact retains raw rounds, configurations, dependency resolution, executable hashes, exact experiment diff and logs; no test private keys.

Findings, not blanket claims

  • Fresh HTTP/1 small responses: Rust 5,240.8 vs C 3,107.1 req/s at concurrency 32, with lower p99 in this environment.
  • HTTP/2 small responses revealed an approximately 41 ms baseline wait. NODELAY reduced TLS-H2 concurrency-1 p99 from 41.103 to 0.383 ms. At 32 TLS-H2 streams, throughput increased from 754.6 to 6,305.5 req/s.
  • NODELAY is not a free bulk-throughput win: POST throughput fell 10.2%, while p99 improved; H1 tunnel echo throughput fell 4.5%.
  • C used less resident memory with 512 idle tunnels (10.77 MiB vs Rust H1 20.32 MiB and Rust H2/TLS 17.82 MiB). Rust used two threads rather than 513; H2 used 16 client TCP connections rather than 512.

Limits and review notes

Short closed-loop loopback measurements on a shared Ubuntu CI VM, 1 s warmup + 4 s measurement per round. Separate logical CPU affinities are not physical-core isolation: proxy and origin share an SMT core in the recorded topology. Numbers are medians of per-round values. Fresh/reused H1 and cleartext/TLS cases are labeled separately. No WAN, full conformance, DoS, long-soak or universal performance superiority claim.

Earlier attempts failed due harness configuration/procfs-monitor errors; those incomplete runs are excluded and documented. The NODELAY result supports a TCP buffering explanation but no packet capture was taken. Follow-up runtime tuning should separately evaluate latency, relay buffers and bulk-transfer CPU costs.

No merge or release is performed by this PR.

…urements

Use an independent Go client/origin, per-process CPU affinity, validated
response payloads, three shuffled rounds, explicit fresh/reused HTTP1
cases, HTTP2 TLS/cleartext, CONNECT throughput and held-tunnel resources.
Build and test d500d25 unchanged against C Tinyproxy 1.11.3.
Keep workflow read-only and preserve raw results without test private keys.
…xperiment

Keep main and baseline source unchanged. Compare accepted/outgoing TCP
NODELAY as a separate tested binary; retain its patch and binary hash.
Capture startup errors and distinguish logical-CPU affinity from physical
core isolation. Do not count the failed C harness setup as a proxy result.
C file-path directives require quotes; no PID or log file is needed for
these foreground test processes. Validate C, Rust, and experimental
plain/TLS modes through the independent client before timed rounds.
A process can disappear or temporarily deny fd inspection after poll()
reports it alive. Skip that sample on OSError; network successes/errors
remain independently recorded by the load generator.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant