Skip to content

Fix HTTP/2 latency with client-side NODELAY and vectored I/O - #4

Open
Ray0907 wants to merge 4 commits into
mainfrom
perf/http2-transport
Open

Ray0907 wants to merge 4 commits into
mainfrom
perf/http2-transport

Conversation

@Ray0907

@Ray0907 Ray0907 commented Sep 19, 2026 •

Copy link
Copy Markdown
Owner

Implementation

Follow up the measured transport findings in #3 with an actual runtime change, without applying blanket TCP_NODELAY to every socket.

  • Enable TCP_NODELAY on accepted/client-facing sockets before TLS/protocol negotiation. Keep outbound sockets' existing buffering policy, including the Bind path.
  • Forward is_write_vectored and poll_write_vectored through ActivityIo and PrefixedIo. Each call delegates once; metrics count only accepted bytes. Pending, errors, zero-byte writes and flushes do not add traffic or refresh write activity.
  • Move the fixed 8 KiB H2 CONNECT download buffer out of nested async future state. Keep bounded streaming, half-close, reset polling, cancellation and shutdown behavior.
  • Add nine socket/wrapper/future-size regression tests without exposing private protocol types or adding dependencies/configuration.

Completed verification

Verified candidate: de6053d12c3259f33125e112866774d8ba60aa8a.

CI: https://github.com/Ray0907/tinyproxy-rust/actions/runs/35421891598

  • 66 regression tests passed on Linux and macOS.
  • Format, Clippy with warnings denied, release builds and configuration checks passed.
  • Real curl/verified-TLS interoperability passed on Linux (macOS curl step is intentionally not run).

Corrected benchmark: https://github.com/Ray0907/tinyproxy-rust/actions/runs/35421889856

  • Baseline: unchanged d500d2510475daafa8269e09caea6d2043c26aa5.
  • Independent Go generator/origin pinned at fbd6141e946261011c5f01696050f0d15ef3a99a.
  • Separate build directories, identical Cargo.lock, Rust 1.98.1, Go 1.27.1. Both release binaries passed verified-TLS curl checks.
  • 10 paired workloads × 3 shuffled rounds = 60 observations; 303,844 validated measured completions, zero measured errors and zero warmup errors.
  • Four 512-held-tunnel setups completed without errors; descriptors returned to their pre-load counts after clients closed.

Same-run results

Medians of three per-round values. Small responses are 1 KiB; H2 uses one client connection. The 1 MiB bulk workloads use eight concurrent workers. Tunnel echo throughput counts payload in one direction, not both directions combined.

Workload Baseline Candidate Observation
TLS H2, concurrency 1, p99 41.075 ms 0.359 ms Fixed-delay behavior removed
TLS H2, concurrency 32 763.6 req/s 7,022.5 req/s 9.20x throughput; p99 42.992 → 6.059 ms
h2c, concurrency 1, p99 41.120 ms 0.356 ms Fixed-delay behavior removed
h2c, concurrency 32 763.3 req/s 7,150.4 req/s 9.37x throughput
HTTP/1 fresh GET, 1 MiB 1,036.5 MiB/s 1,140.3 MiB/s +10.0%
HTTP/1 fresh POST/echo, 1 MiB 394.7 MiB/s 421.4 MiB/s +6.8%; p99 remains 54.808 ms
Established HTTP/1 CONNECT echo, 1 MiB 299.2 MiB/s 395.6 MiB/s +32.2%; p99 50.063 → 24.471 ms
Established TLS H2 CONNECT echo, 1 MiB 308.4 MiB/s 293.8 MiB/s 4.7% lower throughput; p99 28.520 → 30.080 ms

Small HTTP/1 responses at concurrency 32 were also slightly slower: fresh-connection throughput 5,402.6 → 5,271.4 req/s (-2.4%); reuse-enabled throughput 6,634.8 → 6,510.0 req/s (-1.9%). This is not an across-the-board performance win. The main benefit is eliminating the observed HTTP/2 fixed delay while improving plain bulk transfers; the H2/TLS bulk-tunnel cost remains a documented tradeoff.

512 held tunnels (single short observations):

Mode Baseline RSS Candidate RSS Threads FDs while held → after close
HTTP/1 20.34 MiB 17.43 MiB 2 in both 1,034 → 10 in both
TLS H2 18.03 MiB 17.42 MiB 2 in both 538 → 10 in both

H2 uses 16 client TCP connections for 512 tunnels; H1 uses 512. These RSS observations exclude kernel socket memory and do not prove long-duration leak freedom or that Rust uses less memory than C.

Evidence and excluded attempt

Corrected raw artifact: https://github.com/Ray0907/tinyproxy-rust/actions/runs/35421889856/artifacts/10577384588

ZIP SHA-256: 4f02b91f635362e71cc540160ca4861000c7f90fee1b8de0b91cdab51ffc681d.

Candidate binary SHA-256: 7ed28cc8a1360e0c3c4c9f849a29f2eef6773ff0c6e7fb013e3666f90564b87e.
Baseline binary SHA-256: 174324da57fb3b05e57c9a20dcbba7436fe3b5eb743f278a7384866d0e478034.

The artifact retains full raw timing/resource observations, errors, configurations, lockfiles, source/compiler identities and binary hashes, but no generated private keys. Retention is 30 days.

Run 35421546117 is excluded: shared Cargo target output incorrectly reused the candidate under both binary labels. Identical hashes exposed the harness bug. The corrected workflow uses isolated target directories, checks baseline identity and matching lockfiles, and refuses identical executable bytes. Do not pool the discarded run with the corrected results.

Scope and limitations

Short closed-loop loopback measurements: 1 second warmup and 2 seconds measured admission per round, including drain time in throughput and complete successful bodies in latency. Proxy and origin run on SMT siblings of a shared CI VM, not physically isolated cores. This compares the combined patch, not individual-change ablations. No WAN, production-SLO, comprehensive security, DoS, long-soak or universal speedup claim.

No merge to main, release, origin connection pool, new configuration/dependencies or TLS interception. Authentication, filtering and existing security boundaries are unchanged. This PR is independently reviewable from the still-open benchmark tooling PR #3.

Keep upstream Nagle policy unchanged, propagate writev through both I/O
wrappers with exact accepted-byte accounting, and move the fixed H2 relay
buffer out of the nested future state. Add socket, partial-write, prefix,
capability, metric and future-size regression tests.
Keep private protocol types private by testing inside the crate. Add a
read-only bounded loopback benchmark comparing the unchanged d500d25
baseline with this candidate, using the pinned independent Go driver.
Retain measured errors, warmup errors, payload checks, CPU/RSS samples,
compiler/source identities and binary hashes. No private keys uploaded.
Replace the manual no-op Wake implementation with Waker::noop so strict
Clippy checks pass. Runtime code and performance settings are unchanged.
The first transport run incorrectly reused the candidate executable as
its baseline via a shared Cargo target directory. Exclude that run from
before/after claims. Build each checkout in its own target directory,
verify pinned baseline and matching lockfiles, reject identical binaries,
and run verified-TLS curl checks for both. Runtime source is unchanged.

Ray0907 commented Sep 19, 2026

Copy link
Copy Markdown
Owner Author

Validation note: the first transport benchmark run (35421546117) completed its workload checks, but must not be used for baseline-versus-candidate performance claims. Its artifact contains identical SHA-256 hashes for both executable labels (7ed28cc8a1360e0c3c4c9f849a29f2eef6773ff0c6e7fb013e3666f90564b87e), and the supposed baseline build finished in 0.07s. The shared Cargo target directory reused the candidate executable.

Commit de6053d12c3259f33125e112866774d8ba60aa8a fixes the harness with completely separate target directories, checks the pinned baseline and matching lockfiles, refuses identical binaries, and runs verified-TLS curl tests for both. The runtime change itself is unchanged. A separate preceding test-only commit replaces the no-op waker helper flagged by Clippy. The corrected run and full CI will be assessed separately; the discarded run is not evidence of a speedup or no regression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant