Repository navigation
perf(web): parse each sync response body once - #2069
Conversation
With retry handlers set (the default), the sync client parsed a JSON response body for the retry handlers and then parsed it again in _urllib_api_call and _request_for_pagination. Keep the parsed body on the internal response dict and reuse it. Co-Authored-By: Claude <svc-devxp-claude@slack-corp.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2069 +/- ##
=======================================
Coverage 84.71% 84.71%
=======================================
Files 118 118
Lines 14119 14122 +3
=======================================
+ Hits 11961 11964 +3
Misses 2158 2158 ☔ View full report in Codecov by Harness. |
mwbrooks
left a comment
There was a problem hiding this comment.
✅ Thanks @WilliamBergamin! Halving the JSON parsing on big history pages is a nice free win for every sync client.
| headers = transport.call_args[1]["args"]["headers"] | ||
| self.assertEqual("application/json;charset=utf-8", headers["Content-Type"]) | ||
|
|
||
| def test_api_calls_parse_the_response_body_once(self): |
There was a problem hiding this comment.
praise: Love that these call-count tests fail on main and pass here, it makes the perf claim easy to trust.
cchensh
left a comment
There was a problem hiding this comment.
LGTM. Each retry gets a fresh resp, so the cached parsed_body always comes from the final attempt, and the non-JSON, binary and error paths are unchanged. A few optional nits:
RetryHttpResponse.bodyandSlackResponse.dataare now the same dict, so a custom retry handler that changesresponse.bodyincan_retryand then declines to retry now changes what the caller sees. The docs don't suggest mutatingbody, but a short comment (or a copy) would make the sharing explicit.- Test gap: there's no test where a handler forces a second attempt with a different JSON body, checking that
resp.datacomes from the final attempt. That's the main correctness claim. Aretry_handlers=[]case would be good too. test_non_json_response_body_raises_api_errornever hits the cache, since the body doesn't start with{. That's fine as a regression guard.- The
Returns:docstring of_perform_urllib_http_requestshould mention the optionalparsed_body.
Summary
This pull request makes the sync
WebClientparse each response body once instead of twice._perform_urllib_http_requestparses a JSON body to build theRetryHttpResponse, then_urllib_api_calland_request_for_paginationparse the same string again.json.loadsper call and per page: about 2.3 ms on an ~830 KBconversations.historypage (Python 3.15), next to nothing on small bodies.RetryHttpResponse.bodyandSlackResponse.datanow share one dict, as they already do in the async client. No built-in handler reads or editsbody; a custom handler that edits it now changes the returnedSlackResponse.Testing
The two call-count tests fail on
main(2 parses per response) and pass on this branch:api_callwith the default retry handlers parses the body once.SlackApiErrorwith the "non-JSON format" error (passes before and after).Category
/docs(Documents)/tutorial(PythOnBoardingBot tutorial)tests/integration_tests(Automated tests for this library)Requirements
python3 -m venv .venv && source .venv/bin/activate && ./scripts/run_validation.shafter making the changes.🤖 Generated with Claude Code