Repository navigation
Conversation
…than 2 GiB uploadLargeParts computed the start offset of each chunk as bufferSize * partNumber with two ints and widened the result to long afterwards. Once the offset passes 2^31 bytes the multiplication overflows, the Content-Range header gets a negative start offset and the upload fails with "Invalid Content-Range header". Widen to long before multiplying and add an offline test that uploads 2.3 GB through uploadLarge and checks the Content-Range of every chunk.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brief Summary of Changes
Uploader.uploadLargePartscomputed the start offset of each chunk asbufferSize * partNumberwith twoints and widened the result tolongonly afterwards. Once the offset passes 2^31 bytes, the multiplication overflows, theContent-Rangeheader gets a negative start offset (for examplebytes -2134900736--2113929217/...with 20 MiB chunks) and the upload fails withInvalid Content-Range header. Details and a standalone reproduction are in #394.The fix widens to
longbefore multiplying:The new test
UploaderUploadLargeTestruns offline incloudinary-core: it plugs a recordingAbstractUploaderStrategyintoUploader, uploads 2.3 GB from anInputStreamthat reports bytes without allocating them, and checks theContent-Rangeof every 20 MiB chunk. It takes well under a second.What does this PR address?
Fixes #394
Are tests included?
Reviewer, please note:
Content-Range of chunk 103 should start with 'bytes 2160066560-' but was 'bytes -2134900736--2113929217/-1'.cloudinary-coretests pass, and thecloudinary-core,cloudinary-http5andcloudinary-taglibjars build. Thecloudinary-http5integration tests need a test environment and were not run locally, they run in CI.long) was verified against a real product environment with a 2.26 GB video and 20 MiB chunks: the unpatched SDK fails withInvalid Content-Range header, the file uploads completely withlongoffsets.Checklist: