Repository navigation
Conversation
ace6085 to
ff58721
Compare
|
Some Linux builds keep failing as the installation of dependencies runs into time-outs. I'll push another empty commit later to restart the build. |
ff58721 to
134e5a6
Compare
|
Thanks for the quick merge of #437! I've rebased on dev. But the theory of constraints wins again. Now that the Linux builds are successful, it's the MacOS builds that fail due to a shortage of runners... I'll push another empty commit later. |
134e5a6 to
301d056
Compare
|
And it's green! 🥬 |
24622fa to
80f0378
Compare
80f0378 to
7d115cd
Compare
Reading RLE-compressed sas7bdat files failed with "Row #0 decompressed to -1 bytes" when a row held a non-repeating run longer than 8255 bytes. SAS-generated files show that commands 0x0X-0x3X form a single copy command with a 14-bit length (64..16447 bytes); ReadStat decoded 0x2X as a short copy and ignored 0x3X. - Decode commands 0-3 with the shared 14-bit length and declare their length byte for the input bounds check. - Return -1 for unknown commands instead of skipping them. - Let the compressor emit copies of up to 16447 bytes accordingly. Add a round-trip test with long literal runs (a 0x2X copy and a split 0x3F 0xFF run), which fails against the previous reader.
7d115cd to
3f936f2
Compare
|
With the latest empty push, the build is now successful again. The reliability of the Windows build has much improved with #426 and #439, and there is a meaningful speed-up in the Linux builds with #437. But I think the non-availability of ARM MacOS runners in the European late afternoon/evening is a problem that we won't be able to solve on a project level. If there are concerns or questions about this PR or #434, please let me know. I'd be more than happy to implement requested changes or to generate test data with SAS. |
Resolves #307.
Problem
Reading RLE-compressed (
COMPRESS=CHAR) sas7bdat files failed withRow #0 decompressed to -1 bytes (expected N bytes)when a row contained a non-repeating stretch longer than 8255 bytes, e.g. a long string of repeated"test". Repeated single characters were unaffected because they compress into insert commands instead of literal copies.Root cause
SAS uses RLE commands 0x0X–0x3X as one copy command with a 14-bit length (up to 16447 bytes). ReadStat only decoded 0x0X and 0x1X this way, so longer copies broke the decoding of the whole row.
Changes
Validation