Repository navigation
Tidy up wfdb_to_mat and its tests (#642 follow-ups) - #653
goktugozkanmd wants to merge 1 commit into
Conversation
Follow-ups from the review of MIT-LCP#633, tracked in MIT-LCP#642. No behaviour change: the checksum helper is the same expression the inline code used. - Drop the no-op `record.n_sig = num_channels` assignment and the `num_channels` alias, and read `record.sig_len` directly instead of copying it into `desired_length`. - Share the WFDB checksum through `_signal._checksum`, so the header checksum in `wfdb_to_mat` and `calc_checksum` cannot drift apart. - Split `test_missing_samples_and_skew_preserved` into two tests, so a failure in the first part no longer hides the skew check. - Note in `test_forward_slash_path` that it only guards Windows paths. Closes MIT-LCP#642. Tests: 397 passed, 3 skipped (`pytest -q`, Windows, CPython 3.12); `ruff check src tests` clean. The 3 skips are the POSIX-permission and network-dependent tests that skip on this machine regardless of the patch.
tompollard
left a comment
There was a problem hiding this comment.
Thanks for picking these up. The tidy-ups read well, and I checked the "no behaviour change" claim rather than just reading the diff:
pytest -qon macOS (numpy 2.5.4): 400 passed, including the 3 tests that skip on Windows.ruff check src testsis clean.- I compared the old and new checksum expressions on 14 arrays (int8 to int64, minimum and maximum values, sums that wrap past 65536, negatives, and an empty signal): identical results, so written headers are unchanged.
Splitting the combined missing-samples/skew test, and dropping num_channels and desired_length, both make this clearer. A few things inline, plus two that are not on changed lines:
- The checksum is still computed in two other places.
src/wfdb/io/convert/edf.py:429andsrc/wfdb/io/convert/wav.py:317both have their ownint(np.sum(v) % 65536) for v in np.transpose(sig_data), andsrc/wfdb/io/record.py:936uses a bare65536when comparing checksums. So the description's "cannot drift apart" only holds forwfdb_to_matandcalc_checksumso far. Either route those through the shared helper (see the inline suggestion oncalc_checksum, which fits both converters since theirsig_dataalso has one column per channel), or narrow the claim in the description. - The round-trip checksum asserts are now partly circular.
tests/io/test_convert.py:493,:541and:560assertreloaded.checksum == reloaded_d.calc_checksum(), and after this change both sides go through_checksum, so a bug in_checksumitself would still pass them. The hard-coded expectations at lines 198 and 278 still catch that, so nothing is unprotected. A small direct test of the helper (wrap-around, negative sums, an empty channel) would make its contract explicit. Fine as a follow-up.
None of this blocks merging. The suggestion on calc_checksum is the one I would most like to see here, since it also covers the first point above.
| else: | ||
| cs = np.sum(self.d_signal, 0) % 65536 | ||
| cs = [int(c) for c in cs] | ||
| cs = [_checksum(s) for s in self.d_signal.T] |
There was a problem hiding this comment.
Two side effects of switching from np.sum(self.d_signal, 0) to a loop over self.d_signal.T:
-
Speed depends a lot on the number of channels. Each column of the C-ordered
d_signalis a strided view, so summing column by column reads memory differently from oneaxis=0reduction. Timings of the old and new expressions (numpy 2.5.4, best of 5):d_signalbefore after 10M x 12, int16 74 ms 40 ms 10M x 2, int32 39 ms 6 ms 1M x 64, int16 24 ms 137 ms Faster for typical ECG records, but about 5.6x slower for 64-channel recordings such as EEG. The absolute cost is small next to writing the files, so this does not block anything.
-
A 1-D
d_signalis now silently wrong instead of an error. Before,calc_checksum()raisedTypeError: 'numpy.int64' object is not iterable. Now the loop runs over samples, and returns one "checksum" per sample ([0, 1, 2, ...]fornp.arange(10)). Normal use does not reach this, becaused_signal.shape[1]is read earlier (line 508) andwrsampreshapes 1-D input, so only a hand-builtRecordcallingcalc_checksum()directly hits it.
A helper for whole signals would keep the vectorised reduction and avoid both:
def _checksums(signal):
"""Return the WFDB checksum of each channel (column) of a 2-D array of digital samples."""
return [int(c) for c in np.sum(signal, axis=0) % CHECKSUM_MODULUS]used here (cs = _checksums(self.d_signal)), in wfdb_to_mat, and in edf.py:429 and wav.py:317, keeping _checksum for the expanded branch, where the channels of e_d_signal can differ in length.
| CHECKSUM_MODULUS = 65536 | ||
|
|
||
|
|
||
| def _checksum(values): |
There was a problem hiding this comment.
Nice to have the checksum in one place. As mentioned in the review summary, edf.py:429 and wav.py:317 still compute the same expression inline, and record.py:936 uses a bare 65536, so for the checksums to really be unable to drift apart those would need to use this helper (or the _checksums variant suggested on calc_checksum) and CHECKSUM_MODULUS too.
| MAX_I32 = 2147483647 | ||
| MIN_I32 = -2147483648 | ||
|
|
||
| CHECKSUM_MODULUS = 65536 |
There was a problem hiding this comment.
Minor: the new function now sits between MIN_I32 and the format constants below (ALIGNED_FMTS and so on). Keeping CHECKSUM_MODULUS with the other constants and moving _checksum after them would keep the constants together.
| # Only meaningful on Windows: elsewhere the old code handled this | ||
| # path already, so keep the test to guard the basename handling. |
There was a problem hiding this comment.
This reads as both "the test is pointless here" and "keep it". Maybe:
| # Only meaningful on Windows: elsewhere the old code handled this | |
| # path already, so keep the test to guard the basename handling. | |
| # The bug this guards against only appeared on Windows; on other | |
| # platforms this checks that the basename is still used. |
Follow-ups from the review of #633, tracked in #642. No behaviour change.
record.n_sig = num_channelsassignment and thenum_channelsalias, and readrecord.sig_lendirectly instead of copying it intodesired_length._signal._checksum, so the header checksum inwfdb_to_matandcalc_checksumcannot drift apart._checksumis the same expression the inline code used (int(np.sum(channel) % 65536)), so written headers are unchanged.test_missing_samples_and_skew_preservedinto two tests, so a failure in the first part no longer hides the skew check.test_forward_slash_paththat it only guards Windows paths.Closes #642.The range check from the same review (#641) is already in
mainvia #650, so it is not repeated here.Tests:
pytest -q→ 397 passed, 3 skipped (Windows, CPython 3.12);ruff check src testsclean. The 3 skips are the POSIX-permission and network tests that skip on this machine regardless of the patch.