Skip to content

Tidy up wfdb_to_mat and its tests (#642 follow-ups) - #653

Open
goktugozkanmd wants to merge 1 commit into
MIT-LCP:mainfrom
goktugozkanmd:tidy/wfdb-642
Open

goktugozkanmd wants to merge 1 commit into
MIT-LCP:mainfrom
goktugozkanmd:tidy/wfdb-642

Conversation

@goktugozkanmd

Copy link
Copy Markdown
Contributor

Follow-ups from the review of #633, tracked in #642. No behaviour change.

  • Drop the no-op record.n_sig = num_channels assignment and the num_channels alias, and read record.sig_len directly instead of copying it into desired_length.
  • Share the WFDB checksum through _signal._checksum, so the header checksum in wfdb_to_mat and calc_checksum cannot drift apart. _checksum is the same expression the inline code used (int(np.sum(channel) % 65536)), so written headers are unchanged.
  • Split test_missing_samples_and_skew_preserved into two tests, so a failure in the first part no longer hides the skew check.
  • Note in test_forward_slash_path that it only guards Windows paths.

Closes #642.

The range check from the same review (#641) is already in main via #650, so it is not repeated here.

Tests: pytest -q → 397 passed, 3 skipped (Windows, CPython 3.12); ruff check src tests clean. The 3 skips are the POSIX-permission and network tests that skip on this machine regardless of the patch.

Follow-ups from the review of MIT-LCP#633, tracked in MIT-LCP#642. No behaviour change:
the checksum helper is the same expression the inline code used.

- Drop the no-op `record.n_sig = num_channels` assignment and the
  `num_channels` alias, and read `record.sig_len` directly instead of
  copying it into `desired_length`.
- Share the WFDB checksum through `_signal._checksum`, so the header
  checksum in `wfdb_to_mat` and `calc_checksum` cannot drift apart.
- Split `test_missing_samples_and_skew_preserved` into two tests, so a
  failure in the first part no longer hides the skew check.
- Note in `test_forward_slash_path` that it only guards Windows paths.

Closes MIT-LCP#642.

Tests: 397 passed, 3 skipped (`pytest -q`, Windows, CPython 3.12);
`ruff check src tests` clean. The 3 skips are the POSIX-permission and
network-dependent tests that skip on this machine regardless of the patch.

@tompollard tompollard left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking these up. The tidy-ups read well, and I checked the "no behaviour change" claim rather than just reading the diff:

  • pytest -q on macOS (numpy 2.5.4): 400 passed, including the 3 tests that skip on Windows. ruff check src tests is clean.
  • I compared the old and new checksum expressions on 14 arrays (int8 to int64, minimum and maximum values, sums that wrap past 65536, negatives, and an empty signal): identical results, so written headers are unchanged.

Splitting the combined missing-samples/skew test, and dropping num_channels and desired_length, both make this clearer. A few things inline, plus two that are not on changed lines:

  • The checksum is still computed in two other places. src/wfdb/io/convert/edf.py:429 and src/wfdb/io/convert/wav.py:317 both have their own int(np.sum(v) % 65536) for v in np.transpose(sig_data), and src/wfdb/io/record.py:936 uses a bare 65536 when comparing checksums. So the description's "cannot drift apart" only holds for wfdb_to_mat and calc_checksum so far. Either route those through the shared helper (see the inline suggestion on calc_checksum, which fits both converters since their sig_data also has one column per channel), or narrow the claim in the description.
  • The round-trip checksum asserts are now partly circular. tests/io/test_convert.py:493, :541 and :560 assert reloaded.checksum == reloaded_d.calc_checksum(), and after this change both sides go through _checksum, so a bug in _checksum itself would still pass them. The hard-coded expectations at lines 198 and 278 still catch that, so nothing is unprotected. A small direct test of the helper (wrap-around, negative sums, an empty channel) would make its contract explicit. Fine as a follow-up.

None of this blocks merging. The suggestion on calc_checksum is the one I would most like to see here, since it also covers the first point above.

Comment thread src/wfdb/io/_signal.py
else:
cs = np.sum(self.d_signal, 0) % 65536
cs = [int(c) for c in cs]
cs = [_checksum(s) for s in self.d_signal.T]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two side effects of switching from np.sum(self.d_signal, 0) to a loop over self.d_signal.T:

  1. Speed depends a lot on the number of channels. Each column of the C-ordered d_signal is a strided view, so summing column by column reads memory differently from one axis=0 reduction. Timings of the old and new expressions (numpy 2.5.4, best of 5):

    d_signal before after
    10M x 12, int16 74 ms 40 ms
    10M x 2, int32 39 ms 6 ms
    1M x 64, int16 24 ms 137 ms

    Faster for typical ECG records, but about 5.6x slower for 64-channel recordings such as EEG. The absolute cost is small next to writing the files, so this does not block anything.

  2. A 1-D d_signal is now silently wrong instead of an error. Before, calc_checksum() raised TypeError: 'numpy.int64' object is not iterable. Now the loop runs over samples, and returns one "checksum" per sample ([0, 1, 2, ...] for np.arange(10)). Normal use does not reach this, because d_signal.shape[1] is read earlier (line 508) and wrsamp reshapes 1-D input, so only a hand-built Record calling calc_checksum() directly hits it.

A helper for whole signals would keep the vectorised reduction and avoid both:

def _checksums(signal):
    """Return the WFDB checksum of each channel (column) of a 2-D array of digital samples."""
    return [int(c) for c in np.sum(signal, axis=0) % CHECKSUM_MODULUS]

used here (cs = _checksums(self.d_signal)), in wfdb_to_mat, and in edf.py:429 and wav.py:317, keeping _checksum for the expanded branch, where the channels of e_d_signal can differ in length.

Comment thread src/wfdb/io/_signal.py
CHECKSUM_MODULUS = 65536


def _checksum(values):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to have the checksum in one place. As mentioned in the review summary, edf.py:429 and wav.py:317 still compute the same expression inline, and record.py:936 uses a bare 65536, so for the checksums to really be unable to drift apart those would need to use this helper (or the _checksums variant suggested on calc_checksum) and CHECKSUM_MODULUS too.

Comment thread src/wfdb/io/_signal.py
MAX_I32 = 2147483647
MIN_I32 = -2147483648

CHECKSUM_MODULUS = 65536

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: the new function now sits between MIN_I32 and the format constants below (ALIGNED_FMTS and so on). Keeping CHECKSUM_MODULUS with the other constants and moving _checksum after them would keep the constants together.

Comment thread tests/io/test_convert.py
Comment on lines +677 to +678
# Only meaningful on Windows: elsewhere the old code handled this
# path already, so keep the test to guard the basename handling.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads as both "the test is pointless here" and "keep it". Maybe:

Suggested change
# Only meaningful on Windows: elsewhere the old code handled this
# path already, so keep the test to guard the basename handling.
# The bug this guards against only appeared on Windows; on other
# platforms this checks that the basename is still used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants