Repository navigation
Gate FFI declarations and add key validation for optional wolfSSL build configurations - #144
Conversation
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved FFI capability-gating and no-RNG API issues block approval.
Review effort: Lite
Findings: 1
Open (2)
What changed in this PR
This PR adds capability-gated wolfSSL FFI/Python APIs, ECC validation, ChaCha IV hardening, and improved build portability.
Changes:
- Detects optional wolfSSL features and gates declarations and APIs.
- Adds ECC key and digest validation plus DER sizing fixes.
- Expands regression tests and arm64 Linux CI coverage.
| File | Description |
|---|---|
wolfcrypt/random.py |
RNG availability handling |
wolfcrypt/pwdbased.py |
PBKDF2 gating |
wolfcrypt/hashes.py |
Per-variant SHA-3 gating |
wolfcrypt/ciphers.py |
Capability gating, ECC hardening, and ChaCha updates |
wolfcrypt/asn.py |
Independent PEM/DER capability gates |
wolfcrypt/_ffi/lib.pyi |
Feature flags and API declarations |
tests/test_random.py |
RNG regression coverage |
tests/test_pwdbased.py |
PBKDF2 coverage |
tests/test_mlkem.py |
ML-KEM coverage |
tests/test_mldsa.py |
ML-DSA coverage |
tests/test_hkdf.py |
HKDF coverage |
tests/test_hashes.py |
Hash capability coverage |
tests/test_delete_descriptor_binding.py |
Descriptor binding coverage |
tests/test_ciphers.py |
Cipher and key API coverage |
tests/test_chacha_iv.py |
ChaCha IV-state regression coverage |
tests/test_build_ffi.py |
Feature detection and FFI generation coverage |
tests/test_asn.py |
ASN capability coverage |
tests/test_aesgcmstream.py |
AES-GCM streaming coverage |
scripts/build_ffi.py |
Feature detection and conditional FFI generation |
.github/workflows/python-app.yml |
arm64 Linux CI coverage |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
cccacb7 to
08dc337
Compare
There was a problem hiding this comment.
┌───────────────────────────────────────────────────┬───────────────┬───────────────────────────────────┐
│ wolfSSL config │ PR #144 │ master │
├───────────────────────────────────────────────────┼───────────────┼───────────────────────────────────┤
│ bundled (normal pip install) │ 398 passed, │ — │
│ │ 14 skipped │ │
├───────────────────────────────────────────────────┼───────────────┼───────────────────────────────────┤
│ full, CI option list with PQC on │ 360 passed, │ — │
│ │ 14 skipped │ │
├───────────────────────────────────────────────────┼───────────────┼───────────────────────────────────┤
│ nopk: no │ 1 failed, 195 │ import fails: undefined symbol: │
│ RSA/ECC/Ed*/PQC/ASN/pwdbased/HKDF/SHA3/CBC/CTR │ passed │ wc_AesCbcEncrypt │
├───────────────────────────────────────────────────┼───────────────┼───────────────────────────────────┤
│ ecconly: no RSA, SHA3, GCM-stream, HKDF │ 231 passed, │ import fails: wc_AesCtrEncrypt │
│ │ 21 skipped │ │
├───────────────────────────────────────────────────┼───────────────┼───────────────────────────────────┤
│ rsaonly: no ECC, no OAEP, PQC on │ 292 passed, │ import fails: │
│ │ 22 skipped │ wc_RsaPublicEncrypt_ex │
├───────────────────────────────────────────────────┼───────────────┼───────────────────────────────────┤
│ norng: no RNG, RSA verify-only │ 193 passed, │ build fails │
│ │ 44 skipped │ │
├───────────────────────────────────────────────────┼───────────────┼───────────────────────────────────┤
│ norng2: no RNG, full RSA │ 193 passed, │ build fails │
│ │ 44 skipped │ │
└───────────────────────────────────────────────────┴───────────────┴───────────────────────────────────┘
Issues found (none block merging):
- tests/test_asn.py fails on a no-ASN build. In the pem_der_conversion_vectors fixture, return vectors is indented inside if _lib.ASN_ENABLED:. Without ASN the fixture returns None, and test_pem_der_conversion fails with TypeError: 'NoneType' object is not iterable. Master has the same bug, but master couldn't even import without ASN, so this PR is the first time it shows up. The fix is to un-indent that one line.
- Nothing tests RSA verify with no RNG. I checked it by hand on norng2: an RSA signature made with OpenSSL verifies correctly, so the lazy-RNG fix works. But the test suite skips every RSA test when there is no RNG. The test comment "RSA key objects always create a Random" and the RNG_ENABLED condition in the signature_vectors fixture became out of date with the commit "Address review: create RSA RNG on first use".
- With --enable-rsavfy (verify-only RSA), wolfSSL only builds wc_RsaSSL_VerifyInline, so RsaPublic.verify() raises NotImplementedError. The gating is correct. Supporting verify there would mean a separate VerifyInline path, which can be a follow-up.
▎ Tested locally against wolfSSL built with optional features disabled (no-PK/ASN, ECC-only, RSA-only without OAEP, no RNG with verify-only RSA, no RNG with full RSA). Master fails to build or import on all of them; this PR builds and passes. Two small test items:
▎ - tests/test_asn.py: in pem_der_conversion_vectors, return vectors is inside if _lib.ASN_ENABLED:, so a no-ASN build fails test_pem_der_conversion with TypeError: 'NoneType' object is not iterable. Dedenting the return fixes it.
▎ - RSA verify now works without an RNG (checked by hand against an OpenSSL signature with --disable-rng), but no test covers it: the RNG_ENABLED condition and the "RSA key objects always create a Random" comment in signature_vectors predate the lazy RNG change.
|
@dgarske addressed |
dgarske
left a comment
There was a problem hiding this comment.
1. No RNG plus crypto callbacks does not compile
scripts/build_ffi.py:1836 declares the rng member of wc_CryptoInfo unconditionally, but wolfssl/wolfcrypt/cryptocb.h:494 only has it under #ifndef WC_NO_RNG.
wolfcrypt._ffi.c: error: 'wc_CryptoInfo' has no member named 'rng'
Same config without crypto callbacks: 197 passed, 43 skipped. The Nov 2 no-RNG runs had no callbacks, so they could not catch this, and neither CI job combines the two. The fix also needs wolfcrypt/cryptocb.py:189 (info.rng) gated.
2. ECC, Ed25519 or Ed448 with --disable-asn does not import
The DER functions live in asn.c, but the cdef gates them only on the per-algorithm import/export flags, not on features["ASN"]:
- ECC:
scripts/build_ffi.py:1387-1389(wc_EccKeyToDer,wc_EccKeyDerSize,wc_EccPublicKeyToDer).wc_EccKeyDerSizeis new in this PR. - Ed25519:
:1470-1471and:1479-1480. - Ed448:
:1521-1522and:1530-1531.
nm confirms each is undefined in the extension with no definition in libwolfssl.a. The Nov 2 nopk config had ASN and public key algorithms off together, which hides this. The Python methods that call these functions need the same gate. The failing configs also set -DNO_ECC_KEY_IMPORT, so the Decode symbols there are expected to go away with import gating on ECC only; the ECC export trio and all the Ed symbols are independent of it.
3. -DNO_CERTS with key generation does not import
scripts/build_ffi.py:1357 declares wc_RsaKeyToPublicDer under features["KEYGEN"] only. detect_features already treats NO_CERTS as removing the PEM functions, so this path was simply missed. -DNO_CERTS --disable-keygen passes (395 passed, 17 skipped), which isolates it. The symbol is defined nowhere in the library and nothing inside the library references it.
4. Feature detection treats every preprocessor branch as active
defined() at scripts/build_ffi.py:370 matches any #define line, now including indented ones, with no notion of #if, #else or #undef. Master's exact-line match ignored indented lines, so it mostly under-detected, which is the safe direction (a smaller API that still builds). The PR over-detects, which produces exactly the undefined-symbol and compile failures it set out to eliminate.
| Settings file and mode | Feature | PR detects | Preprocessor says |
|---|---|---|---|
IDE/WIN, extension compile (no WOLFSSL_LIB) |
ECC, KEYGEN, SHA384, SHA512, AESGCM_STREAM | 1 | 0 |
IDE/WIN10, FIPS v2 |
MPAPI, WC_RNG_SEED_CB | 1 | 0 |
IDE/WIN10, FIPS v2 |
DES3 | 0 | 1 |
POSIX user_settings.h, #if 0 block with #define NO_RSA |
RSA | 0 | 1 |
POSIX user_settings.h, #define HAVE_ECC then #undef HAVE_ECC |
ECC | 1 | 0 |
The Windows rows are a macro-level simulation with gcc -E -dM on Linux, not a Windows build; those files #error off Windows and preprocessing continues past it. Treat them as strong evidence rather than a reproduced failure. windows/non_fips/user_settings.h, used by the bundled Windows build, is unaffected. There is no Windows job in CI.
Master was also wrong on these files (for example ECC=0 on FIPS), so this is not a pure regression; it changes which way the detection is wrong.
Design point: detect_features now hand-copies roughly 100 lines of settings.h derivation into Python regexes, and findings 1-3 show that model has already drifted. Asking the compiler avoids both problems: preprocess settings.h with -E -dM (or a small probe compile where that is unavailable, such as MSVC) and read the real macro set. I would make this change before merging, because it deletes code from this PR instead of adding to it.
5. ECC private-only keys skip validation entirely
wolfcrypt/ciphers.py:1434 only runs _check_key() when type == ECC_PRIVATEKEY, so an ECPrivateKey DER without the optional public key is never checked. All of these import and sign successfully at this head:
| Private-only DER, P-256 | Result |
|---|---|
d = 0 |
accepted, sign() returns a signature |
d = n |
accepted, sign() returns a signature |
d = n + 1 |
accepted, sign() returns a signature |
d = 2^256 - 1 |
accepted, sign() returns a signature |
The raw path does reject these (d=0 -173, d=n -216), so the two import paths disagree. This is not a regression, but the PR says EccPrivate now validates imported keys, and test_ecc_private_key_without_public_key covers only a valid key. Suggested fix: derive the public key with wc_ecc_make_pub, which returns ECC_PRIV_KEY_E for an out-of-range d (ecc.c:5480), or range-check explicitly. Add d=0 and d>=n tests.
|
dgarske
left a comment
There was a problem hiding this comment.
Re-tested 0f9dd40 against the same 74 wolfSSL configs. Items 1-4 are fixed: no-RNG with crypto callbacks, --disable-asn with ECC/Ed25519/Ed448, and -DNO_CERTS with keygen now build, import and pass, and no config regressed. Agreed on item 5 since the range check is in v5.9.4-stable; please bump the bundled wolfSSL to 5.9.4 here or right after, as anything built against an older USE_LOCAL_WOLFSSL stays exposed.
Still needed:
- CI is red on all four jobs. preprocess() imports setuptools for distutils, but the workflow test steps only install pytest and the package, so 103 tests in test_build_ffi.py fail with ModuleNotFoundError. Either install setuptools for the tests or avoid distutils at test time.
- tests/test_cryptocb.py:322 test_hash_outlives_callback needs @needs_rng; it fails with --disable-rng --enable-cryptocb.
- The probe does not see CFLAGS. distutils adds CPPFLAGS to the preprocessor but CFLAGS only to the compile, so CFLAGS=-DNO_ECC_SIGN still detects HAVE_ECC_SIGN while the extension builds with NO_ECC_SIGN. Running compiler_so with -E would match the real compile.
Earlier items I did not post before, all still reproducible on 0f9dd40:
- A key that fails _check_key() stays loaded. An off-curve import raises -214, but the object keeps the point and shared_secret() on it returns a value. Reset the native key on failure.
- decode_key_raw() accepts a mismatched pair (Q from one key, d from another) and signs with it; the signatures do not verify against the embedded public key. wc_ecc_check_key() only checks d*G == Q under WOLFSSL_VALIDATE_ECC_IMPORT.
- The digest-size allowlist is an API change. verify() now raises ValueError on a wrong-length digest where it returned False, and sign() rejects 16- and 30-byte inputs wolfSSL accepts. I'd keep verify() returning False, and this needs a ChangeLog entry.
- The description says unavailable methods raise NotImplementedError, but they are left off the class, so callers get AttributeError (e.g. EccPrivate.sign with NO_ECC_SIGN). Fix the wording or add stubs.
- Hash-subset builds: --disable-sha fails collection at tests/test_hkdf.py:30 (unconditional HmacSha import); WOLFSSL_NOSHA3_224 fails test_hash_unsupported_type_falls_back, and WOLFSSL_NOSHA3_384 fails five Sha3 cases in test_cryptocb.py with NotImplementedError.
- No CI job builds a reduced config, so none of the new gates are exercised. One or two jobs (e.g. --disable-asn, --disable-rng --enable-cryptocb, -DNO_ECC_SIGN) would keep them from regressing.
- P-256 DER import goes from about 1.5 us to about 245 us (P-521 about 1.6 ms) from wc_ecc_check_key(). Probably fine, but worth a line in the ChangeLog.
- Please squash the "Address review" commits and drop the internal F-xxxx references from the public tests before merge.
dgarske
left a comment
There was a problem hiding this comment.
Tested 1e6b713. Remaining items:
- preprocess() still needs distutils. On Python 3.12+ it only comes from setuptools, so without setuptools the 103 tests in tests/test_build_ffi.py fail with ModuleNotFoundError: No module named 'distutils'. CI only runs 3.10, so it passes there. Taking the compiler from sysconfig.get_config_var("CC") plus CPP/CPPFLAGS/CFLAGS from the environment avoids distutils entirely.
- The probe ignores CFLAGS (scripts/build_ffi.py:390 uses compiler.preprocessor). CFLAGS=-DNO_ECC_SIGN still detects HAVE_ECC_SIGN while the extension compiles with NO_ECC_SIGN.
- A key that fails _check_key() stays loaded: an off-curve decode_key_raw() raises -214, but shared_secret() with that object still returns a value. Reset the native key on failure.
- decode_key_raw() accepts Q from one key and d from another, and signs with the pair; its own signatures then fail to verify.
- verify() raises ValueError on a wrong-length digest instead of returning False, and sign() rejects 16- and 30-byte inputs. This is an API change and needs a ChangeLog entry.
- Unavailable methods are left off the class, so callers get AttributeError, not the NotImplementedError the description states.
- Test failures in reduced builds: tests/test_cryptocb.py:304 test_hash_outlives_callback needs @needs_rng; --disable-sha fails collection at tests/test_hkdf.py:30; WOLFSSL_NOSHA3_224 and WOLFSSL_NOSHA3_384 fail Sha3 cases in tests/test_cryptocb.py.
- No CI job builds a reduced config (e.g. --disable-asn, --disable-rng --enable-cryptocb, -DNO_ECC_SIGN), so the new gates are untested.
- Please squash before merge and drop the internal F-xxxx references from the tests.
1e6b713 to
f1ec757
Compare
dgarske
left a comment
There was a problem hiding this comment.
Re-tested f1ec757. This push is the rebase onto #145 plus the ruff blank-line fix, so all nine items from the Nov 10 review are still open. Re-confirmed on this head:
- Python 3.12 without setuptools: 103 failures in tests/test_build_ffi.py (No module named 'distutils').
- An off-curve decode_key_raw() raises -214, but shared_secret() with the same object still returns a value.
- decode_key_raw() with Q from one key and d from another is accepted, and its own signatures fail to verify.
- sign() still rejects 16- and 30-byte digests, with no ChangeLog entry.
Items 2 and 6-9 are unchanged as listed in that review.
|
@dgarske Addressed |
dgarske
left a comment
There was a problem hiding this comment.
Re-tested 1fc7aff. Thanks, items 1-8 from the Nov 10 review are fixed:
preprocess()no longer needs distutils off Windows. The bundled build passes on Python 3.10 and 3.12 with no setuptools installed.- The probe reads
CC/CPP/CFLAGS/CPPFLAGS. Building the extension withCFLAGS=-DNO_ECC_SIGNagainst a full wolfSSL now drops ECC signing, imports and passes. - A key that fails the check is cleared. After an off-curve
decode_key_raw(),shared_secret()fails instead of using the old key. - A private key whose Q does not match d is rejected with
ECC_PRIV_KEY_E. A matching pair still round-trips. sign()acceptsWC_MIN_DIGEST_SIZEtoWC_MAX_DIGEST_SIZEbytes, andverify()returns False outside that range. ChangeLog entry added.- The reduced builds all build, import and pass:
--disable-asn,--disable-rng --enable-cryptocb,-DNO_ECC_SIGN,--disable-sha,-DWOLFSSL_NOSHA3_224and-DWOLFSSL_NOSHA3_384. The newbuild-reducedCI jobs cover three of them. - The F-xxxx references added by this PR are gone.
Remaining:
- ECC private key import now requires
wc_ecc_make_pub(). wolfSSL builds withoutHAVE_ECC_MAKE_PUBreturnNOT_COMPILED_INfrom it, so_check_pair()fails and every private key import (EccPrivate(der),decode_key_raw(qx, qy, d)) raisesKey check error (-174). That coversNO_ECC_MAKE_PUB,WOLF_CRYPTO_CB_ONLY_ECC, KCAPI, CryptoCell, Silabs SE and ATECC/TA100. On master the same-DNO_ECC_MAKE_PUBbuild imports the key and verifies and exports with it.HAVE_ECC_MAKE_PUBis set insideecc.c, so the preprocessor probe cannot see it. TreatingNOT_COMPILED_INfromwc_ecc_make_pub()as "pair check unavailable" and keeping thewc_ecc_check_key()result would avoid the regression. _check_pair()uses_lib.ECC_PRIV_KEY_E, but that constant is only declared inside theERROR_STRINGSblock of the cdef. With--disable-errorstrings, a mismatched pair raisesAttributeError: ... no function, constant or global variable named 'ECC_PRIV_KEY_E'. The_fail()cleanup is skipped, so the key stays loaded, andtest_ecc_private_import_rejects_mismatched_pairfails. DeclaringECC_PRIV_KEY_Eoutside that block fixes it.- The PR description still says unavailable methods raise
NotImplementedError. They are left off the class, as the ChangeLog now says. Please update the description to match. - Nit: the import timing figure in the ChangeLog (0.3 to 0.8 ms) depends on the machine and will go stale. I would drop it.
- Please squash before merge (37 commits).
|
@dgarske Thanks for re-testing. Fixes for items 1, 2 and 4 are in:
|
Build and import the extension against wolfSSL builds that omit optional features. - Detect features with the C preprocessor, honoring CC/CPP/CFLAGS/CPPFLAGS. setuptools is no longer needed to preprocess the headers. - Declare FFI functions and expose Python methods only when wolfSSL builds them: AES-CTR, AES-CBC, streaming AES-GCM decrypt, SHA-3 per size, HKDF, PBKDF2, PEM/DER conversion, PKCS#8, RNG, RSA, ECC, Ed25519, Ed448, ML-KEM and ML-DSA. Key DER functions are gated on ASN and wc_CryptoInfo.rng on RNG support. - Validate imported ECC keys with wc_ecc_check_key() and check that a private key matches its public point. Skip the pair check when wc_ecc_make_pub() is not compiled in. Clear a key that fails the check. - Size the ECC private key DER buffer with wc_EccKeyDerSize(). - Check the digest size in ECDSA sign(), verify(), sign_raw() and verify_raw(). - Disarm ChaCha when set_iv() rejects the nonce length. - Create the RSA RNG on first use. - Build the bundled wolfSSL with -fPIC on every Linux platform. - CI: run on arm64 Linux and add reduced wolfSSL build jobs. Fenrir: F-8277, F-8279, F-8280, F-8281, F-10069, F-10070, F-10071, F-11250, F-12224, F-12225, F-12226, F-12227, F-12228, F-12229, F-12230, F-12231, F-12232, F-12233, F-12234, F-13020, F-13021, F-13023
bd85622 to
59a6edc
Compare
dgarske
left a comment
There was a problem hiding this comment.
Re-tested 59a6edc. Thanks, items 1-5 from the Nov 17 review are resolved:
- With
-DNO_ECC_MAKE_PUB, private key import works again and the mismatched-pair test skips. The tests that still fail in that build are sign and keygen tests, which returnNOT_COMPILED_INfrom wolfSSL itself. Master fails the same way with that wolfSSL build, so this is not a regression. --disable-errorstringsbuilds, imports and passes, and the new CI job covers it.- The description and ChangeLog are updated, and the branch is squashed.
- Also passing: default,
--disable-asn,--disable-rng --enable-cryptocb,-DNO_ECC_SIGNand--disable-sha.
One remaining item, low severity:
_check_pair()raisesMemoryErrordirectly whenwc_ecc_new_point()returns NULL._check_key()does not catch it, so_fail()never runs and the imported key stays loaded. Every other validation failure clears the key. I reproduced this by makingwc_ecc_new_pointreturn NULL:decode_key_raw(qx, qy, d)raisesMemoryError, andpriv.sizeis still 32 afterwards. Raising throughself._fail(MemoryError(...))keeps it consistent.
Nit: test_ecc_rejected_import_clears_key uses EccPrivate.make_key(), so it fails on builds without HAVE_ECC_MAKE_PUB, where wolfSSL can't generate keys. Building the key from the test vectors would let it run there too. This is optional, since CI has no such build.
_check_pair() raised MemoryError directly, so _fail() never ran and the rejected key stayed loaded.
make_key() fails on builds without HAVE_ECC_MAKE_PUB.


This PR makes the extension buildable and importable against wolfSSL builds
that omit optional features, instead of failing at import time with an
undefined symbol or at compile time.
build_ffi.pyis split intodetect_features(),make_source(),make_cdef()anddefault_features(), withmain()running only wheninvoked directly or via cffi's setuptools hook. Feature detection now uses
a
defined()helper so indentation, values, trailing comments andlast-line macros are all matched correctly, including for
WC_RSA_BLINDING,ECC_TIMING_RESISTANTand the ML-DSA no-context macros.declarations and Python methods are only exposed when wolfSSL actually
builds them: AES-CTR, AES-CBC (encrypt/decrypt), streaming AES-GCM
decryption, SHA-3 (per size), HKDF, PBKDF2, PEM-to-DER/DER-to-PEM,
PKCS#8, RNG, RSA (encrypt/private/sign/verify/OAEP), ECC
(sign/verify/DHE/key import/export), Ed25519, Ed448 and ML-KEM/ML-DSA
operations.
EccPublicandEccPrivatenow validate imported keys withwc_ecc_check_key(), rejecting points that are off-curve, out ofrange, or of the wrong order.
EccPrivate.encode_key()sizes its DER buffer withwc_EccKeyDerSizeinstead of a fixed multiple of the field size, fixing
BAD_FUNC_ARGoncurves like P-192.
sign(),verify(),sign_raw()andverify_raw()now check that theinput is a valid digest size before calling into wolfSSL.
set_iv()now clears the IV-set state before validating thenonce length, so a rejected nonce blocks subsequent
encrypt()/decrypt()calls instead of leaving the previous IV in effect.
-fPICon every Linuxplatform, not just x86, and CI now also runs on arm64 Linux.