Repository navigation
test: add conformance tests over apache/iceberg-verification fixtures - #1144
Merged
Merged
Conversation
This was referenced Nov 19, 2026
WZhuo
force-pushed
the
feat/iceberg-verification-phase1
branch
from
November 19, 2026 06:11
f7878fe to
276925e
Compare
wgtmac
approved these changes
Nov 20, 2026
wgtmac
left a comment
Member
There was a problem hiding this comment.
Generally looks good! Please rebase to resolve ci failure.
Pin apache/iceberg-verification as a git submodule at testing/iceberg-verification and add verification_test, which runs the schema and types surfaces of its table-spec fixtures through SchemaFromJson and TypeFromJson. The test is built only with -DICEBERG_BUILD_VERIFICATION_TESTS=ON (off by default) and is not run in CI yet. With the option on, CMake fails at configure time if the fixtures are missing. Every case that diverges from its fixture fails; advisory (normative_level: should) cases skip. Four cases currently fail on implementation gaps: - schema unknown/reject-unknown-required: SchemaFromJson accepts a required field of type unknown. - types primitive/decimal-9-2 and decimal-9-2-spaced: ToJson writes decimal(P,S) without the space the fixtures expect. - types primitive/decimal-precision-over-38: TypeFromJson throws from the DecimalType constructor instead of returning an error.
WZhuo
force-pushed
the
feat/iceberg-verification-phase1
branch
from
November 20, 2026 03:52
276925e to
912ccb9
Compare
Member
|
@manuzhang @zhjwpku Do you want to take a look? |
manuzhang
reviewed
Nov 20, 2026
manuzhang
reviewed
Nov 20, 2026
manuzhang
reviewed
Nov 20, 2026
manuzhang
reviewed
Nov 20, 2026
manuzhang
reviewed
Nov 20, 2026
A `cases` entry that is not an object, or has no string `id`, made LoadCases throw while GTest registered the parameterized tests, which aborted the binary without naming the fixture. Report each such entry as a load error labeled `cases-<index>`, so every load error gets a unique test name. An empty `cases` array still loads as no cases. Move opening, parsing and checking a fixture file into ParseFixture, which returns its `cases` array, and chain the object, array and scalar branches of DiffJson with else-if.
Member
|
Thanks @WZhuo for adding iceberg-verification fixtures and @wgtmac and @manuzhang for the review! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 1 of #909: pin apache/iceberg-verification as a git submodule at
testing/iceberg-verification, and addverification_test, which runs thetypesandschemasurfaces of its table-spec fixtures.Each fixture case becomes one GTest parameterized test named after
<subdir>/<case-id>. The types cases go throughTypeFromJsonand the schema cases throughSchemaFromJson. Each result is then projected into the fixture'sdecodedshape through public accessors and compared field by field. Every difference is reported with its path, for examplefields[2].parent. The subscribed subdirectories aretypes/{primitive,nested,variant,geospatial}andschema/{core,unknown}.The test is behind a new
ICEBERG_BUILD_VERIFICATION_TESTSoption, off by default. To run it:git submodule update --init testing/iceberg-verification cmake -S . -B build -DICEBERG_BUILD_VERIFICATION_TESTS=ON cmake --build build --target verification_test ./build/src/iceberg/test/verification_testICEBERG_VERIFICATION_DIRpoints the test at another checkout of the fixtures.Two things differ from the plan in #909:
normative_level: should) are reported as skips. The test is not added to CI until all cases pass.Currently 77 cases pass and 4 fail:
schema/unknown/reject-unknown-requiredtypes/primitive/decimal-9-2,types/primitive/decimal-9-2-spacedtypes/primitive/decimal-precision-over-38Validation: built and ran
verification_testlocally on macOS with the results above. Configuring with a nonexistentICEBERG_VERIFICATION_DIRfails as expected. pre-commit passes.