Skip to content

Network Extension improvements adding VPC PGs, VPC static routes and PG ACLs. - #14734

Open
kiwiflyer wants to merge 1 commit into
apache:mainfrom
kiwiflyer:ovn-support
Open

kiwiflyer wants to merge 1 commit into
apache:mainfrom
kiwiflyer:ovn-support

Conversation

@kiwiflyer

Copy link
Copy Markdown
Contributor

Description

This PR lets a NetworkOrchestrator network extension implement VPC private gateways,
VPC static routes and private gateway ACLs.

NetworkExtensionElement implements every other VpcProvider method by building a
JSON payload and handing it to the extension script, but four of them throw
UnsupportedOperationException:

createPrivateGateway()      -> "Private gateways are not supported by the network extension element."
deletePrivateGateway()      -> "Private gateways are not supported by the network extension element."
applyStaticRoutes()         -> "Static routes are not supported by the network extension element."
applyACLItemsToPrivateGw()  -> "ACL items on private gateways are not supported by the network extension element."

The restriction is at the framework level, so it applies to every network extension
whatever its backend can do. An extension-backed VPC cannot reach parity with the VPC
virtual router today, and no extension author can fix that on their own side.

Functional change. The four methods are wired up the same way as their
neighbours, adding four commands to the script protocol:

Command Sent when
create-private-gateway a private gateway is created on an extension-backed VPC
delete-private-gateway that gateway is removed
apply-static-routes the VPC's static routes change
apply-private-gw-acl a network ACL is applied to a private gateway

An extension that does not implement these verbs is unaffected: the commands are new,
so nothing dispatches them to a script that has not been updated, and the existing
behaviour of every other command is unchanged. Before this PR these operations failed
with UnsupportedOperationException regardless of the script, so there is no case
where previously-working behaviour changes.

Private gateway operations resolve the owning VPC and go through the existing
VPC-level script path, carrying the gateway's id, uuid, address, netmask, broadcast
URI and derived VLAN alongside the usual VPC fields. applyStaticRoutes and
applyACLItemsToPrivateGw are full rebuilds that send the complete non-revoked set on
every call, matching applyFWRules and applyNetworkACLs rather than the per-object
style of add-static-nat; the backend reconciles instead of tracking deltas.

One behavioural fix is included. addPrivateGatewayToPayload documents
gateway_vlan as conditional but emitted it unconditionally whenever a broadcast URI
was present. BroadcastDomainType.getValue() returns null for a URI whose scheme it
does not recognise, so a non-VLAN gateway shipped "gateway_vlan": "" to the script.
It is now emitted only when it has a value, which is what the javadoc already claimed
and what every neighbouring field does.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

n/a

How Has This Been Tested?

Unit tests. NetworkExtensionElementTest is new — this 2600-line class had no unit
test at all before. 26 cases covering the four methods: the payload fields each one
sends, the VPC and gateway identity the script needs to find the router, the
omit-when-blank branch of every optional field, both failure styles (createPrivateGateway
and deletePrivateGateway return false, applyStaticRoutes and applyACLItemsToPrivateGw
throw ResourceUnavailableException), full serialisation of a static route and of an ACL
rule, rule ordering by number and routes by CIDR, and a null CIDR passing through the
route comparator.

NetworkExtensionElementPayloadContractTest adds 6 more. It asserts the keys the
script cannot run without — a transcription of the reference implementation's
require_arg calls, with the source named in each test — and writes the payloads this
class actually produces to target/extension-payloads/.

The whole framework/extensions module: 377 tests, 0 failures, checkstyle clean.

Against a real OVN. The payloads captured above were replayed against a live OVN
24.03 northbound database with ovn-northd running, using a reference extension that
implements the four commands. 21 assertions: the element's create-private-gateway
payload produces a router port carrying its address, its apply-static-routes payload
installs routes that northd resolves into logical flows, its apply-network-acl
payload produces ACLs with rule 10 ahead of rule 20 at the right OVN priority and the
source CIDRs intact, its apply-private-gw-acl payload builds the gateway ACLs, and
its delete-private-gateway payload removes the port. implement-vpc is included,
captured in the state a VPC is first implemented in — no source NAT IP yet.

That replay exists because the unit tests assert what this class writes and an
extension's tests assert what it reads, with each side checked against a fixture
written by the same hand. Both suites would still pass if a key name drifted between
them. Feeding the real payloads to a real backend is what closes that.

Not tested: this has not run on a CloudStack management server driving a real zone.
The reference extension is not part of this PR.

How did you try to break this feature and the system with this change?

Adversarial cases, all now regression tests:

  • No VPC to resolve. A gateway with a null vpcId, and a gateway whose vpcId
    points at a VPC that no longer exists. Both must return false and must not reach the
    script. Verified with verify(..., never()) rather than by return value alone.
  • Blank and whitespace-only fields. A gateway with "", null and " " for its
    address, gateway, netmask and broadcast URI, and a null state. Blank is not the same
    as absent: a script testing -n "$gateway_ip_address" must not see an empty string
    and configure a port with it. Asserts the keys are missing, and that the mandatory
    ones survive.
  • Malformed broadcast URI. vlan://bad host throws URISyntaxException inside
    addPrivateGatewayToPayload. The call must not abort and the raw value must still go
    out; only the derived VLAN is dropped. A schemeless URI (untagged) is the case that
    exposed the empty-string bug above.
  • Null and empty collections. Null route and rule lists must send an empty array,
    not omit the key — an empty array means "no routes", a missing key means "unchanged",
    and the two are not interchangeable to a reconciling backend.
  • Half-built objects. A static route with a null CIDR, to prove the sort comparator
    cannot NPE on a route that is mid-construction.
  • Backend failure in both styles, since the asymmetry is inherited from
    VpcProvider and is easy to "fix" wrongly in either direction.
  • Mutation testing of the fix. Reverting the one-line gateway_vlan change makes
    testPrivateGatewayPayloadOmitsVlanWhenBroadcastUriCarriesNone fail, confirming the
    test is not vacuous.

Two things found while writing these, worth flagging to reviewers:

  • Mockito returns 0, not null, for an unstubbed boxed Integer. The first version
    of the ACL serialisation test passed for the wrong reason — the omit-when-null branch
    looked taken when it was not. The test now stubs the nulls explicitly and says why.
  • The command name constants are asserted against their string literals. They are the
    wire contract with every deployed script, so a rename should break a test here rather
    than break deployments silently.

Blast radius. The four methods previously threw unconditionally, so no existing
deployment can be executing them. addPrivateGatewayToPayload, buildStaticRoutesArray
and getVpcForPrivateGateway are new and private. The only pre-existing code path
touched is the gateway_vlan emission described above, which is reachable only from
the new methods. buildAclRulesArray is reused unchanged and is still covered by the
existing apply-network-acl path.

@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 79.13043% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 20.83%. Comparing base (6a0fcd6) to head (a00e19c).

Files with missing lines Patch % Lines
...rk/extensions/network/NetworkExtensionElement.java 79.13% 1 Missing and 23 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #14734      +/-   ##
============================================
+ Coverage     20.81%   20.83%   +0.02%     
- Complexity    20923    20975      +52     
============================================
  Files          6053     6053              
  Lines        548451   548562     +111     
  Branches      73279    73303      +24     
============================================
+ Hits         114138   114300     +162     
+ Misses       420948   420849      -99     
- Partials      13365    13413      +48     
Flag Coverage Δ
uitests 3.86% <ø> (ø)
unittests 22.27% <79.13%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…d routes

NetworkExtensionElement implements every other VpcProvider method by delegating
to the extension script, but four of them throw UnsupportedOperationException:
createPrivateGateway, deletePrivateGateway, applyStaticRoutes and
applyACLItemsToPrivateGw. The restriction is at the framework level, so it
applies to every network extension whatever its backend can do, and an
extension-backed VPC cannot reach parity with the VPC virtual router.

This wires them up the same way as their neighbours, adding four script
commands:

  create-private-gateway
  delete-private-gateway
  apply-static-routes
  apply-private-gw-acl

Private gateway operations resolve the owning VPC and go through the existing
VPC-level script path, carrying the gateway's id, uuid, address, netmask,
broadcast URI and derived VLAN alongside the usual VPC fields.
applyStaticRoutes and applyACLItemsToPrivateGw are full rebuilds that send the
complete non-revoked set on every call, matching applyFWRules and
applyNetworkACLs rather than the per-object style of add-static-nat; a backend
reconciles instead of tracking deltas.

An extension that does not implement these verbs is unaffected: the commands
are new, and before this change the operations failed with
UnsupportedOperationException regardless of the script, so no previously
working behaviour changes.

addPrivateGatewayToPayload documents gateway_vlan as conditional but emitted it
unconditionally whenever a broadcast URI was present. BroadcastDomainType
returns null for a URI whose scheme it does not recognise, so a non-VLAN
gateway shipped "gateway_vlan": "" to the script. It is now emitted only when
it has a value, which is what the javadoc already claimed and what every
neighbouring field does.

Adds NetworkExtensionElementTest, the first unit test this class has had. 26
cases over the four methods: the payload fields each sends, the VPC and gateway
identity the script needs to find the router, the omit-when-blank branch of
every optional field, both failure styles (createPrivateGateway and
deletePrivateGateway return false, applyStaticRoutes and
applyACLItemsToPrivateGw throw ResourceUnavailableException), full
serialisation of a static route and of an ACL rule, rule ordering by number and
routes by CIDR, and a null CIDR through the route comparator. The command name
constants are asserted against their literals, since they are the wire contract
with every deployed script.

Adds NetworkExtensionElementPayloadContractTest, 6 more. It asserts the keys
the script cannot run without -- a transcription of a reference
implementation's require_arg calls, with the source named in each test -- and
writes the payloads this class actually produces to target/extension-payloads/.
The unit tests assert what this class writes and an extension's own tests
assert what it reads, each checked against a fixture written by the same hand,
so both would pass if a key name drifted between them. Replaying the real
payloads against a real backend is what closes that: the six captured payloads
pass 21 assertions against a live OVN northbound database, including that
ovn-northd resolves the static routes this class emitted.

377 tests pass in framework/extensions.
@weizhouapache

Copy link
Copy Markdown
Member

great, thanks @kiwiflyer !

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants