Skip to content

server: report a domain's bucket, object storage and backup limits correctly - #14293

Open
stag7824 wants to merge 3 commits into
apache:mainfrom
stag7824:fix-domain-bucket-objectstorage-limits
Open

stag7824 wants to merge 3 commits into
apache:mainfrom
stag7824:fix-domain-bucket-objectstorage-limits

Conversation

@stag7824

@stag7824 stag7824 commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #13944

listDomains reports the bucket and object storage limits of every non-root domain as Unlimited, whatever has been configured. @ms30063600's re-test on #13944 shows it clearly: all 17 limits read back correctly from listResourceLimits, but the usage view (which reads listDomains) shows Bucket and Object Storage as Unlimited.

DomainJoinDaoImpl.setResourceLimits resolves those two types through the account helper:

long bucketLimit = ApiDBUtils.findCorrectResourceLimit(domain.getBucketLimit(), domain.getId(), ResourceType.bucket);
long objectStorageLimit = ApiDBUtils.findCorrectResourceLimit(domain.getObjectStorageLimit(), domain.getId(), ResourceType.object_storage);

findCorrectResourceLimit(limit, accountId, type) delegates to ResourceLimitManagerImpl.findCorrectResourceLimitForAccount, which first checks isRootAdmin(accountId) and then _accountDao.findById(accountId), returning unlimited for either. Passed a domain id, it is looking up an unrelated account — typically the root admin (account 2 vs. a domain with id 2) or nothing at all — so the configured limit is never reached. The other fifteen types in this method already use findCorrectResourceLimitForDomain; this changes the two outliers to match.

While in there, the backup rows had their unlimited checks copied from the row above:

String backupLimitDisplay = (fullView || snapshotLimit == -1) ? ...           // should be backupLimit
String backupAvail = (fullView || snapshotLimit == -1) ? ...                  // should be backupLimit
String backupStorageLimitDisplay = (fullView || backupLimit == -1) ? ...      // should be backupStorageLimit

so a domain with unlimited snapshots shows its backup limit as Unlimited, and one with unlimited backups shows its backup storage limit as Unlimited. AccountJoinDaoImpl does not have either problem.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)

Feature/Enhancement Scale or Bug Severity

Bug Severity

  • Minor

How Has This Been Tested?

Added DomainJoinDaoImplTest, which calls setResourceLimits directly with ApiDBUtils stubbed (the domain lookup returns the domain's own limit; the account lookup returns unlimited, as it does in practice for a domain id) and verifies the values set on the response:

  • bucket 147 and object storage 157 GiB are reported as 147 / 157
  • backup 127 is reported while the snapshot limit is unlimited
  • backup storage 137 GiB is reported while the backup limit is unlimited
mvn test -pl server -Dtest=DomainJoinDaoImplTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0

Without the change to DomainJoinDaoImpl, all three fail. mvn checkstyle:check -pl server reports no violations.

I do not have a live environment to check the UI against, so this is unit-level; the reporter's JSON and screenshots on #13944 are the end-to-end reproduction.

One thing I could not explain from the code: the same screenshot shows VPC as "77 Available" but "0 / 67". Both values come from the same vpcLimit variable in this method, so that mismatch does not originate here and this PR does not address it.

@rootnomad

Copy link
Copy Markdown
Contributor

Reproduced this on a 4.23 build and the change resolves it. On a non-root domain with the limits set through updateResourceLimit:

  • listResourceLimits reports them as configured: bucket 47, object_storage 57, backup 127, backup_storage 137
  • listDomains with details=resource, before the change, reports bucket, object storage and backup all as Unlimited

With this PR applied, the same listDomains call returns bucket 47, object storage 57, backup 127. The backup-storage gate is covered too: with the backup limit set to -1, backup storage keeps its own value instead of being pulled to Unlimited.

That lines up with both the account-vs-domain helper on the bucket and object-storage lines and the snapshot/backup mix-up in the backup rows.

@stag7824

Copy link
Copy Markdown
Contributor Author

For anyone looking at CI: the 8 workflow runs on the current head (59fb20a) have been waiting for approval since Oct 2, so nothing has run against this change yet. The unit tests were run locally (3 new tests, which fail without the change and pass with it; checkstyle clean), and @rootnomad's check above confirms the behaviour on a 4.23 build.

@DaanHoogland, since you were on #13944, could you or another committer approve the workflow runs? Thanks.

@DaanHoogland

Copy link
Copy Markdown
Contributor

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@codecov

codecov Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.86%. Comparing base (1a48a87) to head (bce946c).
⚠️ Report is 129 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (1a48a87) and HEAD (bce946c). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (1a48a87) HEAD (bce946c)
unittests 1 0
Additional details and impacted files
@@              Coverage Diff              @@
##               main   #14293       +/-   ##
=============================================
- Coverage     19.91%    3.86%   -16.06%     
=============================================
  Files          6373      494     -5879     
  Lines        577230    42888   -534342     
  Branches      70696     8141    -62555     
=============================================
- Hits         114950     1657   -113293     
+ Misses       449713    40999   -408714     
+ Partials      12567      232    -12335     
Flag Coverage Δ
uitests 3.86% <ø> (+0.15%) ⬆️
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.


//get resource limits for buckets
long bucketLimit = ApiDBUtils.findCorrectResourceLimit(domain.getBucketLimit(), domain.getId(), ResourceType.bucket);
long bucketLimit = ApiDBUtils.findCorrectResourceLimitForDomain(domain.getBucketLimit(), ResourceType.bucket, domain.getId());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happens if the domain has no bucket limit set? looks like the fallback only reads max.domain.buckets for storage types, so this still shows unlimited while creating stops at 20

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, you're right. When a domain has no limit of its own, ApiDBUtils.findCorrectResourceLimitForDomain(Long, ResourceType, long) fell back to findDefaultResourceLimitForDomain, which only applies the max.domain.* default to storage types and returns unlimited for everything else, while enforcement uses ResourceLimitService.findCorrectResourceLimitForDomain (parent domain's limit, then the default). So it wasn't only buckets: IPs, volumes, snapshots, networks, CPU and memory showed unlimited the same way; VMs looked right because they already go through the other overload.

I've pushed a change (8f6592a) so that fallback uses the same lookup as enforcement, with a unit test that fails without it (expected:<20> but was:<-1>). One thing to be aware of: this means a domain without its own limits will now show the inherited or default limits for those types instead of "Unlimited", which is what's actually enforced.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that covers it

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19532

}
//--If limit doesn't have a value then fetch default limit from the configs
return (limit == null) ? s_resourceLimitMgr.findDefaultResourceLimitForDomain(resourceType) : limit;
//--If limit doesn't have a value then fetch the limit that is enforced: a parent domain's limit or the default from the configs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
//--If limit doesn't have a value then fetch the limit that is enforced: a parent domain's limit or the default from the configs

I think this comment doesn't say anything beyond what the code below says

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 4a0d85e, thanks.

@nvazquez

Copy link
Copy Markdown
Contributor

@blueorangutan package

@nvazquez nvazquez added this to the 24.0 milestone Oct 30, 2026
@blueorangutan

Copy link
Copy Markdown

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19840

@nvazquez

Copy link
Copy Markdown
Contributor

@blueorangutan test

@blueorangutan

Copy link
Copy Markdown

@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan

Copy link
Copy Markdown

[SF] Trillian test result (tid-17343)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 54352 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr14293-t17343-kvm-ol8.zip
Smoke tests completed. 147 look OK, 2 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestClusterDRS>:setup Error 0.00 test_cluster_drs.py
test_list_system_vms_metrics_history Failure 0.21 test_metrics_api.py

@RosiKyu

RosiKyu commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

@stag7824 thanks for the fix. Could you please rebase it onto main and change the base branch to main? It can be backported to 4.22 afterwards.

…rrectly

listDomains reported the bucket and object storage limits of every
non-root domain as Unlimited, whatever was configured. DomainJoinDaoImpl
resolved those two through ApiDBUtils.findCorrectResourceLimit, which is
the account variant: it looks the id up in the account table. Given a
domain id it finds either the root admin account or no account, and
answers unlimited either way. The other fifteen resource types already go
through findCorrectResourceLimitForDomain.

The backup and backup storage rows had their unlimited checks copied from
the row above: the backup limit was hidden whenever the snapshot limit was
unlimited, and the backup storage limit whenever the backup limit was.

listResourceLimits was never affected, which is why the values set
through updateResourceLimit read back correctly there and only the usage
view in the UI, which reads listDomains, showed them as Unlimited.

Fixes apache#13944
… own

When a domain has no limit set, the domain response fell back to
findDefaultResourceLimitForDomain, which only reads the max.domain.*
default for storage types and answers unlimited for the rest. Creating
a resource still stops at the default (or at a parent domain's limit),
so for example the bucket limit showed unlimited while bucket creation
stopped at 20. Use the same lookup that is enforced.
@stag7824
stag7824 force-pushed the fix-domain-bucket-objectstorage-limits branch from 4a0d85e to bce946c Compare November 18, 2026 14:12
@stag7824
stag7824 changed the base branch from 4.22 to main November 18, 2026 14:12
@stag7824

Copy link
Copy Markdown
Contributor Author

@RosiKyu done: the three commits are now on top of main (bce946c) and the base branch is main. They applied cleanly; the diff is still the same four files, and ApiDBUtilsTest and DomainJoinDaoImplTest pass on main (6 tests, checkstyle clean). Thanks!

@RosiKyu

RosiKyu commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@RosiKyu a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan

Copy link
Copy Markdown

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 20095

@RosiKyu

RosiKyu commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

@blueorangutan package

@blueorangutan

Copy link
Copy Markdown

@RosiKyu a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[UI BUG] Resource limits for type 15 (ObjectStorage) and 16 (GPU) do not sync/display correctly in Web UI

7 participants