Skip to content

Spike: check for RavenDB index rebuilds before an upgrade - #6165

Draft
ramonsmits wants to merge 1 commit into
masterfrom
spike/pre-upgrade-index-check
Draft

ramonsmits wants to merge 1 commit into
masterfrom
spike/pre-upgrade-index-check

Conversation

@ramonsmits

Copy link
Copy Markdown
Member

ServiceControl reports a changed index definition only at start-up, after the upgrade. This spike moves the check in front of the upgrade, so the operator can cancel before RavenDB rebuilds a large index.

This is a spike to support the decisions below. It is not ready to merge.

What it does

  • Each RavenDB persister ships index-definitions.manifest. It holds the parts of each index definition that make RavenDB rebuild the index: maps, reduce, fields, additional sources, and additional assemblies. A test fails when the manifest differs from the definitions in code.
  • The installer engine embeds the manifest. Before an upgrade it reads the index definitions from the running embedded RavenDB over HTTP and compares them. The check runs in CanUpgradeInstance, before the instance stops, because the database answers only while the instance runs.
  • ServiceControl Management shows a yes/no dialog that lists each changed index with its database, entry count, the changed parts, and whether it is locked. The text states the effect on CPU, RAM, disk IO, disk space, and duration.
  • The PowerShell module stops the upgrade with the same text unless the caller passes -Acknowledgements RavenDBIndexRebuild.

The comparison is plain JSON, so ServiceControl Management and the PowerShell module do not load the persister assembly or RavenDB.Client. The RavenDB persistence test projects compile the same file and run it against a real server. These tests cover that a normal deployment reports no change for every audit and primary index, and that a changed map is reported. Line endings are normalized, because the client renders a map with the line ending of its operating system.

Open decisions

  1. PowerShell: a required acknowledgement breaks existing unattended upgrade scripts. Alternatives are a warning only, or a warning in 6.x and a required acknowledgement from the next major version. -Force is not available as the bypass, because on the upgrade cmdlets it means "replace the RavenDB 3.5 database".
  2. A locked index with a changed definition: the spike warns and lets the operator continue. Blocking the upgrade may be better, because the upgraded instance does not start until the index is unlocked.
  3. An instance that does not run gets no check and no message. Options: report that the check was not possible, or compare the manifest in the install folder with the bundled one.
  4. An index change needs an update of the approval file and of the manifest. The alternative is to generate the manifest at build time.

Known gaps

  • Instances with an external RavenDB connection string get no check.
  • A pending side-by-side replacement, from a rebuild that already runs, is ignored.
  • New indexes are not reported, only changed ones.
  • The ServiceControl Management dialog and the cmdlet path are not verified on Windows.

Conflict with an ADR

The RavenDB index search engine and rebuilds ADR records a pre-upgrade check in ServiceControl Management as not pursued, because that work shipped in a patch release. The ADR needs an update if this spike continues.

Docs

Reviewed docs/README.md, the ADR above, and docs/guidelines/database-schema-and-index-changes.md. No docs/ page is updated, because this is a spike.

Guidelines: none applies. No index definition, hosted service, service registration, or persistence operation changes.

ServiceControl reports a changed index definition only at start-up, after
the upgrade. This spike moves the check in front of the upgrade.

Each RavenDB persister ships index-definitions.manifest, which holds the
parts of every index definition that make RavenDB rebuild the index. A test
keeps the manifest equal to the definitions in code. The installer engine
embeds the manifest, reads the definitions from the running embedded RavenDB
over HTTP, and compares them.

ServiceControl Management asks for confirmation when an index changes. The
PowerShell module stops the upgrade unless the caller passes
-Acknowledgements RavenDBIndexRebuild.

The check is skipped when the instance does not run, or uses an external
RavenDB. The ADR for index rebuilds records this check as not pursued, so
the ADR needs an update if the spike continues.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant