diff --git a/docs/guide/platform/installation/ci.md b/docs/guide/platform/installation/ci.md index b4b49bda4..c93cbe075 100644 --- a/docs/guide/platform/installation/ci.md +++ b/docs/guide/platform/installation/ci.md @@ -21,7 +21,7 @@ Inject the following values from protected CI secrets. Do not commit them in a w | `VOID_PLATFORM_HYPERDRIVE_ORIGIN_USER` | Hyperdrive runtime database user | | `VOID_PLATFORM_HYPERDRIVE_ORIGIN_SERVICE_ID` | Workers VPC service ID, for a private database origin | | `VOID_PLATFORM_R2_ACCESS_KEY_ID` | R2 Access Key ID | -| `VOID_PLATFORM_R2_SECRET_ACCESS_KEY` | R2 Secret Access Key | +| `VOID_PLATFORM_R2_SECRET_ACCESS_KEY` | R2 Secret Access Key (not the token value) | | `VOID_PLATFORM_JWT_SECRET` | Original JWT signing secret | | `VOID_PLATFORM_EMAIL_SIGNING_SECRET` | Dedicated email signing secret when email is enabled | | `VOID_PLATFORM_PROJECT_SECRET_KEY` | Original base64-encoded project-encryption key | diff --git a/docs/guide/platform/installation/credentials.md b/docs/guide/platform/installation/credentials.md index c4b09b9af..6d85e845b 100644 --- a/docs/guide/platform/installation/credentials.md +++ b/docs/guide/platform/installation/credentials.md @@ -113,6 +113,8 @@ The installer configures the email gateway and shared mail route. If setup fails The installer opens the **R2 token creation** form with a token name based on your platform's display name, **Object Read & Write**, and **Apply to all buckets in this account (including newly created buckets)** preselected. Review the settings and create an account token, or a user token if your role requires it. Save its **Access Key ID** and **Secret Access Key** in your password manager, then paste both into the installer. These differ from the management and runtime API tokens. See [R2’s token instructions](https://developers.cloudflare.com/r2/api/tokens/). +If you create the token with the Cloudflare API instead, the Access Key ID is the token's ID and the Secret Access Key is the SHA-256 hex digest of the token value. Paste the Secret Access Key, not the token value. + ## Signing and Encryption Keys {#signing-and-encryption-keys} The **JWT signing key** signs platform login tokens. The **Project encryption key** encrypts app secrets stored by your platform. An interactive install generates both and keeps them in this machine's encrypted recovery data. Lifecycle commands on this machine read them from there. diff --git a/docs/guide/platform/installation/maintenance.md b/docs/guide/platform/installation/maintenance.md index 959202e58..156a0953c 100644 --- a/docs/guide/platform/installation/maintenance.md +++ b/docs/guide/platform/installation/maintenance.md @@ -116,7 +116,9 @@ An upgrade validates the runtime, applies supported pending migrations, deploys Install, upgrade, and repair also publish saved [AI Gateway logging](/reference/cli/platform-users#ai-gateway-logging) settings. Affected projects may show `pending` for about 2 minutes afterward, including projects returning to `inherit`. Check progress with `void platform project show `. -Before starting, Void checks access to the installation's Cloudflare account, Workers, and existing R2 buckets. If credentials are rejected, set `CLOUDFLARE_API_TOKEN` to a management token scoped to that account and retry. This overrides the installation's saved management token. +Before starting, Void checks access to the installation's Cloudflare account, Workers, and existing R2 buckets. It lists every missing permission at once and names the credential Cloudflare rejected. If the management credential is rejected, set `CLOUDFLARE_API_TOKEN` to a management token scoped to that account and retry. This overrides the installation's saved management token. + +With browser login, Void uses the runtime token for AI Gateway: `VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN`, or the one saved on this machine. If an error names the runtime token, set `VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN` to a valid runtime token, or set `CLOUDFLARE_API_TOKEN` to a management token with **Account → AI Gateway → Edit**. Void then uses that token instead of browser login. Some upgrades pause application traffic until the new version passes health checks. If a command fails while traffic is paused: @@ -127,12 +129,16 @@ Some upgrades pause application traffic until the new version passes health chec ::: details Migration safety, credential rotation, and rollback To update the runtime Cloudflare token, GitHub OAuth credentials when GitHub login is enabled, or R2 credentials, -provide their `VOID_PLATFORM_*` environment variables when running the upgrade. +provide their `VOID_PLATFORM_*` environment variables when running `upgrade` or `repair`. Only the supplied values replace live credentials. Recovery data from an older workstation does not overwrite them, and upgrades preserve the live signing keys and complete project-encryption keyring. Keep current credentials in your secret manager so they can also restore a missing Worker during repair. +Set `VOID_PLATFORM_R2_ACCESS_KEY_ID` and `VOID_PLATFORM_R2_SECRET_ACCESS_KEY` together, even if +only the secret changed. Void checks the R2 pair against the assets bucket before it changes the +platform. + Database migrations only move forward. Void checks compatibility before upgrading and tells you if an intermediate release is needed. If deployment fails, it attempts to restore the previous Workers against the compatible database schema. Retrying does not repeat completed migrations. After a successful upgrade, you can restore a declared-compatible earlier runtime without reversing migrations: diff --git a/packages/void/src/cli/direct-r2-upload.ts b/packages/void/src/cli/direct-r2-upload.ts index fb4436e7c..e87e2aab3 100644 --- a/packages/void/src/cli/direct-r2-upload.ts +++ b/packages/void/src/cli/direct-r2-upload.ts @@ -20,6 +20,80 @@ const MAX_R2_PUT_RETRIES = 3; */ const RETRY_BACKOFF_MS = [250, 500, 1000]; +export const R2_CREDENTIALS_DOCS_URL = + 'https://void.cloud/guide/platform/installation/credentials#r2-upload-credentials'; + +/** + * S3 error codes that mean R2 did not accept the signature on a presigned + * upload link. The platform signs every link with its own R2 key pair, so + * these usually point at that pair. Expiry and clock-skew codes are left out: + * they have other causes. + */ +export const R2_CREDENTIAL_ERROR_CODES: ReadonlySet = new Set([ + 'Unauthorized', + 'AccessDenied', + 'SignatureDoesNotMatch', + 'InvalidAccessKeyId', +]); + +/** + * S3 error codes R2 answers with HTTP 400 when the Access Key ID is malformed, + * for example the token value pasted in place of the 32-character key ID. + */ +export const R2_MALFORMED_CREDENTIAL_ERROR_CODES: ReadonlySet = new Set([ + 'InvalidArgument', + 'AuthorizationHeaderMalformed', +]); + +/** Whether an R2 error answer means R2 rejected the key pair itself. */ +export function isR2CredentialRejection(status: number, code: string | undefined): boolean { + if (!code) { + return false; + } + if (status === 401 || status === 403) { + return R2_CREDENTIAL_ERROR_CODES.has(code); + } + return status === 400 && R2_MALFORMED_CREDENTIAL_ERROR_CODES.has(code); +} + +/** + * Read the `` value from an S3-style XML error body. Returns + * `undefined` when the body has no complete, non-empty `` element. + */ +export function r2ErrorCode(body: string): string | undefined { + const open = ''; + const start = body.indexOf(open); + if (start === -1) { + return undefined; + } + const valueStart = start + open.length; + const end = body.indexOf('', valueStart); + if (end === -1) { + return undefined; + } + const code = body.slice(valueStart, end).trim(); + return code === '' ? undefined : code; +} + +function directR2PutErrorMessage( + status: number, + hash: string, + path: string, + body: string, + code: string | undefined, +): string { + if (isR2CredentialRejection(status, code)) { + return ( + `R2 rejected the upload signature (${code}) for ${path} (${hash}). ` + + "This usually means the platform's R2 upload credentials are wrong. " + + 'Ask the platform operator to fix it. ' + + 'If you run this platform: set VOID_PLATFORM_R2_ACCESS_KEY_ID and VOID_PLATFORM_R2_SECRET_ACCESS_KEY ' + + `to a working pair and run \`void platform repair\`. See ${R2_CREDENTIALS_DOCS_URL}` + ); + } + return `R2 PUT failed for ${path} (${hash}): ${status}${body ? ` ${body}` : ''}`; +} + export class DirectR2PutError extends Error { readonly status: number; readonly hash: string; @@ -30,14 +104,18 @@ export class DirectR2PutError extends Error { * `BadDigest` (Content-MD5 mismatch) or signature errors. */ readonly body: string; + /** S3 `` parsed from `body`, when R2 returned an XML error. */ + readonly code: string | undefined; constructor(status: number, hash: string, path: string, body: string) { - super(`R2 PUT failed for ${path} (${hash}): ${status}${body ? ` ${body}` : ''}`); + const code = r2ErrorCode(body); + super(directR2PutErrorMessage(status, hash, path, body, code)); this.name = 'DirectR2PutError'; this.status = status; this.hash = hash; this.path = path; this.body = body; + this.code = code; } } diff --git a/packages/void/src/cli/platform-auth-protection.ts b/packages/void/src/cli/platform-auth-protection.ts index 3a72deb97..adce5ca78 100644 --- a/packages/void/src/cli/platform-auth-protection.ts +++ b/packages/void/src/cli/platform-auth-protection.ts @@ -16,7 +16,10 @@ import { parsePlatformAccessSetupSpec, preparePlatformAccessSetup, } from './platform-access-setup.ts'; -import { resolvePlatformCloudflareCredential } from './platform-auth.ts'; +import { + platformOperatorCredentialLabel, + resolvePlatformCloudflareCredential, +} from './platform-auth.ts'; import { PlatformCloudflareClient } from './platform-cloudflare.ts'; import type { PlatformConfigArgs } from './platform-config-args.ts'; import { acquirePlatformDatabaseLifecycleLock } from './platform-installer.ts'; @@ -237,7 +240,9 @@ export async function runPlatformProtectionConfig(args: PlatformConfigArgs): Pro ); } const credential = await resolvePlatformCloudflareCredential(); - const cloudflare = new PlatformCloudflareClient(credential.token); + const cloudflare = new PlatformCloudflareClient(credential.token, { + credential: platformOperatorCredentialLabel(credential), + }); let protection = null; const checkpoint = { ...recovery }; const save = (values: Record) => { diff --git a/packages/void/src/cli/platform-auth-recovery.ts b/packages/void/src/cli/platform-auth-recovery.ts index a522463c3..33e01f6c6 100644 --- a/packages/void/src/cli/platform-auth-recovery.ts +++ b/packages/void/src/cli/platform-auth-recovery.ts @@ -16,7 +16,10 @@ import { transaction, type Connection, } from './platform-auth-recovery-core.ts'; -import { resolvePlatformCloudflareCredential } from './platform-auth.ts'; +import { + platformOperatorCredentialLabel, + resolvePlatformCloudflareCredential, +} from './platform-auth.ts'; import { PlatformCloudflareClient } from './platform-cloudflare.ts'; import type { PlatformConfigArgs } from './platform-config-args.ts'; import { @@ -85,7 +88,9 @@ export async function runPlatformAuthRecovery(args: PlatformConfigArgs): Promise releaseDatabase.authoritativeManifest, ); await verifyPlatformInstallationOwnership( - new PlatformCloudflareClient(credential.token), + new PlatformCloudflareClient(credential.token, { + credential: platformOperatorCredentialLabel(credential), + }), manifest, { databaseUrl }, ); diff --git a/packages/void/src/cli/platform-auth.ts b/packages/void/src/cli/platform-auth.ts index 4e4de6af7..84df7c3e7 100644 --- a/packages/void/src/cli/platform-auth.ts +++ b/packages/void/src/cli/platform-auth.ts @@ -4,11 +4,19 @@ import { type CloudflareAccount, type PlatformCloudflareClient, } from './platform-cloudflare.ts'; +import { + describePlatformCredential, + formatPlatformCredentialDenials, + type PlatformCredentialDenial, + type PlatformCredentialLabel, +} from './platform-credential-label.ts'; import { planCreatesZone, planWritesDns } from './platform-plan-writes.ts'; import type { PlatformInstallPlan } from './platform-plan.ts'; export type PlatformCloudflareCredential = Readonly<{ source: 'environment' | 'saved' | 'oauth'; + /** Which variable held an `environment` token. */ + environmentVariable?: 'CLOUDFLARE_API_TOKEN' | 'CF_API_TOKEN'; token: string; }>; @@ -24,76 +32,106 @@ export type PlatformManagementAccess = Readonly<{ readOnly: boolean; }>; +/** The label of the operator credential, so Cloudflare errors can name it. */ +export function platformOperatorCredentialLabel( + credential: Pick, +): PlatformCredentialLabel { + return { + role: 'management', + source: credential.source, + ...(credential.environmentVariable + ? { environmentVariable: credential.environmentVariable } + : {}), + }; +} + /** Use the selected credential for read-only checks before taking lifecycle locks. */ export async function verifyPlatformCloudflareCredential( client: Pick, input: { account: CloudflareAccount; source: PlatformCloudflareCredential['source']; + environmentVariable?: PlatformCloudflareCredential['environmentVariable']; r2Buckets: ReadonlyArray; allowMissingR2Buckets?: boolean; }, ): Promise { - const source = - input.source === 'saved' - ? 'saved management token' - : input.source === 'environment' - ? 'environment API token' - : 'Cloudflare browser login'; + const label = platformOperatorCredentialLabel(input); + const name = describePlatformCredential(label); const retry = input.source === 'oauth' ? 'Run `void cloudflare login`, or set CLOUDFLARE_API_TOKEN for this account, then retry.' - : 'Set CLOUDFLARE_API_TOKEN for this account, then retry.'; + : `Set ${input.environmentVariable ?? 'CLOUDFLARE_API_TOKEN'} for this account, then retry.`; + // Run every check, so one error lists every missing permission. + const denials: Array = []; + let firstDenial: PlatformCloudflareApiError | undefined; + const deniedError = (denial: PlatformCloudflareApiError) => + new Error(`platform: ${formatPlatformCredentialDenials(label, denials)}`, { cause: denial }); const check = async ( - label: string, + what: string, permission: string, action: () => Promise, allowMissing = false, - ) => { + ): Promise => { try { await action(); + return true; } catch (cause) { const status = cause instanceof PlatformCloudflareApiError ? cause.status : null; if (status === 404 && allowMissing) { - return; + return true; + } + if (status === 401 || status === 403) { + denials.push({ permission, request: what, status }); + firstDenial ??= cause as PlatformCloudflareApiError; + return false; } const detail = - status === 401 || status === 403 - ? `Check that the credential is valid, scoped to this account, and includes ${permission}. ${retry}` - : status === 404 - ? 'Check the bucket name and account scope before retrying.' - : status === 429 - ? 'Cloudflare rate-limited this check. Retry shortly.' - : 'Retry the command after Cloudflare connectivity is restored.'; + status === 404 + ? 'Check the bucket name and account scope before retrying.' + : status === 429 + ? 'Cloudflare rate-limited this check. Retry shortly.' + : 'Retry the command after Cloudflare connectivity is restored.'; + // Report a denial already seen, so callers still see the rejected token. + if (firstDenial) { + throw deniedError(firstDenial); + } throw new Error( - `platform: ${label} is unavailable${status === null ? '' : ` (HTTP ${status})`} using the ${source}. ${detail}`, + `platform: ${what} is unavailable${status === null ? '' : ` (HTTP ${status})`} using the ${name}. ${detail}`, { cause }, ); } }; let accounts: ReadonlyArray = []; - await check('Cloudflare account access', 'Account → Account Settings → Read', async () => { - accounts = await client.listAccounts(); - }); - if (!accounts.some((account) => account.id === input.account.id)) { + const listedAccounts = await check( + 'Cloudflare account access', + 'Account → Account Settings → Read', + async () => { + accounts = await client.listAccounts(); + }, + ); + if (listedAccounts && !accounts.some((account) => account.id === input.account.id)) { throw new Error( - `platform: The ${source} cannot access Cloudflare account ${JSON.stringify(input.account.name)} (${input.account.id}). ${retry}`, + `platform: The ${name} cannot access Cloudflare account ${JSON.stringify(input.account.name)} (${input.account.id}). ${retry}`, ); } await check('Cloudflare Workers access', 'Account → Workers Scripts → Edit', () => client.listWorkerNames(input.account.id), ); - for (const name of new Set(input.r2Buckets)) { + for (const bucket of new Set(input.r2Buckets)) { await check( - `R2 bucket ${JSON.stringify(name)}`, + `R2 bucket ${JSON.stringify(bucket)}`, 'Account → Workers R2 Storage → Edit', () => client.request( - `/accounts/${encodeURIComponent(input.account.id)}/r2/buckets/${encodeURIComponent(name)}`, + `/accounts/${encodeURIComponent(input.account.id)}/r2/buckets/${encodeURIComponent(bucket)}`, ), input.allowMissingR2Buckets, ); } + if (firstDenial) { + throw deniedError(firstDenial); + } } /** Check Access inventory permissions before the installer starts provisioning. */ @@ -225,10 +263,11 @@ async function readExistingCredential( dependencies: PlatformCloudflareAuthDependencies, runWrangler: CloudflareAuthRunner, ): Promise { - const environmentToken = - process.env.CLOUDFLARE_API_TOKEN?.trim() || process.env.CF_API_TOKEN?.trim() || ''; - if (environmentToken) { - return { source: 'environment', token: environmentToken }; + for (const environmentVariable of ['CLOUDFLARE_API_TOKEN', 'CF_API_TOKEN'] as const) { + const environmentToken = process.env[environmentVariable]?.trim(); + if (environmentToken) { + return { source: 'environment', environmentVariable, token: environmentToken }; + } } const savedManagementToken = dependencies.savedManagementToken?.trim(); if (savedManagementToken) { diff --git a/packages/void/src/cli/platform-cloudflare.ts b/packages/void/src/cli/platform-cloudflare.ts index 20cfc8789..56108ed72 100644 --- a/packages/void/src/cli/platform-cloudflare.ts +++ b/packages/void/src/cli/platform-cloudflare.ts @@ -1,5 +1,12 @@ import { createHash, randomUUID } from 'node:crypto'; import { formatUrl } from './output.ts'; +import { + describePlatformCredential, + formatPlatformCredentialDenials, + platformCredentialHelp, + type PlatformCredentialDenial, + type PlatformCredentialLabel, +} from './platform-credential-label.ts'; import { resolvePinnedApiBaseUrl } from './provision.ts'; type CloudflareEnvelope = Readonly<{ @@ -17,12 +24,22 @@ type CloudflareEnvelope = Readonly<{ export class PlatformCloudflareApiError extends Error { readonly status: number; readonly requestPath?: string; + /** The credential Cloudflare rejected, when the client knows it. */ + readonly credential?: PlatformCredentialLabel; - constructor(message: string, status: number, requestPath?: string) { + constructor( + message: string, + status: number, + requestPath?: string, + credential?: PlatformCredentialLabel, + ) { super(message); this.name = 'PlatformCloudflareApiError'; this.status = status; this.requestPath = requestPath; + if (credential) { + this.credential = credential; + } } } @@ -57,11 +74,19 @@ export class PlatformWorkerRoutesChangedError extends Error { export class PlatformRuntimeTokenEmailPermissionError extends Error { /** Cloudflare answered HTTP 401 at least once: it did not accept the token. */ readonly tokenNotValid: boolean; + /** Each email check Cloudflare denied. */ + readonly denials: ReadonlyArray; - constructor(message: string, tokenNotValid: boolean, options?: ErrorOptions) { + constructor( + message: string, + tokenNotValid: boolean, + options?: ErrorOptions, + denials: ReadonlyArray = [], + ) { super(message, options); this.name = 'PlatformRuntimeTokenEmailPermissionError'; this.tokenNotValid = tokenNotValid; + this.denials = denials; } } @@ -177,47 +202,80 @@ export async function verifyPlatformRuntimeTokenPermissions( accountId: string; applicationDomain: string; customDomains: boolean; + /** Where the token came from, so a failure can name it. */ + credential?: PlatformCredentialLabel; /** The shared email zone, only when platform email is on. */ emailZoneId?: string; zoneId: string | null; }, ): Promise { - try { - await client.listHyperdriveConfigs(input.accountId); - } catch (error) { - throw runtimeTokenPreflightError('Account → Hyperdrive → Write', error); - } - - if (input.emailZoneId) { - await verifyPlatformRuntimeTokenEmailPermissions(client, { - accountId: input.accountId, - zoneId: input.emailZoneId, - }); - } + const credential = input.credential ?? { role: 'runtime' }; + // Run every check, so one error lists every missing permission. Only an + // authorization answer shows a missing permission; anything else stops. + const denials: Array = []; + let emailDenial: PlatformRuntimeTokenEmailPermissionError | undefined; + // Callers keep a token that only lacks email access, so this error keeps + // its own class when email is the only problem. + const deniedError = (): Error | undefined => + denials.length === 0 + ? emailDenial + : new Error( + `platform: ${formatPlatformCredentialDenials(credential, [...denials, ...(emailDenial?.denials ?? [])])}`, + { cause: denials[0]!.error }, + ); + const check = async (permission: string, action: () => Promise) => { + try { + await action(); + } catch (error) { + if ( + error instanceof PlatformCloudflareApiError && + (error.status === 401 || error.status === 403) + ) { + denials.push({ error, permission, status: error.status }); + return; + } + // Report a denial already seen, so callers still see the rejected token. + throw deniedError() ?? runtimeTokenPreflightError(permission, error, credential); + } + }; - if (!input.zoneId) { - return; - } + await check('Account → Hyperdrive → Write', () => client.listHyperdriveConfigs(input.accountId)); - const probeUrl = `https://${input.applicationDomain}/.well-known/void/runtime-token-preflight-${randomUUID()}`; - try { - await client.request(`/zones/${encodeURIComponent(input.zoneId)}/purge_cache`, { - body: JSON.stringify({ files: [probeUrl] }), - method: 'POST', - }); - } catch (error) { - throw runtimeTokenPreflightError('Zone → Cache Purge → Purge', error); + if (input.emailZoneId) { + try { + await verifyPlatformRuntimeTokenEmailPermissions(client, { + accountId: input.accountId, + credential, + zoneId: input.emailZoneId, + }); + } catch (error) { + if (!(error instanceof PlatformRuntimeTokenEmailPermissionError)) { + throw error; + } + emailDenial = error; + } } - if (input.customDomains) { - try { - await client.request( - `/zones/${encodeURIComponent(input.zoneId)}/custom_hostnames?per_page=1`, + if (input.zoneId) { + const zoneId = input.zoneId; + const probeUrl = `https://${input.applicationDomain}/.well-known/void/runtime-token-preflight-${randomUUID()}`; + await check('Zone → Cache Purge → Purge', () => + client.request(`/zones/${encodeURIComponent(zoneId)}/purge_cache`, { + body: JSON.stringify({ files: [probeUrl] }), + method: 'POST', + }), + ); + if (input.customDomains) { + await check('Zone → SSL and Certificates → Edit', () => + client.request(`/zones/${encodeURIComponent(zoneId)}/custom_hostnames?per_page=1`), ); - } catch (error) { - throw runtimeTokenPreflightError('Zone → SSL and Certificates → Edit', error); } } + + const denied = deniedError(); + if (denied) { + throw denied; + } } /** @@ -230,6 +288,8 @@ export async function verifyPlatformRuntimeTokenEmailPermissions( client: Pick, input: { accountId: string; + /** Where the token came from, so a failure can name it. */ + credential?: PlatformCredentialLabel; zoneId: string; /** * The token is the one saved on this machine, and the run does not send @@ -248,7 +308,7 @@ export async function verifyPlatformRuntimeTokenEmailPermissions( `/accounts/${encodeURIComponent(input.accountId)}/email/routing/addresses?per_page=5`, ], ] as const; - const missing: Array = []; + const denials: Array = []; let denial: PlatformCloudflareApiError | undefined; let tokenNotValid = false; for (const [permission, path] of probes) { @@ -261,7 +321,7 @@ export async function verifyPlatformRuntimeTokenEmailPermissions( error instanceof PlatformCloudflareApiError && (error.status === 401 || error.status === 403) ) { - missing.push(permission); + denials.push({ permission, status: error.status }); denial ??= error; // A token that is not valid gets 403 on the zone read, but 401 on // the Email Routing reads. A valid token without access gets 403. @@ -270,25 +330,34 @@ export async function verifyPlatformRuntimeTokenEmailPermissions( } } if (denial) { - const denied = `Cloudflare denied: ${missing.join(', ')} (HTTP ${denial.status}).`; + const denied = `Cloudflare denied: ${denials.map((entry) => entry.permission).join(', ')} (HTTP ${denial.status}).`; + const name = describePlatformCredential( + input.savedToken + ? { role: 'runtime', source: 'saved' } + : (input.credential ?? { role: 'runtime' }), + ); throw new PlatformRuntimeTokenEmailPermissionError( input.savedToken - ? `platform: The runtime API token saved on this machine cannot use platform email. ${denied} Void checked this token because VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN is not set. If the platform uses a different runtime token, set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to that token, and Void checks that token instead. Otherwise, check that the zone permissions of the saved token include the email zone ${input.zoneId}. To add a missing permission, edit the token in the Cloudflare dashboard. The token value stays the same. Then run the command again.` - : `platform: The runtime API token cannot use platform email. ${denied} Check that the token is valid and that its zone permissions include the email zone ${input.zoneId}. To add a missing permission, edit the token in the Cloudflare dashboard. The token value stays the same. Or set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to a valid token that has these permissions. Then run the command again.`, + ? `platform: The ${name} cannot use platform email. ${denied} Void checked this token because VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN is not set. If the platform uses a different runtime token, set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to that token, and Void checks that token instead. Otherwise, check that the zone permissions of the saved token include the email zone ${input.zoneId}. To add a missing permission, edit the token in the Cloudflare dashboard. The token value stays the same. Then run the command again.` + : `platform: The ${name} cannot use platform email. ${denied} Check that the token is valid and that its zone permissions include the email zone ${input.zoneId}. To add a missing permission, edit the token in the Cloudflare dashboard. The token value stays the same. Or set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to a valid token that has these permissions. Then run the command again.`, tokenNotValid, { cause: denial }, + denials, ); } } -function runtimeTokenPreflightError(permission: string, cause: unknown): Error { +function runtimeTokenPreflightError( + permission: string, + cause: unknown, + credential: PlatformCredentialLabel, +): Error { const status = cause instanceof PlatformCloudflareApiError ? cause.status : null; + const name = describePlatformCredential(credential); const message = - status === 401 || status === 403 - ? `Cloudflare denied the runtime-token check (HTTP ${status}). Check that the token is valid, scoped to this account, and includes ${permission}.` - : status === 429 - ? `Cloudflare rate-limited the runtime-token check for ${permission} (HTTP 429). Retry shortly; this does not mean the token is missing a permission.` - : `Could not complete the runtime-token check for ${permission}${status === null ? '' : ` (Cloudflare HTTP ${status})`}. Retry the check; this failure does not establish that a permission is missing.`; + status === 429 + ? `Cloudflare rate-limited the check of the ${name} for ${permission} (HTTP 429). Retry shortly; this does not mean the token is missing a permission.` + : `Could not complete the check of the ${name} for ${permission}${status === null ? '' : ` (Cloudflare HTTP ${status})`}. Retry the check; this failure does not establish that a permission is missing.`; return new Error(`platform: ${message}`, { cause }); } @@ -297,10 +366,22 @@ export class PlatformCloudflareClient { readonly #fetcher: typeof fetch; readonly #token: string; readonly #aiGatewayClient?: PlatformCloudflareClient; + readonly #credential?: PlatformCredentialLabel; constructor( token: string, - options?: { apiBase?: string; fetcher?: typeof fetch; aiGatewayToken?: string }, + options?: { + apiBase?: string; + fetcher?: typeof fetch; + aiGatewayToken?: string; + /** Where the AI Gateway token came from. */ + aiGatewayCredential?: PlatformCredentialLabel; + /** + * Where the token came from. A 401 or 403 answer then names it and + * says what to do. + */ + credential?: PlatformCredentialLabel; + }, ) { if (!token) { throw new Error( @@ -310,20 +391,31 @@ export class PlatformCloudflareClient { this.#token = token; this.apiBaseUrl = (options?.apiBase ?? resolvePinnedApiBaseUrl()).replace(/\/+$/, ''); this.#fetcher = options?.fetcher ?? fetch; + this.#credential = options?.credential; if (options?.aiGatewayToken) { + const aiGatewayCredential = options.aiGatewayCredential ?? { role: 'runtime' }; this.#aiGatewayClient = new PlatformCloudflareClient(options.aiGatewayToken, { apiBase: this.apiBaseUrl, fetcher: this.#fetcher, + credential: { + ...aiGatewayCredential, + aiGatewayFor: this.#credential ?? { role: 'management' }, + }, }); } } /** Keep operator auth for infrastructure; use the existing runtime credential only for AI Gateway. */ - withAiGatewayToken(token: string): PlatformCloudflareClient { + withAiGatewayToken( + token: string, + credential?: PlatformCredentialLabel, + ): PlatformCloudflareClient { return new PlatformCloudflareClient(this.#token, { apiBase: this.apiBaseUrl, fetcher: this.#fetcher, aiGatewayToken: token, + ...(credential ? { aiGatewayCredential: credential } : {}), + ...(this.#credential ? { credential: this.#credential } : {}), }); } @@ -359,8 +451,8 @@ export class PlatformCloudflareClient { } if (!response.ok || envelope.success !== true || envelope.result === undefined) { const details = cloudflareErrorDetails(envelope, response.status); - const requestPath = path.split('?')[0]; - const error = new PlatformCloudflareApiError( + const requestPath = path.split('?')[0]!; + const error = this.#apiError( `Cloudflare API error: ${details} (HTTP ${response.status}; ${init?.method ?? 'GET'} ${requestPath}).`, response.status, requestPath, @@ -374,6 +466,24 @@ export class PlatformCloudflareClient { return { result: envelope.result, resultInfo: envelope.result_info }; } + /** + * An API error that, on 401 or 403, names the credential Cloudflare + * rejected and how to fix it. The error keeps its class, status and path: + * callers decide on those. + */ + #apiError(message: string, status: number, requestPath: string): PlatformCloudflareApiError { + const credential = + this.#credential && (status === 401 || status === 403) ? this.#credential : undefined; + return new PlatformCloudflareApiError( + credential + ? `${message} ${platformCredentialHelp(credential, status, requestPath)}` + : message, + status, + requestPath, + credential, + ); + } + async listAccounts(): Promise> { const entries: Array<{ id?: unknown; name?: unknown }> = []; for (let page = 1; ; page++) { @@ -399,11 +509,15 @@ export class PlatformCloudflareClient { `/accounts/${encodeURIComponent(accountId)}/workers/subdomain`, ); } catch (error) { - const status = error instanceof PlatformCloudflareApiError ? error.status : null; + const apiError = error instanceof PlatformCloudflareApiError ? error : undefined; + const status = apiError?.status ?? null; const reason = status === null ? 'the request failed' : `HTTP ${status}`; + // Keep the name of the rejected credential: the CLI prints only this message. const guidance = status === 401 || status === 403 - ? 'Check your Cloudflare login or API token permissions for this account, then rerun the command.' + ? apiError?.credential + ? platformCredentialHelp(apiError.credential, status, apiError.requestPath) + : 'Check your Cloudflare login or API token permissions for this account, then rerun the command.' : status === 404 ? `Open ${setupUrl} once to provision your account for Workers automatically, then retry the install.` : 'Rerun the command. If the dashboard already shows a subdomain, no Workers onboarding change is needed.'; @@ -1021,20 +1135,19 @@ export class PlatformCloudflareClient { } async getKvValue(accountId: string, namespaceId: string, key: string): Promise { - const response = await this.#fetcher( - `${this.apiBaseUrl}/accounts/${encodeURIComponent(accountId)}/storage/kv/namespaces/${encodeURIComponent(namespaceId)}/values/${encodeURIComponent(key)}`, - { - headers: { Authorization: `Bearer ${this.#token}` }, - signal: AbortSignal.timeout(30_000), - }, - ); + const path = `/accounts/${encodeURIComponent(accountId)}/storage/kv/namespaces/${encodeURIComponent(namespaceId)}/values/${encodeURIComponent(key)}`; + const response = await this.#fetcher(`${this.apiBaseUrl}${path}`, { + headers: { Authorization: `Bearer ${this.#token}` }, + signal: AbortSignal.timeout(30_000), + }); if (response.status === 404) { return null; } if (!response.ok) { - throw new PlatformCloudflareApiError( + throw this.#apiError( `Cloudflare API error reading the installation manifest (HTTP ${response.status}).`, response.status, + path, ); } return response.text(); @@ -1046,41 +1159,39 @@ export class PlatformCloudflareClient { key: string, value: string, ): Promise { - const response = await this.#fetcher( - `${this.apiBaseUrl}/accounts/${encodeURIComponent(accountId)}/r2/buckets/${encodeURIComponent(bucketName)}/objects/${encodeR2ObjectKey(key)}`, - { - body: value, - headers: { - Authorization: `Bearer ${this.#token}`, - 'Content-Type': 'application/json', - }, - method: 'PUT', - signal: AbortSignal.timeout(30_000), + const path = `/accounts/${encodeURIComponent(accountId)}/r2/buckets/${encodeURIComponent(bucketName)}/objects/${encodeR2ObjectKey(key)}`; + const response = await this.#fetcher(`${this.apiBaseUrl}${path}`, { + body: value, + headers: { + Authorization: `Bearer ${this.#token}`, + 'Content-Type': 'application/json', }, - ); + method: 'PUT', + signal: AbortSignal.timeout(30_000), + }); if (!response.ok) { - throw new PlatformCloudflareApiError( + throw this.#apiError( `Cloudflare API error writing the R2 ownership marker (HTTP ${response.status}).`, response.status, + path, ); } } async getR2Object(accountId: string, bucketName: string, key: string): Promise { - const response = await this.#fetcher( - `${this.apiBaseUrl}/accounts/${encodeURIComponent(accountId)}/r2/buckets/${encodeURIComponent(bucketName)}/objects/${encodeR2ObjectKey(key)}`, - { - headers: { Authorization: `Bearer ${this.#token}` }, - signal: AbortSignal.timeout(30_000), - }, - ); + const path = `/accounts/${encodeURIComponent(accountId)}/r2/buckets/${encodeURIComponent(bucketName)}/objects/${encodeR2ObjectKey(key)}`; + const response = await this.#fetcher(`${this.apiBaseUrl}${path}`, { + headers: { Authorization: `Bearer ${this.#token}` }, + signal: AbortSignal.timeout(30_000), + }); if (response.status === 404) { return null; } if (!response.ok) { - throw new PlatformCloudflareApiError( + throw this.#apiError( `Cloudflare API error reading the R2 ownership marker (HTTP ${response.status}).`, response.status, + path, ); } return response.text(); @@ -1168,16 +1279,14 @@ export class PlatformCloudflareClient { if (cursor) { query.set('cursor', cursor); } - const response = await this.#fetcher( - `${this.apiBaseUrl}/accounts/${encodeURIComponent(accountId)}/r2/buckets/${encodeURIComponent(bucketName)}/objects?${query}`, - { - headers: { - Accept: 'application/json', - Authorization: `Bearer ${this.#token}`, - }, - signal: AbortSignal.timeout(30_000), + const path = `/accounts/${encodeURIComponent(accountId)}/r2/buckets/${encodeURIComponent(bucketName)}/objects`; + const response = await this.#fetcher(`${this.apiBaseUrl}${path}?${query}`, { + headers: { + Accept: 'application/json', + Authorization: `Bearer ${this.#token}`, }, - ); + signal: AbortSignal.timeout(30_000), + }); if (response.status === 404) { return; } @@ -1190,9 +1299,10 @@ export class PlatformCloudflareClient { ?.map((error) => error.message) .filter(Boolean) .join(', ') || `HTTP ${response.status}`; - throw new PlatformCloudflareApiError( + throw this.#apiError( `Cloudflare API error listing R2 objects: ${details}`, response.status, + path, ); } const keys = envelope.result @@ -1281,7 +1391,7 @@ export class PlatformCloudflareClient { envelope = { success: response.ok }; } if (!response.ok || envelope.success !== true) { - throw new PlatformCloudflareApiError( + throw this.#apiError( `Cloudflare API error: ${cloudflareErrorDetails(envelope, response.status)} (HTTP ${response.status}; DELETE ${path}).`, response.status, path, diff --git a/packages/void/src/cli/platform-credential-label.ts b/packages/void/src/cli/platform-credential-label.ts new file mode 100644 index 000000000..f5a10cd6f --- /dev/null +++ b/packages/void/src/cli/platform-credential-label.ts @@ -0,0 +1,208 @@ +/** + * Names a Cloudflare credential and where it came from, so an error can say + * which credential Cloudflare rejected and how to fix it. + */ +export type PlatformCredentialLabel = Readonly<{ + /** `management` is the operator credential; `runtime` is the token the platform Workers use. */ + role: 'management' | 'runtime'; + /** Unknown when a caller does not say. */ + source?: 'environment' | 'saved' | 'oauth' | 'entered'; + /** The variable that held an `environment` credential. */ + environmentVariable?: string; + /** + * Set on the runtime token when Void uses it for AI Gateway in place of + * this operator credential. + */ + aiGatewayFor?: PlatformCredentialLabel; +}>; + +export type PlatformCredentialDenial = Readonly<{ + permission: string; + status: number; + /** The request, such as `GET /accounts/…/queues`, when it helps. */ + request?: string; +}>; + +export const PLATFORM_RUNTIME_TOKEN_VARIABLE = 'VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN'; +export const PLATFORM_AI_GATEWAY_PERMISSION = 'Account → AI Gateway → Edit'; + +/** For example `management token from CLOUDFLARE_API_TOKEN`. */ +export function describePlatformCredential(label: PlatformCredentialLabel): string { + if (label.source === 'oauth') { + // No "Wrangler": the CLI rewrites that word in printed errors. + return 'Cloudflare browser login'; + } + const kind = credentialKind(label); + switch (label.source) { + case 'environment': + return `${kind} from ${fixVariable(label)}`; + case 'saved': + return `${kind} saved on this machine`; + case 'entered': + return `${kind} you entered`; + default: + return kind; + } +} + +/** + * What to tell the operator after Cloudflare answered 401 or 403 to one + * request made with this credential. + */ +export function platformCredentialHelp( + label: PlatformCredentialLabel, + status: number, + requestPath: string | undefined, +): string { + const permission = requestPath ? cloudflarePermissionForPath(requestPath) : undefined; + if (label.aiGatewayFor) { + return `${aiGatewayStandIn(label)} ${credentialAdvice(label, status, permission ? [permission] : [])}`; + } + if (label.source === 'oauth' && permission === PLATFORM_AI_GATEWAY_PERMISSION) { + return browserLoginAiGatewayHelp(); + } + return `Void used the ${describePlatformCredential(label)}. ${credentialAdvice(label, status, permission ? [permission] : [])}`; +} + +/** + * One message for every check Cloudflare denied to one credential: the list + * of missing permissions, then what to do. + */ +export function formatPlatformCredentialDenials( + label: PlatformCredentialLabel, + denials: ReadonlyArray, +): string { + // Several checks can share one permission, status and request; list it once. + const lines = [ + ...new Set( + denials.map( + (denial) => + `- ${denial.permission} (HTTP ${denial.status}${denial.request ? `; ${denial.request}` : ''})`, + ), + ), + ]; + const help: Array = []; + if (label.aiGatewayFor) { + help.push(aiGatewayStandIn(label)); + } + // Browser login never has AI Gateway access, whatever Cloudflare answered. + const rest = + label.source === 'oauth' + ? denials.filter((denial) => denial.permission !== PLATFORM_AI_GATEWAY_PERMISSION) + : denials; + if (rest.length < denials.length) { + help.push(browserLoginAiGatewayHelp()); + } + if (rest.some((denial) => denial.status === 401)) { + help.push(credentialAdvice(label, 401, [])); + } + const forbidden = [ + ...new Set(rest.filter((denial) => denial.status !== 401).map((denial) => denial.permission)), + ]; + if (forbidden.length > 0) { + help.push(credentialAdvice(label, 403, forbidden)); + } + return `Cloudflare denied the ${describePlatformCredential(label)}:\n${lines.join('\n')}\n${help.join(' ')}`; +} + +/** The Cloudflare permission a request path needs, when Void knows it. */ +export function cloudflarePermissionForPath(requestPath: string): string | undefined { + const parts = requestPath.split('?')[0]!.split('/').filter(Boolean); + if (parts[0] === 'accounts') { + if (parts.length === 1) { + return 'Account → Account Settings → Read'; + } + const area = parts[2]; + if (area === 'ai-gateway') { + return PLATFORM_AI_GATEWAY_PERMISSION; + } + if (area === 'storage' && parts[3] === 'kv') { + return 'Account → Workers KV Storage → Edit'; + } + if (area === 'r2') { + return 'Account → Workers R2 Storage → Edit'; + } + if (area === 'queues') { + return 'Account → Queues → Edit'; + } + if (area === 'hyperdrive') { + return 'Account → Hyperdrive → Edit'; + } + if (area === 'workers') { + return 'Account → Workers Scripts → Edit'; + } + if (area === 'email' && parts[3] === 'routing') { + return 'Account → Email Routing Addresses → Edit'; + } + return undefined; + } + if (parts[0] === 'zones') { + if (parts.length <= 2) { + return 'Zone → Zone → Read'; + } + const area = parts[2]; + if (area === 'workers' && parts[3] === 'routes') { + return 'Zone → Workers Routes → Edit'; + } + if (area === 'dns_records') { + return 'Zone → DNS → Edit'; + } + if (area === 'purge_cache') { + return 'Zone → Cache Purge → Purge'; + } + if (area === 'custom_hostnames') { + return 'Zone → SSL and Certificates → Edit'; + } + if (area === 'email' && parts[3] === 'routing') { + return parts[4] === 'rules' + ? 'Zone → Email Routing Rules → Edit' + : 'Zone → Zone Settings → Edit'; + } + } + return undefined; +} + +function credentialKind(label: PlatformCredentialLabel): string { + return label.role === 'runtime' ? 'runtime token' : 'management token'; +} + +/** The variable that replaces this credential. */ +function fixVariable(label: PlatformCredentialLabel): string { + if (label.role === 'runtime') { + return PLATFORM_RUNTIME_TOKEN_VARIABLE; + } + return (label.source === 'environment' && label.environmentVariable) || 'CLOUDFLARE_API_TOKEN'; +} + +function credentialAdvice( + label: PlatformCredentialLabel, + status: number, + permissions: ReadonlyArray, +): string { + const variable = fixVariable(label); + const kind = credentialKind(label); + if (status === 401) { + return label.source === 'oauth' + ? 'HTTP 401 means the browser login is no longer valid. Run `void cloudflare login` again, or set CLOUDFLARE_API_TOKEN to a management token for this account, then retry.' + : `HTTP 401 means Cloudflare did not accept the token: it is not valid, has expired, or was replaced. Set ${variable} to a valid ${kind}, then retry.`; + } + const missing = + permissions.length > 1 + ? 'the permissions listed above' + : (permissions[0] ?? 'a permission this request needs'); + const them = permissions.length > 1 ? 'them' : 'it'; + return label.source === 'oauth' + ? `HTTP 403 means browser login does not grant ${missing}. Set CLOUDFLARE_API_TOKEN to a management token with ${them} for this account, then retry.` + : `HTTP 403 means the token works but is missing ${missing}, or is not scoped to this account or zone. Add ${them} to the token in the Cloudflare dashboard (the token value stays the same), or set ${variable} to a ${kind} that has ${them}, then retry.`; +} + +function aiGatewayStandIn(label: PlatformCredentialLabel): string { + const name = describePlatformCredential(label); + return label.aiGatewayFor?.source === 'oauth' + ? `Browser login cannot use AI Gateway, so Void used the ${name} for it. Fix the runtime token (${PLATFORM_RUNTIME_TOKEN_VARIABLE}), or set CLOUDFLARE_API_TOKEN to a management token with ${PLATFORM_AI_GATEWAY_PERMISSION}.` + : `Void used the ${name} for AI Gateway. Fix the runtime token.`; +} + +function browserLoginAiGatewayHelp(): string { + return `Browser login cannot use AI Gateway. Set ${PLATFORM_RUNTIME_TOKEN_VARIABLE} to the platform runtime token, or set CLOUDFLARE_API_TOKEN to a management token with ${PLATFORM_AI_GATEWAY_PERMISSION}, then retry.`; +} diff --git a/packages/void/src/cli/platform-domain.ts b/packages/void/src/cli/platform-domain.ts index 33d59650b..40c2fec2e 100644 --- a/packages/void/src/cli/platform-domain.ts +++ b/packages/void/src/cli/platform-domain.ts @@ -13,6 +13,7 @@ import { printSuccess, note } from './output.ts'; import type { PlatformArgs } from './platform-args.ts'; import { assertPlatformOperatorCredentialCanApplyPlan, + platformOperatorCredentialLabel, resolvePlatformCloudflareCredential, } from './platform-auth.ts'; import { PlatformCloudflareClient } from './platform-cloudflare.ts'; @@ -288,7 +289,9 @@ export async function runPlatformDomainCommand(args: DomainArgs): Promise assertNoninteractivePlatformMutationAcknowledged(args, process.stdin.isTTY === true); const selected = await resolveInstalledPlatformManifest(args.selector); const credential = await resolvePlatformCloudflareCredential(); - const client = new PlatformCloudflareClient(credential.token); + const client = new PlatformCloudflareClient(credential.token, { + credential: platformOperatorCredentialLabel(credential), + }); const domainInput = args.domain || (await text({ message: 'Application domain:', placeholder: 'example.com' })); if (isCancel(domainInput)) { diff --git a/packages/void/src/cli/platform-installer-secrets.ts b/packages/void/src/cli/platform-installer-secrets.ts index 3aa118067..813785477 100644 --- a/packages/void/src/cli/platform-installer-secrets.ts +++ b/packages/void/src/cli/platform-installer-secrets.ts @@ -45,6 +45,18 @@ export function createPlatformInstallationSecrets(input: { checkpoint?: Record | null; }): PlatformInstallationSecrets { const checkpoint = input.checkpoint ?? {}; + // The two halves are one key pair. Replacing one half would leave the + // Workers signing with a mismatched pair. + if (!input.r2AccessKeyId !== !input.r2SecretAccessKey) { + const [given, missing] = input.r2AccessKeyId + ? ['VOID_PLATFORM_R2_ACCESS_KEY_ID', 'VOID_PLATFORM_R2_SECRET_ACCESS_KEY'] + : ['VOID_PLATFORM_R2_SECRET_ACCESS_KEY', 'VOID_PLATFORM_R2_ACCESS_KEY_ID']; + throw new Error( + `platform: ${given} is set but ${missing} is not. ` + + 'Set VOID_PLATFORM_R2_ACCESS_KEY_ID and VOID_PLATFORM_R2_SECRET_ACCESS_KEY together, ' + + 'even if only the secret changed.', + ); + } const required = (value: string, name: string): string => { if (!value) { throw new Error(`platform: ${name} is required to install the platform.`); diff --git a/packages/void/src/cli/platform-installer-workers.ts b/packages/void/src/cli/platform-installer-workers.ts index 7ebf1955d..2c5c3f491 100644 --- a/packages/void/src/cli/platform-installer-workers.ts +++ b/packages/void/src/cli/platform-installer-workers.ts @@ -799,6 +799,9 @@ export async function deployPlatformWorker( await options.onStaged(previousVersionId, uploadedVersionId, previousTriggers); await options.onReadyForActivation?.(); await options.waitForPriorWorker?.(); + // Deployments already listed before activation. A stale read can return + // one of these at the head; it is not evidence of a newer deployment. + const knownDeploymentIds = new Set(); if (options.workerExisted) { const deploymentsBeforeActivation = await readPlatformWorkerDeployments( configPath, @@ -814,6 +817,9 @@ export async function deployPlatformWorker( `platform: Worker ${workerName} changed concurrently before activation; refusing to overwrite it.`, ); } + for (const deployment of deploymentsBeforeActivation) { + knownDeploymentIds.add(deployment.id); + } activatedVersionId = uploadedVersionId; const activated = await options.runner( { @@ -832,9 +838,14 @@ export async function deployPlatformWorker( readPlatformWorkerDeployments(configPath, accountId, workerName, readRunner); let deploymentsAfterActivation = await readDeploymentHistory(); // A successful activation may not be visible in history yet. Only wait - // while the same deployment we replaced is still at the head. + // while the list is empty or its head is a deployment we listed before + // activation. A head we have never seen is a real newer deployment. + const headIsStale = () => { + const head = deploymentsAfterActivation[0]; + return head === undefined || knownDeploymentIds.has(head.id); + }; for (const wait of [250, 500, 1_000]) { - if (deploymentsAfterActivation[0]?.id !== previousDeploymentId) { + if (!headIsStale()) { break; } await delay(wait); @@ -848,12 +859,19 @@ export async function deployPlatformWorker( ); } } - if (deploymentsAfterActivation[0]?.id === previousDeploymentId) { + if (headIsStale()) { // Neither the new deployment nor a competitor has been confirmed. // Leave the staged cutover for recovery rather than trying to roll it back. + const head = deploymentsAfterActivation[0]; activatedVersionId = null; throw new PlatformWorkerRecoveryIncompleteError( - `platform: Void could not confirm activation of ${workerName} because Cloudflare still reports the previous deployment.`, + `platform: Void could not confirm activation of ${workerName} because Cloudflare ${ + head === undefined + ? 'lists no deployments for it yet' + : head.id === previousDeploymentId + ? 'still reports the previous deployment' + : `still reports older deployment ${head.id}` + }.`, ); } const lineage = classifyPlatformWorkerDeploymentLineage( diff --git a/packages/void/src/cli/platform-lifecycle.ts b/packages/void/src/cli/platform-lifecycle.ts index db56ab540..496b63b40 100644 --- a/packages/void/src/cli/platform-lifecycle.ts +++ b/packages/void/src/cli/platform-lifecycle.ts @@ -27,6 +27,7 @@ import { } from './platform-auth-database.ts'; import { assertPlatformOperatorCredentialCanApplyPlan, + platformOperatorCredentialLabel, platformOperatorNeedsManagementToken, verifyPlatformManagementAccess, resolvePlatformCloudflareCredential, @@ -40,6 +41,7 @@ import { verifyPlatformRuntimeTokenPermissions, type CloudflareAccount, } from './platform-cloudflare.ts'; +import { platformCredentialHelp } from './platform-credential-label.ts'; import { selectInstallationAuthentication, collectInstallationAuthentication, @@ -163,7 +165,9 @@ export async function runPlatformLifecycleCommand(commandArgs: PlatformArgs): Pr if (commandArgs.subcommand === 'discover') { log.step('Preparing platform discovery'); const credential = await resolvePlatformCloudflareCredential(); - const client = new PlatformCloudflareClient(credential.token); + const client = new PlatformCloudflareClient(credential.token, { + credential: platformOperatorCredentialLabel(credential), + }); const accounts = await withPlatformProgress('Reading Cloudflare accounts', () => readAccounts(client), ); @@ -378,7 +382,9 @@ export async function runPlatformLifecycleCommand(commandArgs: PlatformArgs): Pr } return existing.deployment.installationSlug; }); - let client = new PlatformCloudflareClient(operatorCredential.token); + let client = new PlatformCloudflareClient(operatorCredential.token, { + credential: platformOperatorCredentialLabel(operatorCredential), + }); const accounts = await withPlatformProgress('Reading Cloudflare accounts', () => readAccounts(client), ); @@ -904,6 +910,8 @@ export async function runPlatformLifecycleCommand(commandArgs: PlatformArgs): Pr operatorCredential.source === 'oauth' ? checkpoint?.managementApiToken || '' : ''; if (needsManagementToken || managementApiToken) { let token = managementApiToken; + // A token from the checkpoint was saved on this machine by an earlier run. + let tokenSource: 'saved' | 'entered' = token ? 'saved' : 'entered'; let openedManagementPage = false; const forgetRejectedManagementToken = () => { managementApiToken = ''; @@ -926,8 +934,12 @@ export async function runPlatformLifecycleCommand(commandArgs: PlatformArgs): Pr value.trim() ? undefined : 'A value is required.', ) ).trim(); + tokenSource = 'entered'; } - const management = new PlatformCloudflareClient(token, { apiBase: client.apiBaseUrl }); + const management = new PlatformCloudflareClient(token, { + apiBase: client.apiBaseUrl, + credential: { role: 'management', source: tokenSource }, + }); try { const authorizedAccounts = await management.listAccounts(); if (!authorizedAccounts.some((account) => account.id === accountId)) { @@ -955,13 +967,27 @@ export async function runPlatformLifecycleCommand(commandArgs: PlatformArgs): Pr if (!isPlatformTokenAuthorizationError(error)) { throw error; } - const details = new Set(); + const details: Array = []; + // Errors an AggregateError above already lists in its own message. + const listed = new Set(); for ( let cause = error, depth = 0; depth < 5 && cause instanceof Error; depth++, cause = cause.cause ) { - details.add(cause.message.replaceAll(token, '[redacted]')); + if (listed.has(cause)) { + continue; + } + if (cause instanceof AggregateError) { + for (const listedError of cause.errors) { + listed.add(listedError); + } + } + const message = cause.message.replaceAll(token, '[redacted]'); + // A wrapping error often repeats its cause's message. + if (!details.some((detail) => detail.includes(message))) { + details.push(message); + } } forgetRejectedManagementToken(); log.warn( @@ -1052,19 +1078,32 @@ export async function runPlatformLifecycleCommand(commandArgs: PlatformArgs): Pr let rejectedSavedRuntimeToken = false; const verifyRuntimeToken = async (token: string) => { let checkingRuntimePermissions = true; + const credential = { + role: 'runtime', + source: + token === process.env.VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN + ? 'environment' + : token === values.cloudflareApiToken + ? 'saved' + : 'entered', + } as const; try { - await verifyPlatformRuntimeTokenPermissions(new PlatformCloudflareClient(token), { - accountId, - applicationDomain, - customDomains: plan.deployment.capabilities.customDomains, - ...(plan.deployment.capabilities.email && plan.deployment.emailSharedZoneId - ? { emailZoneId: plan.deployment.emailSharedZoneId } - : {}), - zoneId: plan.zone?.id ?? null, - }); + await verifyPlatformRuntimeTokenPermissions( + new PlatformCloudflareClient(token, { credential }), + { + accountId, + applicationDomain, + credential, + customDomains: plan.deployment.capabilities.customDomains, + ...(plan.deployment.capabilities.email && plan.deployment.emailSharedZoneId + ? { emailZoneId: plan.deployment.emailSharedZoneId } + : {}), + zoneId: plan.zone?.id ?? null, + }, + ); checkingRuntimePermissions = false; if (plan.actions.some((action) => action.action === 'inspect')) { - client = client.withAiGatewayToken(token); + client = client.withAiGatewayToken(token, credential); plan = await verifyPlatformAiGatewayPlan( client, planInput, @@ -1513,16 +1552,31 @@ export async function runPlatformLifecycleCommand(commandArgs: PlatformArgs): Pr } function isPlatformTokenAuthorizationError(error: unknown, requestPathIncludes?: string): boolean { + return !!findPlatformTokenAuthorizationError(error, requestPathIncludes); +} + +function findPlatformTokenAuthorizationError( + error: unknown, + requestPathIncludes?: string, +): PlatformCloudflareApiError | undefined { for (let depth = 0; depth < 5 && error instanceof Error; depth++, error = error.cause) { if ( error instanceof PlatformCloudflareApiError && [401, 403].includes(error.status) && (!requestPathIncludes || error.requestPath?.includes(requestPathIncludes)) ) { - return true; + return error; } } - return false; + return undefined; +} + +/** What the client said about the AI Gateway credential it used, if it knew it. */ +function platformCredentialHelpInChain(error: unknown): string | undefined { + const denied = findPlatformTokenAuthorizationError(error, '/ai-gateway/'); + return denied?.credential + ? platformCredentialHelp(denied.credential, denied.status, denied.requestPath) + : undefined; } function formatPlatformRuntime(runtime: ReturnType): string { @@ -1551,8 +1605,12 @@ export async function verifyPlatformAiGatewayPlan( }); } catch (error) { if (isPlatformTokenAuthorizationError(error, '/ai-gateway/')) { + // The client already named the token and how to fix it. + const help = platformCredentialHelpInChain(error); throw new Error( - 'platform: The runtime token could not verify AI Gateway access. Include Account → AI Gateway → Edit for this account, then retry. No platform resources were created.', + help + ? `platform: The runtime token could not verify AI Gateway access. ${help} No platform resources were created.` + : 'platform: The runtime token could not verify AI Gateway access. Include Account → AI Gateway → Edit for this account, then retry. No platform resources were created.', { cause: error }, ); } diff --git a/packages/void/src/cli/platform-management.ts b/packages/void/src/cli/platform-management.ts index e587c09a1..0eccb35d3 100644 --- a/packages/void/src/cli/platform-management.ts +++ b/packages/void/src/cli/platform-management.ts @@ -21,6 +21,7 @@ import type { PlatformArgs } from './platform-args.ts'; import { platformDatabaseNameFromOwnerUrl } from './platform-auth-database.ts'; import { assertPlatformOperatorCredentialCanApplyPlan, + platformOperatorCredentialLabel, resolvePlatformCloudflareCredential, verifyPlatformCloudflareCredential, } from './platform-auth.ts'; @@ -73,6 +74,10 @@ import { revalidatePlatformInstallAction, } from './platform-plan.ts'; import { createPlatformProgress, withPlatformProgress } from './platform-progress.ts'; +import { + type PlatformR2CredentialCheck, + verifyPlatformR2Credentials, +} from './platform-r2-credentials.ts'; import { acquirePlatformLifecycleLock, listInstallationManifests, @@ -273,7 +278,10 @@ export function platformLifecycleResumeCommand( return command.join(' '); } -export async function runInstalledPlatformCommand(args: PlatformArgs): Promise { +export async function runInstalledPlatformCommand( + args: PlatformArgs, + dependencies: { fetch?: typeof fetch } = {}, +): Promise { if ( args.subcommand !== 'upgrade' && args.subcommand !== 'rollback' && @@ -302,11 +310,16 @@ export async function runInstalledPlatformCommand(args: PlatformArgs): Promise verifyPlatformCloudflareCredential(operatorClient, { account: manifest.account, source: operatorCredential.source, + ...(operatorCredential.environmentVariable + ? { environmentVariable: operatorCredential.environmentVariable } + : {}), r2Buckets: args.subcommand === 'disable' || args.subcommand === 'uninstall' ? [] @@ -337,9 +350,15 @@ export async function runInstalledPlatformCommand(args: PlatformArgs): Promise; + }, + secrets: Pick< + ReturnType, + 'r2AccessKeyId' | 'r2SecretAccessKey' | 'workerSecretUpdates' + >, +): Omit | undefined { + const bucket = plan.actions.find( + (action) => action.key === 'assets-bucket' && action.kind === 'r2-bucket', + ); + if (!bucket || bucket.action === 'create') { + return undefined; + } + const accessKeyId = secrets.workerSecretUpdates?.R2_ACCESS_KEY_ID; + const secretAccessKey = secrets.workerSecretUpdates?.R2_SECRET_ACCESS_KEY; + if (accessKeyId && secretAccessKey) { + return { bucket: bucket.name, accessKeyId, secretAccessKey, source: 'environment' }; + } + const createsApiWorker = plan.actions.some( + (action) => + action.kind === 'worker' && action.action === 'create' && action.key === 'api-worker', + ); + return createsApiWorker && secrets.r2AccessKeyId && secrets.r2SecretAccessKey + ? { + bucket: bucket.name, + accessKeyId: secrets.r2AccessKeyId, + secretAccessKey: secrets.r2SecretAccessKey, + source: 'saved', + } + : undefined; +} + export async function resolveInstalledPlatformManifest( selector: string, dependencies: { diff --git a/packages/void/src/cli/platform-plan.ts b/packages/void/src/cli/platform-plan.ts index 956fadeb8..3cf576475 100644 --- a/packages/void/src/cli/platform-plan.ts +++ b/packages/void/src/cli/platform-plan.ts @@ -33,6 +33,11 @@ import type { CloudflareZone, PlatformCloudflareClient, } from './platform-cloudflare.ts'; +import { + cloudflarePermissionForPath, + formatPlatformCredentialDenials, + type PlatformCredentialLabel, +} from './platform-credential-label.ts'; import type { PlatformStorageLocation } from './platform-storage.ts'; export type PlatformInstallPlanAction = Readonly<{ @@ -108,6 +113,34 @@ type PlanningClient = Pick< | 'listWorkerRoutes' >; +/** Group denied plan reads by the credential Cloudflare rejected. */ +function formatPlanDenials(denied: ReadonlyArray): Array { + const groups = new Map< + string, + { label?: PlatformCredentialLabel; errors: Array } + >(); + for (const error of denied) { + const key = error.credential ? JSON.stringify(error.credential) : ''; + const group = groups.get(key) ?? { label: error.credential, errors: [] }; + group.errors.push(error); + groups.set(key, group); + } + return [...groups.values()].map(({ label, errors }) => + label + ? formatPlatformCredentialDenials( + label, + errors.map((error) => ({ + permission: + (error.requestPath ? cloudflarePermissionForPath(error.requestPath) : undefined) ?? + 'a permission this read needs', + request: `GET ${error.requestPath ?? ''}`, + status: error.status, + })), + ) + : [...new Set(errors.map((error) => error.message))].join('\n'), + ); +} + export async function createPlatformInstallPlan( client: PlanningClient, input: PlatformInstallPlanInput, @@ -390,13 +423,23 @@ export async function createPlatformInstallPlan( throw error; }); const prerequisites: Array = []; - const collectPrerequisite = (error: unknown): [] => { - if (!(error instanceof PlatformCloudflarePrerequisiteError)) { - throw error; - } - prerequisites.push(error); - return []; - }; + // Collect every denied read in request order, so one error names each + // missing permission. Any other failure still stops the plan. + const denials: Array = []; + const settle = (slot: number, read: Promise>): Promise> => + read.catch((error: unknown) => { + if (error instanceof PlatformCloudflarePrerequisiteError) { + prerequisites.push(error); + } else if ( + error instanceof PlatformCloudflareApiError && + (error.status === 401 || error.status === 403) + ) { + denials[slot] = error; + } else { + throw error; + } + return []; + }); const [ aiGateways, routingKv, @@ -410,21 +453,36 @@ export async function createPlatformInstallPlan( hyperdrives, workers, ] = await Promise.all([ - gatewayInventory, - client.listKvNamespaces(account.id, names.routingKv), - client.listKvNamespaces(account.id, names.isrCacheKv), - client.listKvNamespaces(account.id, names.legacyAssetsKv), - client.listR2Buckets(account.id).catch(collectPrerequisite), - client.listQueues(account.id, names.usageQueue), - client.listQueues(account.id, names.cronQueue), + settle(0, gatewayInventory), + settle(1, client.listKvNamespaces(account.id, names.routingKv)), + settle(2, client.listKvNamespaces(account.id, names.isrCacheKv)), + settle(3, client.listKvNamespaces(account.id, names.legacyAssetsKv)), + settle(4, client.listR2Buckets(account.id)), + settle(5, client.listQueues(account.id, names.usageQueue)), + settle(6, client.listQueues(account.id, names.cronQueue)), (options?.deploymentExecution ?? existing?.resources.some((resource) => resource.key === 'deployment-queue')) - ? client.listQueues(account.id, names.deploymentQueue) + ? settle(7, client.listQueues(account.id, names.deploymentQueue)) : Promise.resolve([]), - client.listDispatchNamespaces(account.id).catch(collectPrerequisite), - client.listHyperdriveConfigs(account.id), - client.listWorkerNames(account.id), + settle(8, client.listDispatchNamespaces(account.id)), + settle(9, client.listHyperdriveConfigs(account.id)), + settle(10, client.listWorkerNames(account.id)), ]); + const denied = denials.filter((error) => error !== undefined); + if (denied.length > 0) { + if (denied.length === 1 && prerequisites.length === 0) { + throw denied[0]; + } + throw new AggregateError( + [...denied, ...prerequisites], + `platform: Cloudflare denied reads the plan needs.\n\n${[ + ...formatPlanDenials(denied), + ...prerequisites.map((error) => error.message), + ].join('\n\n')}`, + // Callers check the first denied request for its status and path. + { cause: denied[0] }, + ); + } if (prerequisites.length === 1) { throw prerequisites[0]; } diff --git a/packages/void/src/cli/platform-r2-credentials.ts b/packages/void/src/cli/platform-r2-credentials.ts new file mode 100644 index 000000000..cc5b5b0d8 --- /dev/null +++ b/packages/void/src/cli/platform-r2-credentials.ts @@ -0,0 +1,144 @@ +import { createHash, createHmac } from 'node:crypto'; +import { log } from '@voidzero-dev/vite-plus-prompts'; +import { + isR2CredentialRejection, + R2_CREDENTIALS_DOCS_URL, + R2_MALFORMED_CREDENTIAL_ERROR_CODES, + r2ErrorCode, +} from './direct-r2-upload.ts'; + +/** Where the R2 key pair under test came from, for the error message. */ +export type PlatformR2CredentialSource = 'environment' | 'saved'; + +export type PlatformR2CredentialCheck = Readonly<{ + accountId: string; + bucket: string; + accessKeyId: string; + secretAccessKey: string; + source: PlatformR2CredentialSource; +}>; + +const R2_REGION = 'auto'; +const R2_SERVICE = 's3'; +const EMPTY_BODY_SHA256 = createHash('sha256').update('').digest('hex'); +const R2_PROBE_TIMEOUT_MS = 15_000; + +/** + * Build one SigV4-signed S3 ListObjectsV2 request (at most one key) for the + * bucket. The platform API signs upload links with the same pair, region and + * service, so R2 accepting this signature means it accepts the pair. + */ +export function createR2ListObjectsRequest( + input: Pick< + PlatformR2CredentialCheck, + 'accountId' | 'bucket' | 'accessKeyId' | 'secretAccessKey' + >, + now: Date, +): Readonly<{ url: string; headers: Record }> { + const host = `${input.accountId}.r2.cloudflarestorage.com`; + const path = `/${encodeURIComponent(input.bucket)}`; + // Keys are already in sorted order, as SigV4 requires. + const query = 'list-type=2&max-keys=1'; + const amzDate = now.toISOString().replaceAll('-', '').replaceAll(':', '').slice(0, 15) + 'Z'; + const date = amzDate.slice(0, 8); + const headers = { + host, + 'x-amz-content-sha256': EMPTY_BODY_SHA256, + 'x-amz-date': amzDate, + }; + const signedHeaders = Object.keys(headers).join(';'); + const canonicalRequest = [ + 'GET', + path, + query, + ...Object.entries(headers).map(([name, value]) => `${name}:${value}`), + '', + signedHeaders, + EMPTY_BODY_SHA256, + ].join('\n'); + const scope = `${date}/${R2_REGION}/${R2_SERVICE}/aws4_request`; + const stringToSign = [ + 'AWS4-HMAC-SHA256', + amzDate, + scope, + createHash('sha256').update(canonicalRequest).digest('hex'), + ].join('\n'); + let key: Buffer = hmac(`AWS4${input.secretAccessKey}`, date); + for (const part of [R2_REGION, R2_SERVICE, 'aws4_request']) { + key = hmac(key, part); + } + const signature = createHmac('sha256', key).update(stringToSign).digest('hex'); + return { + url: `https://${host}${path}?${query}`, + headers: { + authorization: `AWS4-HMAC-SHA256 Credential=${input.accessKeyId}/${scope}, SignedHeaders=${signedHeaders}, Signature=${signature}`, + 'x-amz-content-sha256': EMPTY_BODY_SHA256, + 'x-amz-date': amzDate, + }, + }; +} + +function hmac(key: string | Buffer, data: string): Buffer { + return createHmac('sha256', key).update(data).digest(); +} + +/** + * Check that R2 accepts the platform's R2 key pair before it is sent to the + * Workers. A rejected or malformed key pair stops the command. A clock that R2 + * finds too far off, a network error, an R2 outage or any other answer only + * warns, so a check that cannot run never blocks the command. + */ +export async function verifyPlatformR2Credentials( + input: PlatformR2CredentialCheck, + fetchImpl: typeof fetch = fetch, + options: { now?: () => Date; warn?: (message: string) => void } = {}, +): Promise { + const from = + input.source === 'environment' + ? 'from the VOID_PLATFORM_R2_* environment variables' + : 'saved on this machine'; + const warn = options.warn ?? ((message: string) => log.warn(message)); + const skip = (reason: string) => + warn( + `Void could not check the R2 key pair ${from} on bucket ${input.bucket} (${reason}). It continues without this check.`, + ); + const request = createR2ListObjectsRequest(input, options.now?.() ?? new Date()); + let response: Response; + try { + response = await fetchImpl(request.url, { + method: 'GET', + headers: request.headers, + signal: AbortSignal.timeout(R2_PROBE_TIMEOUT_MS), + }); + } catch (error) { + skip(error instanceof Error ? error.message : String(error)); + return; + } + if (response.ok) { + return; + } + const code = r2ErrorCode(await response.text().catch(() => '')); + // R2 checks the signature before it looks up the bucket. + if (response.status === 404 && code === 'NoSuchBucket') { + return; + } + const denied = response.status === 401 || response.status === 403; + if ((denied && !code) || isR2CredentialRejection(response.status, code)) { + const malformed = code !== undefined && R2_MALFORMED_CREDENTIAL_ERROR_CODES.has(code); + throw new Error( + `platform: R2 rejected the R2 key pair ${from} (${code ?? `HTTP ${response.status}`}) on bucket ${input.bucket}. ` + + 'Void did not change the platform. ' + + (malformed ? 'The Access Key ID is the 32-character token ID, not the token value. ' : '') + + 'Create an R2 API token with Object Read & Write on this bucket. ' + + 'Then set VOID_PLATFORM_R2_ACCESS_KEY_ID to its Access Key ID and VOID_PLATFORM_R2_SECRET_ACCESS_KEY to its Secret Access Key (not the token value), and run this command again. ' + + `See ${R2_CREDENTIALS_DOCS_URL}`, + ); + } + if (code === 'RequestTimeTooSkewed') { + skip( + `HTTP ${response.status} ${code}: the clock on this machine is likely wrong; set it to the correct time`, + ); + return; + } + skip(`HTTP ${response.status}${code ? ` ${code}` : ''}`); +} diff --git a/packages/void/test/unit/direct-r2-deploy.test.ts b/packages/void/test/unit/direct-r2-deploy.test.ts index bbb0beb26..cbf45598b 100644 --- a/packages/void/test/unit/direct-r2-deploy.test.ts +++ b/packages/void/test/unit/direct-r2-deploy.test.ts @@ -501,6 +501,22 @@ describe('direct-to-R2 deploy flow', () => { await expect(deploy()).rejects.toThrow(/R2 PUT failed.+SignatureMismatch/); }); + it('names the platform R2 credentials when R2 rejects the upload signature', async () => { + const state = installPlatformMocks({ + r2PutBehavior() { + return new Response( + 'SignatureDoesNotMatchx', + { status: 403 }, + ); + }, + }); + + await expect(deploy()).rejects.toThrow( + /R2 rejected the upload signature \(SignatureDoesNotMatch\).+VOID_PLATFORM_R2_ACCESS_KEY_ID.+void platform repair/, + ); + expect(state.finalizeCalls).toHaveLength(0); + }); + it('writes presign + r2_put + finalize events to the JSONL log', async () => { installPlatformMocks(); const cliLog = openDeployLog(); diff --git a/packages/void/test/unit/direct-r2-upload.test.ts b/packages/void/test/unit/direct-r2-upload.test.ts index 6531086b4..d7c8095c4 100644 --- a/packages/void/test/unit/direct-r2-upload.test.ts +++ b/packages/void/test/unit/direct-r2-upload.test.ts @@ -1,6 +1,7 @@ import { beforeEach, describe, expect, it, vi } from 'vite-plus/test'; import { DirectR2PutError, + r2ErrorCode, uploadAssetsToR2, type DirectR2PutItem, } from '../../src/cli/direct-r2-upload.ts'; @@ -113,6 +114,72 @@ describe('uploadAssetsToR2', () => { expect(fetchMock).toHaveBeenCalledTimes(1); }); + const r2ErrorXml = (code: string) => + `${code}details`; + + it.each([ + [403, 'SignatureDoesNotMatch'], + [403, 'AccessDenied'], + [403, 'InvalidAccessKeyId'], + [401, 'Unauthorized'], + [400, 'InvalidArgument'], + [400, 'AuthorizationHeaderMalformed'], + ])('names the platform R2 credentials for HTTP %s %s', async (status, code) => { + const fetchMock = vi.fn().mockResolvedValue(new Response(r2ErrorXml(code), { status })); + + const error = await uploadAssetsToR2([makeItem('/a.js', 'h-a')], cliLog, fetchMock).catch( + (err: unknown) => err, + ); + + expect(error).toBeInstanceOf(DirectR2PutError); + const putError = error as DirectR2PutError; + expect(putError.code).toBe(code); + expect(putError.status).toBe(status); + expect(putError.body).toBe(r2ErrorXml(code)); + expect(putError.message).toContain(`R2 rejected the upload signature (${code}) for /a.js`); + expect(putError.message).toContain('Ask the platform operator'); + expect(putError.message).toContain('VOID_PLATFORM_R2_ACCESS_KEY_ID'); + expect(putError.message).toContain('VOID_PLATFORM_R2_SECRET_ACCESS_KEY'); + expect(putError.message).toContain('void platform repair'); + expect(putError.message).toContain( + 'https://void.cloud/guide/platform/installation/credentials#r2-upload-credentials', + ); + expect(fetchMock).toHaveBeenCalledTimes(1); + // The raw R2 body stays in the JSONL log. + const end = cliLog.error.mock.calls.find(([msg]) => msg === 'r2_put_end'); + expect(end?.[2]).toMatchObject({ status, body: r2ErrorXml(code) }); + }); + + it.each([ + [403, r2ErrorXml('ExpiredRequest')], + [403, r2ErrorXml('SignatureExpired')], + [403, r2ErrorXml('RequestTimeTooSkewed')], + [400, r2ErrorXml('BadDigest')], + [400, r2ErrorXml('SignatureDoesNotMatch')], + [403, 'SignatureMismatch'], + ])('keeps the raw R2 response for HTTP %s %s', async (status, body) => { + const fetchMock = vi.fn().mockResolvedValue(new Response(body, { status })); + + const error = (await uploadAssetsToR2([makeItem('/a.js', 'h-a')], cliLog, fetchMock).catch( + (err: unknown) => err, + )) as DirectR2PutError; + + expect(error.message).toBe(`R2 PUT failed for /a.js (h-a): ${status} ${body}`); + expect(error.message).not.toContain('VOID_PLATFORM_R2'); + expect(fetchMock).toHaveBeenCalledTimes(1); + }); + + it.each([ + ['', undefined], + ['SignatureMismatch', undefined], + ['AccessDenied', undefined], + ['', undefined], + [' AccessDenied ', 'AccessDenied'], + ['NoSuchBucketx', 'NoSuchBucket'], + ])('reads the S3 error code from %j', (body, code) => { + expect(r2ErrorCode(body)).toBe(code); + }); + it.each([429, 500])( 'throws DirectR2PutError when HTTP %s exceeds retry budget', async (status) => { diff --git a/packages/void/test/unit/platform-auth-protection.test.ts b/packages/void/test/unit/platform-auth-protection.test.ts index 46d63497c..ae4e6eebd 100644 --- a/packages/void/test/unit/platform-auth-protection.test.ts +++ b/packages/void/test/unit/platform-auth-protection.test.ts @@ -57,6 +57,7 @@ vi.mock('../../src/cli/platform-access-setup.ts', async (importOriginal) => { }); vi.mock('../../src/cli/platform-auth.ts', () => ({ + platformOperatorCredentialLabel: () => ({ role: 'management' }), resolvePlatformCloudflareCredential: vi.fn(async () => ({ token: 'cloudflare-token' })), })); diff --git a/packages/void/test/unit/platform-auth.test.ts b/packages/void/test/unit/platform-auth.test.ts index a4f8990dd..8bfb0e116 100644 --- a/packages/void/test/unit/platform-auth.test.ts +++ b/packages/void/test/unit/platform-auth.test.ts @@ -9,7 +9,14 @@ import { verifyPlatformManagementAccess, verifyPlatformCloudflareCredential, } from '../../src/cli/platform-auth.ts'; -import { PlatformCloudflareClient } from '../../src/cli/platform-cloudflare.ts'; +import { + PlatformCloudflareApiError, + PlatformCloudflareClient, +} from '../../src/cli/platform-cloudflare.ts'; + +const expectedRejection = (): never => { + throw new Error('expected a rejection'); +}; function deferred(): { promise: Promise; resolve: (value: T) => void } { let resolve!: (value: T) => void; @@ -90,7 +97,7 @@ describe('platform Cloudflare authentication', () => { r2Buckets: ['assets'], }).catch((error) => error); expect(error.message).toContain(`HTTP ${status}`); - expect(error.message).toContain('saved management token'); + expect(error.message).toContain('management token saved on this machine'); expect(error.message).not.toContain('secret-token'); expect(error.message).toContain( status === 401 || status === 403 ? 'Workers R2 Storage → Edit' : 'Retry', @@ -98,6 +105,104 @@ describe('platform Cloudflare authentication', () => { }, ); + it('lists every denied permission in one error that names the credential', async () => { + const paths: Array = []; + const client = new PlatformCloudflareClient('token', { + fetcher: async (url) => { + const path = new URL(String(url)).pathname; + paths.push(path); + if (path.endsWith('/accounts')) { + return Response.json({ success: true, result: [{ id: 'account', name: 'Account' }] }); + } + return Response.json( + { success: false, errors: [{ message: 'Authentication error' }] }, + { status: 403 }, + ); + }, + }); + const error = await verifyPlatformCloudflareCredential(client, { + account: { id: 'account', name: 'Account' }, + source: 'environment', + environmentVariable: 'CF_API_TOKEN', + r2Buckets: ['assets', 'cache'], + }).then(expectedRejection, (error: Error) => error); + expect(paths).toEqual([ + '/client/v4/accounts', + '/client/v4/accounts/account/workers/scripts', + '/client/v4/accounts/account/r2/buckets/assets', + '/client/v4/accounts/account/r2/buckets/cache', + ]); + expect(error).toBeInstanceOf(Error); + expect(error.message).toBe( + [ + 'platform: Cloudflare denied the management token from CF_API_TOKEN:', + '- Account → Workers Scripts → Edit (HTTP 403; Cloudflare Workers access)', + '- Account → Workers R2 Storage → Edit (HTTP 403; R2 bucket "assets")', + '- Account → Workers R2 Storage → Edit (HTTP 403; R2 bucket "cache")', + 'HTTP 403 means the token works but is missing the permissions listed above, or is not scoped to this account or zone. Add them to the token in the Cloudflare dashboard (the token value stays the same), or set CF_API_TOKEN to a management token that has them, then retry.', + ].join('\n'), + ); + // Callers read the status and path of the first denied request. + expect(error.cause).toBeInstanceOf(PlatformCloudflareApiError); + expect((error.cause as PlatformCloudflareApiError).status).toBe(403); + }); + + it('keeps a denial seen before a later check fails for another reason', async () => { + const client = new PlatformCloudflareClient('token', { + fetcher: async (url) => { + const path = new URL(String(url)).pathname; + if (path.endsWith('/accounts')) { + return Response.json({ success: true, result: [{ id: 'account', name: 'Account' }] }); + } + return path.includes('/r2/') + ? Response.json({ success: false, errors: [{ message: 'Slow down' }] }, { status: 429 }) + : Response.json( + { success: false, errors: [{ message: 'Authentication error' }] }, + { status: 403 }, + ); + }, + }); + const error = await verifyPlatformCloudflareCredential(client, { + account: { id: 'account', name: 'Account' }, + source: 'saved', + r2Buckets: ['assets'], + }).then(expectedRejection, (error: Error) => error); + expect(error.message).toBe( + [ + 'platform: Cloudflare denied the management token saved on this machine:', + '- Account → Workers Scripts → Edit (HTTP 403; Cloudflare Workers access)', + 'HTTP 403 means the token works but is missing Account → Workers Scripts → Edit, or is not scoped to this account or zone. Add it to the token in the Cloudflare dashboard (the token value stays the same), or set CLOUDFLARE_API_TOKEN to a management token that has it, then retry.', + ].join('\n'), + ); + expect(error.cause).toBeInstanceOf(PlatformCloudflareApiError); + expect((error.cause as PlatformCloudflareApiError).status).toBe(403); + }); + + it('keeps checking after the account list is denied, and says a 401 token is not valid', async () => { + const client = { + listAccounts: async () => { + throw new PlatformCloudflareApiError('Authentication error', 401); + }, + listWorkerNames: vi.fn(async () => { + throw new PlatformCloudflareApiError('Authentication error', 401); + }), + request: vi.fn(), + }; + const error = await verifyPlatformCloudflareCredential(client as never, { + account: { id: 'account', name: 'Account' }, + source: 'saved', + r2Buckets: [], + }).then(expectedRejection, (error: Error) => error); + expect(client.listWorkerNames).toHaveBeenCalledOnce(); + expect(error.message).toContain('Cloudflare denied the management token saved on this machine'); + expect(error.message).toContain('- Account → Account Settings → Read (HTTP 401'); + expect(error.message).toContain('- Account → Workers Scripts → Edit (HTTP 401'); + expect(error.message).toContain( + 'HTTP 401 means Cloudflare did not accept the token: it is not valid, has expired, or was replaced. Set CLOUDFLARE_API_TOKEN to a valid management token, then retry.', + ); + expect(error.message).not.toContain('HTTP 403 means'); + }); + it.each([false, true])( 'only permits a missing bucket when repairing (repair: %s)', async (repair) => { @@ -250,7 +355,11 @@ describe('platform Cloudflare authentication', () => { runWrangler, savedManagementToken: 'saved-manager', }), - ).resolves.toEqual({ source: 'environment', token: 'explicit-manager' }); + ).resolves.toEqual({ + source: 'environment', + environmentVariable: name, + token: 'explicit-manager', + }); expect(runWrangler).not.toHaveBeenCalled(); }, ); @@ -406,7 +515,14 @@ describe('platform authentication during input', () => { interactive: false, runWrangler, }), - ).resolves.toEqual({ input: 'team', credential: { source: 'environment', token: 'ci-token' } }); + ).resolves.toEqual({ + input: 'team', + credential: { + source: 'environment', + environmentVariable: 'CLOUDFLARE_API_TOKEN', + token: 'ci-token', + }, + }); expect(runWrangler).not.toHaveBeenCalled(); }); }); diff --git a/packages/void/test/unit/platform-cloudflare.test.ts b/packages/void/test/unit/platform-cloudflare.test.ts index ca8c4a395..86dd5033d 100644 --- a/packages/void/test/unit/platform-cloudflare.test.ts +++ b/packages/void/test/unit/platform-cloudflare.test.ts @@ -1,16 +1,22 @@ import { describe, expect, it } from 'vite-plus/test'; +import { formatCloudflareUserMessage } from '../../src/cli/cloudflare-user-output.ts'; import { hyperdriveOriginConfigHash, hyperdriveOriginFromConnectionString, parsePostgresConnectionString, PlatformCloudflareApiError, PlatformCloudflareClient, + PlatformCloudflarePrerequisiteError, PlatformRuntimeTokenEmailPermissionError, PlatformWorkerCustomDomainsChangedError, verifyPlatformRuntimeTokenEmailPermissions, verifyPlatformRuntimeTokenPermissions, } from '../../src/cli/platform-cloudflare.ts'; +const expectedRejection = (): never => { + throw new Error('expected a rejection'); +}; + describe('platform Cloudflare client', () => { it('uses the runtime credential only for AI Gateway operations while keeping the API origin pinned', async () => { const requests: Array<{ path: string; authorization: string | null }> = []; @@ -639,9 +645,11 @@ describe('platform Cloudflare client', () => { expect(message).not.toContain(secret); expect(message).not.toContain('upstream echoed'); if (status === 401 || status === 403) { - expect(message).toContain('Cloudflare denied'); - expect(message).toContain('token is valid, scoped to this account'); + expect(message).toContain('Cloudflare denied the runtime token'); expect(message).toContain('Hyperdrive → Write'); + expect(message).toContain( + status === 401 ? 'did not accept the token' : 'the token works but is missing', + ); } else if (status === 429) { expect(message).toContain('rate-limited'); } else { @@ -795,13 +803,13 @@ describe('platform Cloudflare client', () => { }).catch((error: Error) => error.message); await expect(check(true)).resolves.toBe( - 'platform: The runtime API token saved on this machine cannot use platform email. Cloudflare denied: Zone → Email Routing Rules → Edit (HTTP 403). Void checked this token because VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN is not set. If the platform uses a different runtime token, set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to that token, and Void checks that token instead. Otherwise, check that the zone permissions of the saved token include the email zone mail-zone. To add a missing permission, edit the token in the Cloudflare dashboard. The token value stays the same. Then run the command again.', + 'platform: The runtime token saved on this machine cannot use platform email. Cloudflare denied: Zone → Email Routing Rules → Edit (HTTP 403). Void checked this token because VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN is not set. If the platform uses a different runtime token, set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to that token, and Void checks that token instead. Otherwise, check that the zone permissions of the saved token include the email zone mail-zone. To add a missing permission, edit the token in the Cloudflare dashboard. The token value stays the same. Then run the command again.', ); // A token from VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN, or one that // the run sends to the Workers, keeps the message. for (const savedToken of [undefined, false]) { await expect(check(savedToken)).resolves.toBe( - 'platform: The runtime API token cannot use platform email. Cloudflare denied: Zone → Email Routing Rules → Edit (HTTP 403). Check that the token is valid and that its zone permissions include the email zone mail-zone. To add a missing permission, edit the token in the Cloudflare dashboard. The token value stays the same. Or set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to a valid token that has these permissions. Then run the command again.', + 'platform: The runtime token cannot use platform email. Cloudflare denied: Zone → Email Routing Rules → Edit (HTTP 403). Check that the token is valid and that its zone permissions include the email zone mail-zone. To add a missing permission, edit the token in the Cloudflare dashboard. The token value stays the same. Or set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to a valid token that has these permissions. Then run the command again.', ); } }); @@ -1852,3 +1860,306 @@ describe('platform Cloudflare client', () => { ); }); }); + +describe('naming the Cloudflare credential in errors', () => { + const denyingFetcher = + (status: number, deny: (pathname: string) => boolean = () => true): typeof fetch => + async (input) => { + const { pathname } = new URL(String(input)); + return deny(pathname) + ? Response.json( + { success: false, errors: [{ message: 'Authentication error' }] }, + { status }, + ) + : Response.json({ success: true, result: [] }); + }; + const failure = (promise: Promise) => + promise.then( + () => { + throw new Error('expected a failure'); + }, + (error: unknown) => error as PlatformCloudflareApiError, + ); + + it.each([ + [ + 403, + 'Void used the management token from CF_API_TOKEN. HTTP 403 means the token works but is missing Account → Queues → Edit', + ], + [ + 401, + 'Void used the management token from CF_API_TOKEN. HTTP 401 means Cloudflare did not accept the token: it is not valid, has expired, or was replaced. Set CF_API_TOKEN to a valid management token, then retry.', + ], + ])( + 'adds the credential and what to do to an HTTP %s answer, and keeps the error class', + async (status, help) => { + const client = new PlatformCloudflareClient('operator-token', { + credential: { + role: 'management', + source: 'environment', + environmentVariable: 'CF_API_TOKEN', + }, + fetcher: denyingFetcher(status), + }); + const error = await failure(client.listQueues('account', 'usage')); + expect(error).toBeInstanceOf(PlatformCloudflareApiError); + expect(error.status).toBe(status); + expect(error.requestPath).toBe('/accounts/account/queues'); + expect(error.message).toContain(`(HTTP ${status}; GET /accounts/account/queues).`); + expect(error.message).toContain(help); + expect(error.message).not.toContain('operator-token'); + }, + ); + + it('leaves errors from a client without a label unchanged', async () => { + const client = new PlatformCloudflareClient('token', { fetcher: denyingFetcher(403) }); + const error = await failure(client.listQueues('account', 'usage')); + expect(error.message).toBe( + 'Cloudflare API error: Authentication error (HTTP 403; GET /accounts/account/queues).', + ); + }); + + it('does not relabel a known account prerequisite', async () => { + const client = new PlatformCloudflareClient('token', { + credential: { role: 'management', source: 'saved' }, + fetcher: async () => + Response.json( + { success: false, errors: [{ code: 10042, message: 'Please enable R2' }] }, + { status: 403 }, + ), + }); + const error = await failure(client.listR2Buckets('account')); + expect(error).toBeInstanceOf(PlatformCloudflarePrerequisiteError); + expect(error.message).not.toContain('Void used'); + }); + + it.each([ + ['environment', 'runtime token from VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN'], + ['saved', 'runtime token saved on this machine'], + ] as const)( + 'says browser login used the %s runtime token for AI Gateway', + async (source, name) => { + const client = new PlatformCloudflareClient('browser-token', { + credential: { role: 'management', source: 'oauth' }, + fetcher: denyingFetcher(401, (pathname) => pathname.includes('/ai-gateway/')), + }).withAiGatewayToken('runtime-token', { role: 'runtime', source }); + const error = await failure(client.listAiGateways('account')); + expect(error).toBeInstanceOf(PlatformCloudflareApiError); + expect(error.status).toBe(401); + expect(error.requestPath).toBe('/accounts/account/ai-gateway/gateways'); + expect(error.message).toContain( + `Browser login cannot use AI Gateway, so Void used the ${name} for it. Fix the runtime token (VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN), or set CLOUDFLARE_API_TOKEN to a management token with Account → AI Gateway → Edit. HTTP 401 means Cloudflare did not accept the token: it is not valid, has expired, or was replaced. Set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to a valid runtime token, then retry.`, + ); + expect(error.message).not.toContain('management token from'); + expect(error.message).not.toContain('runtime-token'); + // Other requests still use browser login. + await expect(client.listWorkerNames('account')).resolves.toEqual([]); + }, + ); + + it('says browser login cannot use AI Gateway when no runtime token is available', async () => { + const client = new PlatformCloudflareClient('browser-token', { + credential: { role: 'management', source: 'oauth' }, + fetcher: denyingFetcher(403), + }); + const error = await failure(client.createAiGateway('account', 'gateway')); + expect(error.status).toBe(403); + expect(error.message).toContain( + 'Browser login cannot use AI Gateway. Set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to the platform runtime token, or set CLOUDFLARE_API_TOKEN to a management token with Account → AI Gateway → Edit, then retry.', + ); + }); + + it('names browser login in words the CLI prints unchanged', async () => { + const client = new PlatformCloudflareClient('browser-token', { + credential: { role: 'management', source: 'oauth' }, + fetcher: denyingFetcher(403), + }); + const error = await failure(client.listQueues('account', 'usage')); + expect(error.message).toContain('Void used the Cloudflare browser login.'); + expect(formatCloudflareUserMessage(error.message)).toBe(error.message); + }); + + it('lists every runtime-token permission Cloudflare denied', async () => { + const paths: Array = []; + const client = new PlatformCloudflareClient('runtime-token', { + fetcher: async (input, init) => { + const { pathname } = new URL(String(input)); + paths.push(`${init?.method ?? 'GET'} ${pathname}`); + return denyingFetcher(403, (path) => !path.includes('custom_hostnames'))(input, init); + }, + }); + const error = await verifyPlatformRuntimeTokenPermissions(client, { + accountId: 'account', + applicationDomain: 'apps.example.com', + credential: { role: 'runtime', source: 'environment' }, + customDomains: true, + zoneId: 'zone', + }).then(expectedRejection, (error: Error) => error); + expect(paths).toEqual([ + 'GET /client/v4/accounts/account/hyperdrive/configs', + 'POST /client/v4/zones/zone/purge_cache', + 'GET /client/v4/zones/zone/custom_hostnames', + ]); + expect(error).not.toBeInstanceOf(PlatformRuntimeTokenEmailPermissionError); + expect(error.message).toBe( + [ + 'platform: Cloudflare denied the runtime token from VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN:', + '- Account → Hyperdrive → Write (HTTP 403)', + '- Zone → Cache Purge → Purge (HTTP 403)', + 'HTTP 403 means the token works but is missing the permissions listed above, or is not scoped to this account or zone. Add them to the token in the Cloudflare dashboard (the token value stays the same), or set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to a runtime token that has them, then retry.', + ].join('\n'), + ); + expect(error.cause).toBeInstanceOf(PlatformCloudflareApiError); + }); + + it('names a saved runtime token in the preflight error', async () => { + const client = new PlatformCloudflareClient('runtime-token', { fetcher: denyingFetcher(401) }); + const error = await verifyPlatformRuntimeTokenPermissions(client, { + accountId: 'account', + applicationDomain: '', + credential: { role: 'runtime', source: 'saved' }, + customDomains: false, + zoneId: null, + }).then(expectedRejection, (error: Error) => error); + expect(error.message).toContain('Cloudflare denied the runtime token saved on this machine:'); + expect(error.message).toContain( + 'Set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to a valid runtime token, then retry.', + ); + }); + + it('keeps the email error class only when email is the only problem', async () => { + const emailOnly = new PlatformCloudflareClient('runtime-token', { + fetcher: denyingFetcher(403, (path) => path.includes('/email/')), + }); + const input = { + accountId: 'account', + applicationDomain: '', + credential: { role: 'runtime', source: 'saved' }, + customDomains: false, + emailZoneId: 'mail-zone', + zoneId: null, + } as const; + await expect(verifyPlatformRuntimeTokenPermissions(emailOnly, input)).rejects.toBeInstanceOf( + PlatformRuntimeTokenEmailPermissionError, + ); + + const mixed = new PlatformCloudflareClient('runtime-token', { + fetcher: denyingFetcher( + 403, + (path) => path.includes('/email/') || path.includes('/hyperdrive/'), + ), + }); + const error = await verifyPlatformRuntimeTokenPermissions(mixed, input).then( + expectedRejection, + (error: Error) => error, + ); + // Install drops a saved token that lacks more than email access. + expect(error).not.toBeInstanceOf(PlatformRuntimeTokenEmailPermissionError); + expect(error.message).toContain('- Account → Hyperdrive → Write (HTTP 403)'); + expect(error.message).toContain('- Zone → Email Routing Rules → Edit (HTTP 403)'); + expect(error.message).toContain('- Account → Email Routing Addresses → Edit (HTTP 403)'); + }); + + it('keeps a denial seen before a later check fails for another reason', async () => { + const client = new PlatformCloudflareClient('runtime-token', { + fetcher: async (input) => + new URL(String(input)).pathname.endsWith('/purge_cache') + ? Response.json({ success: false, errors: [{ message: 'Slow down' }] }, { status: 429 }) + : denyingFetcher(403)(input), + }); + const error = await verifyPlatformRuntimeTokenPermissions(client, { + accountId: 'account', + applicationDomain: 'apps.example.com', + credential: { role: 'runtime', source: 'saved' }, + customDomains: true, + zoneId: 'zone', + }).then(expectedRejection, (error: Error) => error); + expect(error).not.toBeInstanceOf(PlatformRuntimeTokenEmailPermissionError); + expect(error.message).toContain('Cloudflare denied the runtime token saved on this machine:'); + expect(error.message).toContain('- Account → Hyperdrive → Write (HTTP 403)'); + expect(error.message).not.toContain('429'); + // Install finds the 403 in the cause chain and asks for a new token. + expect(error.cause).toBeInstanceOf(PlatformCloudflareApiError); + expect((error.cause as PlatformCloudflareApiError).status).toBe(403); + }); + + it('keeps the email error class when a later check fails for another reason', async () => { + const client = new PlatformCloudflareClient('runtime-token', { + fetcher: async (input) => { + const { pathname } = new URL(String(input)); + return pathname.endsWith('/purge_cache') + ? Response.json({ success: false, errors: [{ message: 'Down' }] }, { status: 503 }) + : denyingFetcher(403, (path) => path.includes('/email/'))(input); + }, + }); + await expect( + verifyPlatformRuntimeTokenPermissions(client, { + accountId: 'account', + applicationDomain: 'apps.example.com', + credential: { role: 'runtime', source: 'saved' }, + customDomains: false, + emailZoneId: 'mail-zone', + zoneId: 'zone', + }), + ).rejects.toBeInstanceOf(PlatformRuntimeTokenEmailPermissionError); + }); + + it('names the credential when the workers.dev subdomain read is denied', async () => { + const client = new PlatformCloudflareClient('operator-token', { + credential: { role: 'management', source: 'saved' }, + fetcher: denyingFetcher(403), + }); + const error = await failure(client.getWorkersSubdomain('account')); + expect(error.message).toBe( + 'platform: Could not read the workers.dev subdomain for account account (HTTP 403). Void used the management token saved on this machine. HTTP 403 means the token works but is missing Account → Workers Scripts → Edit, or is not scoped to this account or zone. Add it to the token in the Cloudflare dashboard (the token value stays the same), or set CLOUDFLARE_API_TOKEN to a management token that has it, then retry.', + ); + expect(error.cause).toBeInstanceOf(PlatformCloudflareApiError); + }); + + it.each([ + [ + 'reads a KV value', + (client: PlatformCloudflareClient) => client.getKvValue('account', 'namespace', 'manifest'), + '/accounts/account/storage/kv/namespaces/namespace/values/manifest', + 'Account → Workers KV Storage → Edit', + ], + [ + 'deletes a resource', + (client: PlatformCloudflareClient) => client.deleteWorkerRoute('zone', 'route'), + '/zones/zone/workers/routes/route', + 'Zone → Workers Routes → Edit', + ], + ])( + 'names the credential when Cloudflare denies a request that %s', + async (_name, call, path, permission) => { + const client = new PlatformCloudflareClient('operator-token', { + credential: { role: 'management', source: 'saved' }, + fetcher: denyingFetcher(403), + }); + const error = await failure(call(client)); + expect(error).toBeInstanceOf(PlatformCloudflareApiError); + expect(error.status).toBe(403); + expect(error.requestPath).toBe(path); + expect(error.credential).toEqual({ role: 'management', source: 'saved' }); + expect(error.message).toContain('Void used the management token saved on this machine.'); + expect(error.message).toContain(`missing ${permission}`); + expect(error.message).not.toContain('operator-token'); + }, + ); + + it('stops at a rate limit and says it is not a missing permission', async () => { + const client = new PlatformCloudflareClient('runtime-token', { + fetcher: denyingFetcher(429), + }); + const error = await verifyPlatformRuntimeTokenPermissions(client, { + accountId: 'account', + applicationDomain: '', + credential: { role: 'runtime', source: 'environment' }, + customDomains: false, + zoneId: null, + }).then(expectedRejection, (error: Error) => error); + expect(error.message).toBe( + 'platform: Cloudflare rate-limited the check of the runtime token from VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN for Account → Hyperdrive → Write (HTTP 429). Retry shortly; this does not mean the token is missing a permission.', + ); + }); +}); diff --git a/packages/void/test/unit/platform-installer.test.ts b/packages/void/test/unit/platform-installer.test.ts index 29a50dbb8..639e00fa7 100644 --- a/packages/void/test/unit/platform-installer.test.ts +++ b/packages/void/test/unit/platform-installer.test.ts @@ -1382,6 +1382,49 @@ describe('platform installer', () => { ).toThrow('canonical base64'); }); + it('requires both halves of a replaced R2 key pair', () => { + const { + jwtSecret: _jwtSecret, + projectSecretKey: _projectSecretKey, + ...fresh + } = FRESH_SECRETS_INPUT; + const input = { + ...fresh, + cloudflareApiToken: '', + existingInstallation: true, + updateExistingWorkers: true, + checkpoint: { r2AccessKeyId: 'saved-id', r2SecretAccessKey: 'saved-secret' }, + }; + expect(() => + createPlatformInstallationSecrets({ ...input, r2AccessKeyId: '', r2SecretAccessKey: 'new' }), + ).toThrow( + 'platform: VOID_PLATFORM_R2_SECRET_ACCESS_KEY is set but VOID_PLATFORM_R2_ACCESS_KEY_ID is not. Set VOID_PLATFORM_R2_ACCESS_KEY_ID and VOID_PLATFORM_R2_SECRET_ACCESS_KEY together, even if only the secret changed.', + ); + expect(() => + createPlatformInstallationSecrets({ ...input, r2AccessKeyId: 'new', r2SecretAccessKey: '' }), + ).toThrow( + 'VOID_PLATFORM_R2_ACCESS_KEY_ID is set but VOID_PLATFORM_R2_SECRET_ACCESS_KEY is not', + ); + + const replaced = createPlatformInstallationSecrets({ + ...input, + r2AccessKeyId: 'new-id', + r2SecretAccessKey: 'new-secret', + }); + expect(replaced.workerSecretUpdates).toMatchObject({ + R2_ACCESS_KEY_ID: 'new-id', + R2_SECRET_ACCESS_KEY: 'new-secret', + }); + const kept = createPlatformInstallationSecrets({ + ...input, + r2AccessKeyId: '', + r2SecretAccessKey: '', + }); + expect(kept.workerSecretUpdates).not.toHaveProperty('R2_ACCESS_KEY_ID'); + expect(kept.workerSecretUpdates).not.toHaveProperty('R2_SECRET_ACCESS_KEY'); + expect(kept).toMatchObject({ r2AccessKeyId: 'saved-id', r2SecretAccessKey: 'saved-secret' }); + }); + it('attempts Worker rollback even when closing an unsafe routing gate fails', async () => { const events: Array = []; const failures = await recoverFailedPlatformWorkerDeployment({ diff --git a/packages/void/test/unit/platform-management-upgrade.test.ts b/packages/void/test/unit/platform-management-upgrade.test.ts index 9f14cec02..71cd953b5 100644 --- a/packages/void/test/unit/platform-management-upgrade.test.ts +++ b/packages/void/test/unit/platform-management-upgrade.test.ts @@ -24,6 +24,12 @@ const state = vi.hoisted(() => ({ createSecrets: vi.fn(), getWorkerLastDeployedFrom: vi.fn(), clientTokens: [] as Array, + clientCredentials: [] as Array, + /** Answers browser-login AI Gateway requests through the real client. */ + aiGatewayStatus: 0, + operatorCredential: undefined as + | { source: 'environment' | 'saved' | 'oauth'; token: string } + | undefined, listAccounts: vi.fn(), listWorkerNames: vi.fn(), listNamespaceScripts: vi.fn(), @@ -74,18 +80,51 @@ vi.mock('@voidzero-dev/vite-plus-prompts', () => ({ select: vi.fn(), })); -vi.mock('../../src/cli/platform-auth.ts', async (importOriginal) => ({ - ...(await importOriginal()), - assertPlatformOperatorCredentialCanApplyPlan: vi.fn(), -})); +vi.mock('../../src/cli/platform-auth.ts', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + assertPlatformOperatorCredentialCanApplyPlan: vi.fn(), + resolvePlatformCloudflareCredential: vi.fn( + async (...args: Parameters) => + state.operatorCredential ?? actual.resolvePlatformCloudflareCredential(...args), + ), + }; +}); vi.mock('../../src/cli/platform-cloudflare.ts', async (importOriginal) => { const actual = await importOriginal(); return { ...actual, PlatformCloudflareClient: class { - constructor(private readonly token: string) { + constructor( + private readonly token: string, + private readonly options?: { credential?: unknown }, + ) { state.clientTokens.push(token); + state.clientCredentials.push(options?.credential); + } + + withAiGatewayToken( + token: string, + credential: ConstructorParameters[1] extends + | { credential?: infer Label } + | undefined + ? Label + : never, + ) { + // The real client names the token when Cloudflare refuses it. + const real = new actual.PlatformCloudflareClient(this.token, { + credential: this.options?.credential as never, + fetcher: async () => + Response.json( + { success: false, errors: [{ message: 'Authentication error' }] }, + { status: state.aiGatewayStatus || 401 }, + ), + }).withAiGatewayToken(token, credential); + return Object.assign(Object.create(this), { + listAiGateways: (accountId: string) => real.listAiGateways(accountId), + }); } listAccounts = state.listAccounts; listWorkerNames = state.listWorkerNames; @@ -258,7 +297,10 @@ import { platformLifecycleResumeCommand, runInstalledPlatformCommand, } from '../../src/cli/platform-management.ts'; -import { createPlatformInstallPlan } from '../../src/cli/platform-plan.ts'; +import { + createPlatformInstallPlan, + type PlatformInstallPlanAction, +} from '../../src/cli/platform-plan.ts'; import { acquirePlatformLifecycleLock, readInstallationRecoverySecrets, @@ -267,6 +309,10 @@ import { writePendingInstallationSecrets, } from '../../src/cli/platform-registry.ts'; +const expectedRejection = (): never => { + throw new Error('expected a rejection'); +}; + const upgrade = (overrides: { plan?: boolean; yes?: boolean; customDomains?: true } = {}) => runInstalledPlatformCommand({ subcommand: 'upgrade', @@ -333,6 +379,9 @@ describe('installed platform upgrade', () => { vi.stubEnv('CLOUDFLARE_API_TOKEN', 'token'); vi.stubEnv('CF_API_TOKEN', ''); state.clientTokens.length = 0; + state.clientCredentials.length = 0; + state.aiGatewayStatus = 0; + state.operatorCredential = undefined; state.requiredGatewayToken = ''; state.listAccounts.mockReset().mockResolvedValue([{ id: 'account', name: 'Account' }]); state.listWorkerNames.mockReset().mockResolvedValue([]); @@ -481,6 +530,98 @@ describe('installed platform upgrade', () => { }, ); + describe('naming the Cloudflare credential', () => { + const planReadsAiGateway = () => { + const original = vi.mocked(createPlatformInstallPlan).getMockImplementation()!; + vi.mocked(createPlatformInstallPlan).mockImplementationOnce(async (...args) => { + await args[0].listAiGateways(args[1].accountId); + return original(...args); + }); + }; + + it('labels the operator client with the variable that held its token', async () => { + vi.stubEnv('CLOUDFLARE_API_TOKEN', ''); + vi.stubEnv('CF_API_TOKEN', 'legacy-manager'); + await upgrade({ plan: true }); + expect(state.clientCredentials[0]).toEqual({ + role: 'management', + source: 'environment', + environmentVariable: 'CF_API_TOKEN', + }); + }); + + it('names both CF_API_TOKEN and every missing permission before taking locks', async () => { + vi.stubEnv('CLOUDFLARE_API_TOKEN', ''); + vi.stubEnv('CF_API_TOKEN', 'legacy-manager'); + state.manifest = recordInstallationResource( + state.manifest!, + { + configHash: 'hash', + id: 'assets', + key: 'assets-r2', + kind: 'r2-bucket', + name: 'assets', + ownership: 'created', + state: 'ready', + }, + '2026-09-14T00:00:03.000Z', + ); + state.listWorkerNames.mockRejectedValueOnce( + new PlatformCloudflareApiError('Authentication error', 403), + ); + state.request.mockRejectedValueOnce( + new PlatformCloudflareApiError('Authentication error', 403), + ); + const failure = await upgrade().then(expectedRejection, (error: Error) => error); + expect(failure?.message).toContain( + 'Cloudflare denied the management token from CF_API_TOKEN:\n- Account → Workers Scripts → Edit (HTTP 403; Cloudflare Workers access)\n- Account → Workers R2 Storage → Edit (HTTP 403; R2 bucket "assets")', + ); + expect(acquirePlatformLifecycleLock).not.toHaveBeenCalled(); + expect(installPlatform).not.toHaveBeenCalled(); + }); + + it.each([ + ['upgrade', 'saved', 'runtime token saved on this machine'], + ['repair', 'saved', 'runtime token saved on this machine'], + ['upgrade', 'environment', 'runtime token from VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN'], + ] as const)( + 'under browser login, %s names the %s runtime token that AI Gateway used', + async (subcommand, source, name) => { + vi.stubEnv('CLOUDFLARE_API_TOKEN', ''); + state.operatorCredential = { source: 'oauth', token: 'browser-login' }; + if (source === 'environment') { + vi.stubEnv('VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN', 'env-runtime'); + } + vi.mocked(readInstallationRecoverySecrets).mockReturnValue({ + databaseUrl: 'postgres://database', + cloudflareApiToken: 'saved-runtime', + }); + planReadsAiGateway(); + const failure = await runInstalledPlatformCommand({ + subcommand, + selector: state.manifest!.id, + plan: false, + runtime: '', + yes: true, + }).then(expectedRejection, (error: unknown) => error as PlatformCloudflareApiError); + expect(failure).toBeInstanceOf(PlatformCloudflareApiError); + expect(failure.status).toBe(401); + expect(failure.message).toContain( + `Browser login cannot use AI Gateway, so Void used the ${name} for it. Fix the runtime token (VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN), or set CLOUDFLARE_API_TOKEN to a management token with Account → AI Gateway → Edit.`, + ); + expect(failure.message).toContain( + 'Set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to a valid runtime token', + ); + // The 401 is the runtime token's, not browser login's. + expect(failure.message).not.toContain('Void used the Cloudflare browser login'); + expect(failure.message).not.toContain('Set CLOUDFLARE_API_TOKEN to a valid'); + expect(failure.message).not.toContain('runtime-'); + expect(installPlatform).not.toHaveBeenCalled(); + expect(writeInstallationManifest).not.toHaveBeenCalled(); + }, + ); + }); + it('includes shared email configuration when planning an upgrade', async () => { vi.stubEnv('VOID_EMAIL_SHARED_ZONE_ID', 'mail-zone'); vi.stubEnv('VOID_EMAIL_SENDER_DOMAIN', 'mail.example.com'); @@ -1213,6 +1354,11 @@ describe('installed platform upgrade', () => { if (status === 403) { expect(String(failure)).toContain('SSL and Certificates → Edit'); } + if (status === 401 || status === 403) { + expect(String(failure)).toContain( + 'Cloudflare denied the runtime token saved on this machine:', + ); + } expect(installPlatform).not.toHaveBeenCalled(); expect(state.events.filter((event) => event.startsWith('put:'))).toEqual([]); }, @@ -1233,6 +1379,10 @@ describe('installed platform upgrade', () => { }); await upgrade({ customDomains: true }); expect(state.clientTokens).toContain('replacement-token'); + expect(state.clientCredentials[state.clientTokens.indexOf('replacement-token')]).toEqual({ + role: 'runtime', + source: 'environment', + }); expect(state.clientTokens).not.toContain('saved-runtime-token'); expect(state.events).toContain(`request:${customHostnamePath}`); }); @@ -1338,7 +1488,7 @@ describe('installed platform upgrade', () => { // The Workers can use a newer token than the one saved on this machine. await expect(upgrade()).rejects.toThrow( - 'platform: The runtime API token saved on this machine cannot use platform email. Cloudflare denied: Zone → Zone Settings → Edit, Zone → Email Routing Rules → Edit, Account → Email Routing Addresses → Edit (HTTP 403). Void checked this token because VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN is not set. If the platform uses a different runtime token, set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to that token, and Void checks that token instead.', + 'platform: The runtime token saved on this machine cannot use platform email. Cloudflare denied: Zone → Zone Settings → Edit, Zone → Email Routing Rules → Edit, Account → Email Routing Addresses → Edit (HTTP 403). Void checked this token because VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN is not set. If the platform uses a different runtime token, set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to that token, and Void checks that token instead.', ); expect(installPlatform).not.toHaveBeenCalled(); @@ -1353,7 +1503,7 @@ describe('installed platform upgrade', () => { (failure: unknown) => failure as Error, ); expect(explicit?.message).toContain( - 'platform: The runtime API token cannot use platform email. Cloudflare denied: Zone → Zone Settings → Edit, Zone → Email Routing Rules → Edit, Account → Email Routing Addresses → Edit (HTTP 403). Check that the token is valid', + 'platform: The runtime token from VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN cannot use platform email. Cloudflare denied: Zone → Zone Settings → Edit, Zone → Email Routing Rules → Edit, Account → Email Routing Addresses → Edit (HTTP 403). Check that the token is valid', ); expect(explicit?.message).not.toContain('saved on this machine'); expect(installPlatform).not.toHaveBeenCalled(); @@ -1932,4 +2082,142 @@ describe('installed platform upgrade', () => { }); }); }); + describe('R2 upload credentials', () => { + const r2Pair = { R2_ACCESS_KEY_ID: 'new-id', R2_SECRET_ACCESS_KEY: 'new-secret' }; + const planWith = (...actions: Array>) => { + const original = vi.mocked(createPlatformInstallPlan).getMockImplementation()!; + vi.mocked(createPlatformInstallPlan).mockImplementationOnce(async (...args) => { + const plan = await original(...args); + return { + ...plan, + actions: [ + ...plan.actions, + ...actions.map((action) => ({ ...action, id: null, reason: 'test' })), + ], + }; + }); + }; + const assetsBucket = ( + action: 'create' | 'reuse' = 'reuse', + ): Omit => ({ + action, + key: 'assets-bucket', + kind: 'r2-bucket', + name: 'void-example-assets', + }); + const repair = (r2Fetch: typeof fetch) => + runInstalledPlatformCommand( + { + subcommand: 'repair', + selector: state.manifest!.id, + plan: false, + runtime: '', + yes: true, + }, + { fetch: r2Fetch }, + ); + + it('stops before any change when R2 rejects the pair from the environment', async () => { + state.createSecrets.mockReturnValue({ workerSecretUpdates: r2Pair }); + planWith(assetsBucket()); + const r2Fetch = vi.fn( + async () => + new Response( + 'SignatureDoesNotMatch', + { status: 403 }, + ), + ); + + await expect(repair(r2Fetch)).rejects.toThrow( + 'platform: R2 rejected the R2 key pair from the VOID_PLATFORM_R2_* environment variables (SignatureDoesNotMatch) on bucket void-example-assets. Void did not change the platform.', + ); + + expect(r2Fetch).toHaveBeenCalledOnce(); + expect(String((r2Fetch.mock.calls[0] as unknown as [string])[0])).toBe( + 'https://account.r2.cloudflarestorage.com/void-example-assets?list-type=2&max-keys=1', + ); + expect(installPlatform).not.toHaveBeenCalled(); + expect(state.events.filter((event) => event.startsWith('put:'))).toEqual([]); + expect(state.progress.start).not.toHaveBeenCalledWith('Repairing the platform'); + expect(vi.mocked(note).mock.calls).not.toContainEqual([expect.anything(), 'Recovery']); + expect(writeInstallationManifest).not.toHaveBeenCalled(); + expect(state.manifest!.status).toBe('installed'); + expect(state.events).toEqual(['outer-release', 'local-release']); + }); + + it('sends an accepted pair from the environment to the Workers', async () => { + state.createSecrets.mockReturnValue({ workerSecretUpdates: r2Pair }); + planWith(assetsBucket()); + const r2Fetch = vi.fn(async () => new Response('')); + + await repair(r2Fetch); + + expect(r2Fetch).toHaveBeenCalledOnce(); + expect(installPlatform).toHaveBeenCalledOnce(); + expect(vi.mocked(installPlatform).mock.calls[0]![1]).toMatchObject({ + workerSecretUpdates: r2Pair, + }); + }); + + it('makes no R2 request when the Workers keep their pair', async () => { + state.createSecrets.mockReturnValue({ + r2AccessKeyId: 'saved-id', + r2SecretAccessKey: 'saved-secret', + workerSecretUpdates: {}, + }); + planWith(assetsBucket()); + const r2Fetch = vi.fn(); + + await repair(r2Fetch); + + expect(r2Fetch).not.toHaveBeenCalled(); + expect(installPlatform).toHaveBeenCalledOnce(); + }); + + it('checks the pair saved on this machine when the run creates the api Worker', async () => { + state.createSecrets.mockReturnValue({ + r2AccessKeyId: 'saved-id', + r2SecretAccessKey: 'saved-secret', + workerSecretUpdates: {}, + }); + planWith(assetsBucket(), { + action: 'create', + key: 'api-worker', + kind: 'worker', + name: 'void-example-api', + }); + const r2Fetch = vi.fn(async () => new Response('denied', { status: 401 })); + + await expect(repair(r2Fetch)).rejects.toThrow( + 'R2 rejected the R2 key pair saved on this machine (HTTP 401) on bucket void-example-assets.', + ); + expect(installPlatform).not.toHaveBeenCalled(); + }); + + it('makes no R2 request when the run creates the assets bucket', async () => { + state.createSecrets.mockReturnValue({ workerSecretUpdates: r2Pair }); + planWith(assetsBucket('create')); + const r2Fetch = vi.fn(); + + await repair(r2Fetch); + + expect(r2Fetch).not.toHaveBeenCalled(); + expect(installPlatform).toHaveBeenCalledOnce(); + }); + + it('warns and continues when R2 cannot answer', async () => { + state.createSecrets.mockReturnValue({ workerSecretUpdates: r2Pair }); + planWith(assetsBucket()); + const r2Fetch = vi.fn(async () => { + throw new TypeError('fetch failed'); + }); + + await repair(r2Fetch); + + expect(log.warn).toHaveBeenCalledWith( + 'Void could not check the R2 key pair from the VOID_PLATFORM_R2_* environment variables on bucket void-example-assets (fetch failed). It continues without this check.', + ); + expect(installPlatform).toHaveBeenCalledOnce(); + }); + }); }); diff --git a/packages/void/test/unit/platform-plan.test.ts b/packages/void/test/unit/platform-plan.test.ts index a8f2d6b4b..55142a2f0 100644 --- a/packages/void/test/unit/platform-plan.test.ts +++ b/packages/void/test/unit/platform-plan.test.ts @@ -29,6 +29,10 @@ import { } from '../../src/cli/platform-plan.ts'; import { ownershipMarker } from '../helpers/platform-installation.ts'; +const expectedRejection = (): never => { + throw new Error('expected a rejection'); +}; + function client(overrides: Record = {}) { return { listAccounts: async () => [{ id: 'account', name: 'Account' }], @@ -736,6 +740,41 @@ describe('platform installation planning', () => { ); }); + it('names the runtime token once when install cannot verify AI Gateway', async () => { + const preview = await createPlatformInstallPlan( + client({ + listAiGateways: async () => { + throw new PlatformCloudflareApiError( + 'Authentication error', + 403, + '/accounts/account/ai-gateway/gateways', + ); + }, + }), + input, + null, + { allowUnverifiedAiGateway: true }, + ); + const gateway = new PlatformCloudflareClient('browser-login', { + credential: { role: 'management', source: 'oauth' }, + fetcher: async () => + Response.json( + { success: false, errors: [{ message: 'Authentication error' }] }, + { status: 403 }, + ), + }).withAiGatewayToken('runtime-token', { role: 'runtime', source: 'entered' }); + const error = await verifyPlatformAiGatewayPlan( + client({ listAiGateways: gateway.listAiGateways.bind(gateway) }), + input, + null, + preview, + false, + ).then(expectedRejection, (error: Error) => error); + expect(error.message).toBe( + 'platform: The runtime token could not verify AI Gateway access. Browser login cannot use AI Gateway, so Void used the runtime token you entered for it. Fix the runtime token (VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN), or set CLOUDFLARE_API_TOKEN to a management token with Account → AI Gateway → Edit. HTTP 403 means the token works but is missing Account → AI Gateway → Edit, or is not scoped to this account or zone. Add it to the token in the Cloudflare dashboard (the token value stays the same), or set VOID_PLATFORM_RUNTIME_CLOUDFLARE_API_TOKEN to a runtime token that has it, then retry. No platform resources were created.', + ); + }); + it('does not conceal other permission failures or AI Gateway service errors in previews', async () => { await expect( createPlatformInstallPlan( @@ -820,22 +859,83 @@ describe('platform installation planning', () => { { status: 403 }, ), }); - await expect( - createPlatformInstallPlan( - client({ - listR2Buckets: provider.listR2Buckets.bind(provider), - listDispatchNamespaces: async () => { - await Promise.resolve(); - throw unknown; - }, - }), - input, - null, - ), - ).rejects.toBe(unknown); + const error = await createPlatformInstallPlan( + client({ + listR2Buckets: provider.listR2Buckets.bind(provider), + listDispatchNamespaces: async () => { + await Promise.resolve(); + throw unknown; + }, + }), + input, + null, + ).catch((error: unknown) => error); + if (status === 500) { + expect(error).toBe(unknown); + return; + } + // A denied read is reported next to the prerequisite, and stays the + // first cause, so callers still find its status and path. + expect(error).toBeInstanceOf(AggregateError); + expect((error as AggregateError).cause).toBe(unknown); + expect((error as AggregateError).message).toContain('Unrelated provider failure'); + expect((error as AggregateError).message).toContain('Cloudflare R2 is not enabled'); }, ); + it('lists every denied read in one error that names the credential', async () => { + const provider = new PlatformCloudflareClient('operator-token', { + credential: { role: 'management', source: 'saved' }, + fetcher: async (request) => { + const { pathname } = new URL(String(request)); + return pathname.includes('/storage/kv/') || pathname.endsWith('/queues') + ? Response.json( + { success: false, errors: [{ message: 'Authentication error' }] }, + { status: 403 }, + ) + : Response.json({ success: true, result: [] }); + }, + }); + const error = await createPlatformInstallPlan( + client({ + listKvNamespaces: provider.listKvNamespaces.bind(provider), + listQueues: provider.listQueues.bind(provider), + }), + input, + null, + ).then(expectedRejection, (error: unknown) => error as AggregateError); + expect(error).toBeInstanceOf(AggregateError); + expect(error.errors).toHaveLength(5); + expect(error.cause).toBeInstanceOf(PlatformCloudflareApiError); + expect(error.message).toBe( + [ + 'platform: Cloudflare denied reads the plan needs.', + '', + 'Cloudflare denied the management token saved on this machine:', + '- Account → Workers KV Storage → Edit (HTTP 403; GET /accounts/account/storage/kv/namespaces)', + '- Account → Queues → Edit (HTTP 403; GET /accounts/account/queues)', + 'HTTP 403 means the token works but is missing the permissions listed above, or is not scoped to this account or zone. Add them to the token in the Cloudflare dashboard (the token value stays the same), or set CLOUDFLARE_API_TOKEN to a management token that has them, then retry.', + ].join('\n'), + ); + }); + + it('throws one denied read as it is', async () => { + const denied = new PlatformCloudflareApiError( + 'Authentication error', + 401, + '/accounts/account/queues', + ); + await expect( + createPlatformInstallPlan( + client({ + listQueues: vi.fn().mockRejectedValueOnce(denied).mockResolvedValue([]), + }), + input, + null, + ), + ).rejects.toBe(denied); + }); + it('names the DNS permission when the token cannot read DNS records', async () => { const deniedDns = async () => { throw new PlatformCloudflareApiError('Authentication error', 403, '/zones/zone/dns_records'); diff --git a/packages/void/test/unit/platform-r2-credentials.test.ts b/packages/void/test/unit/platform-r2-credentials.test.ts new file mode 100644 index 000000000..c077bf1fb --- /dev/null +++ b/packages/void/test/unit/platform-r2-credentials.test.ts @@ -0,0 +1,145 @@ +import { describe, expect, it, vi } from 'vite-plus/test'; +import { + createR2ListObjectsRequest, + verifyPlatformR2Credentials, +} from '../../src/cli/platform-r2-credentials.ts'; + +const CHECK = { + accountId: '0123456789abcdef0123456789abcdef', + bucket: 'void-example-assets', + accessKeyId: 'AKIDEXAMPLE0123456789', + secretAccessKey: 'wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY', + source: 'environment' as const, +}; +const NOW = new Date('2026-11-20T12:34:56.000Z'); +const EMPTY_SHA256 = 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'; +const r2ErrorXml = (code: string) => + `${code}details`; + +function verify( + response: Response | Error, + overrides: Partial[0]> = {}, +) { + const fetchMock = vi.fn(async () => { + if (response instanceof Error) { + throw response; + } + return response; + }); + const warn = vi.fn(); + const result = verifyPlatformR2Credentials({ ...CHECK, ...overrides }, fetchMock, { + now: () => NOW, + warn, + }); + return { fetchMock, result, warn }; +} + +describe('createR2ListObjectsRequest', () => { + it('signs one ListObjectsV2 request with SigV4 for R2', () => { + // Expected signature cross-checked once against aws4fetch 1.0.20 (the + // signer the platform API uses) with region auto and service s3. + expect(createR2ListObjectsRequest(CHECK, NOW)).toEqual({ + url: 'https://0123456789abcdef0123456789abcdef.r2.cloudflarestorage.com/void-example-assets?list-type=2&max-keys=1', + headers: { + authorization: + 'AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE0123456789/20261120/auto/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=b7132933ad117bda591b59cd04ba4863003a7b3c86e75eb7e1d4fb93940003a7', + 'x-amz-content-sha256': EMPTY_SHA256, + 'x-amz-date': '20261120T123456Z', + }, + }); + }); +}); + +describe('verifyPlatformR2Credentials', () => { + it('sends the signed GET once and passes on 200', async () => { + const { fetchMock, result, warn } = verify(new Response('')); + + await expect(result).resolves.toBeUndefined(); + expect(fetchMock).toHaveBeenCalledTimes(1); + const [url, init] = fetchMock.mock.calls[0] as unknown as [string, RequestInit]; + expect(url).toBe(createR2ListObjectsRequest(CHECK, NOW).url); + expect(init.method).toBe('GET'); + expect(init.headers).toEqual(createR2ListObjectsRequest(CHECK, NOW).headers); + expect(warn).not.toHaveBeenCalled(); + }); + + it('passes when R2 accepts the signature but the bucket is missing', async () => { + const { result, warn } = verify(new Response(r2ErrorXml('NoSuchBucket'), { status: 404 })); + + await expect(result).resolves.toBeUndefined(); + expect(warn).not.toHaveBeenCalled(); + }); + + it.each([ + [403, 'SignatureDoesNotMatch'], + [403, 'AccessDenied'], + [403, 'InvalidAccessKeyId'], + [401, 'Unauthorized'], + ])('stops on HTTP %s %s and names the environment pair', async (status, code) => { + const { result } = verify(new Response(r2ErrorXml(code), { status })); + + await expect(result).rejects.toThrow( + `platform: R2 rejected the R2 key pair from the VOID_PLATFORM_R2_* environment variables (${code}) on bucket void-example-assets. ` + + 'Void did not change the platform. ' + + 'Create an R2 API token with Object Read & Write on this bucket. ' + + 'Then set VOID_PLATFORM_R2_ACCESS_KEY_ID to its Access Key ID and VOID_PLATFORM_R2_SECRET_ACCESS_KEY to its Secret Access Key (not the token value), and run this command again. ' + + 'See https://void.cloud/guide/platform/installation/credentials#r2-upload-credentials', + ); + }); + + it.each(['InvalidArgument', 'AuthorizationHeaderMalformed'])( + 'stops on HTTP 400 %s and says the Access Key ID is the token ID', + async (code) => { + const { result, warn } = verify(new Response(r2ErrorXml(code), { status: 400 })); + + await expect(result).rejects.toThrow( + `platform: R2 rejected the R2 key pair from the VOID_PLATFORM_R2_* environment variables (${code}) on bucket void-example-assets. ` + + 'Void did not change the platform. ' + + 'The Access Key ID is the 32-character token ID, not the token value. ' + + 'Create an R2 API token with Object Read & Write on this bucket.', + ); + expect(warn).not.toHaveBeenCalled(); + }, + ); + + it('warns with a clock hint and continues on RequestTimeTooSkewed', async () => { + const { result, warn } = verify( + new Response(r2ErrorXml('RequestTimeTooSkewed'), { status: 403 }), + ); + + await expect(result).resolves.toBeUndefined(); + expect(warn).toHaveBeenCalledWith( + 'Void could not check the R2 key pair from the VOID_PLATFORM_R2_* environment variables on bucket void-example-assets (HTTP 403 RequestTimeTooSkewed: the clock on this machine is likely wrong; set it to the correct time). It continues without this check.', + ); + }); + + it('names the pair saved on this machine and falls back to the HTTP status', async () => { + const { result } = verify(new Response('denied', { status: 403 }), { source: 'saved' }); + + await expect(result).rejects.toThrow( + 'R2 rejected the R2 key pair saved on this machine (HTTP 403) on bucket void-example-assets.', + ); + }); + + it.each([ + ['a network error', new TypeError('fetch failed'), 'fetch failed'], + ['an R2 outage', new Response('busy', { status: 503 }), 'HTTP 503'], + [ + 'an unexpected answer', + new Response(r2ErrorXml('InvalidRequest'), { status: 400 }), + 'HTTP 400 InvalidRequest', + ], + [ + 'another 403 code', + new Response(r2ErrorXml('ExpiredRequest'), { status: 403 }), + 'HTTP 403 ExpiredRequest', + ], + ])('warns and continues on %s', async (_name, response, reason) => { + const { result, warn } = verify(response); + + await expect(result).resolves.toBeUndefined(); + expect(warn).toHaveBeenCalledWith( + `Void could not check the R2 key pair from the VOID_PLATFORM_R2_* environment variables on bucket void-example-assets (${reason}). It continues without this check.`, + ); + }); +}); diff --git a/packages/void/test/unit/platform-setup-flow.test.ts b/packages/void/test/unit/platform-setup-flow.test.ts index f200d6d89..52d9a7484 100644 --- a/packages/void/test/unit/platform-setup-flow.test.ts +++ b/packages/void/test/unit/platform-setup-flow.test.ts @@ -18,6 +18,7 @@ import { PlatformCloudflareClient, verifyPlatformRuntimeTokenEmailPermissions, } from '../../src/cli/platform-cloudflare.ts'; +import { formatPlatformCredentialDenials } from '../../src/cli/platform-credential-label.ts'; import { recordWorkerCutoverStage } from '../../src/cli/platform-installer-cutover.ts'; import { PlatformAccessBlockedError } from '../../src/cli/platform-installer.ts'; import { runPlatformLifecycleCommand } from '../../src/cli/platform-lifecycle.ts'; @@ -116,6 +117,9 @@ vi.mock('../../src/cli/platform-plan.ts', async (original) => ({ ...(await original()), createPlatformInstallPlan: mocks.plan, })); +const expectedFailure = (): never => { + throw new Error('expected a failure'); +}; let directory: string; const tty = Object.getOwnPropertyDescriptor(process.stdin, 'isTTY'); const args = [ @@ -734,6 +738,17 @@ describe('platform credential onboarding', () => { headers: expect.objectContaining({ Authorization: 'Bearer management-secret' }), }), ); + fetcher.mockResolvedValueOnce( + Response.json( + { success: false, errors: [{ message: 'Authentication error' }] }, + { status: 403 }, + ), + ); + const denied = await management + .request('/accounts/account/queues') + .then(expectedFailure, (error: PlatformCloudflareApiError) => error); + expect(denied.credential).toEqual({ role: 'management', source: 'entered' }); + expect(denied.message).toContain('Void used the management token you entered.'); expect(mocks.open).toHaveBeenCalledWith(expect.stringContaining('name=Test+Management')); const secrets = readPendingInstallationSecrets(listInstallationManifests()[0]!); expect(secrets?.cloudflareApiToken).toBe( @@ -786,6 +801,17 @@ describe('platform credential onboarding', () => { headers: expect.objectContaining({ Authorization: 'Bearer saved-management-secret' }), }), ); + fetcher.mockResolvedValueOnce( + Response.json( + { success: false, errors: [{ message: 'Authentication error' }] }, + { status: 403 }, + ), + ); + const denied = await client + .request('/accounts/account/queues') + .then(expectedFailure, (error: PlatformCloudflareApiError) => error); + expect(denied.credential).toEqual({ role: 'management', source: 'saved' }); + expect(denied.message).toContain('Void used the management token saved on this machine.'); expect(mocks.install.mock.calls[0]![1].cloudflareApiToken).toBe('saved-runtime-token'); expect(readPendingInstallationSecrets(manifest)?.managementApiToken).toBe( 'saved-management-secret', @@ -977,6 +1003,56 @@ describe('platform credential onboarding', () => { expect(listInstallationManifests()).toEqual([]); }); + it('prints a denial once when the error that wraps it repeats its message', async () => { + requireManagementToken(); + const warn = vi.spyOn(log, 'warn').mockImplementation(() => {}); + const original = mocks.plan.getMockImplementation()!; + mocks.plan.mockImplementationOnce(original).mockImplementationOnce(async (client) => { + const deny = (path: string) => + client + .request(path) + .catch((error: unknown) => error) as Promise; + const denied = [ + await deny('/accounts/account/queues'), + await deny('/accounts/account/storage/kv/namespaces'), + ]; + // The shape the plan throws for several reads denied to a labeled client. + throw new AggregateError( + denied, + `platform: Cloudflare denied reads the plan needs.\n\n${formatPlatformCredentialDenials( + denied[0]!.credential!, + denied.map((error) => ({ + permission: 'a permission this read needs', + request: `GET ${error.requestPath}`, + status: error.status, + })), + )}`, + { cause: denied[0] }, + ); + }); + vi.stubGlobal( + 'fetch', + vi.fn(async () => + Response.json( + { success: false, errors: [{ message: 'Authentication error' }] }, + { status: 403 }, + ), + ), + ); + mocks.password.mockResolvedValueOnce('rejected-secret').mockResolvedValueOnce(Symbol('cancel')); + + await expect(runPlatformLifecycleCommand(parsePlatformArgs(args))).rejects.toThrow('Cancelled'); + + expect(warn).toHaveBeenCalledOnce(); + const warning = warn.mock.calls[0]![0] as string; + // The list already names each request; the first denial is not repeated under it. + expect(warning).not.toContain('Cloudflare API error: Authentication error'); + expect(warning).toContain('GET /accounts/account/queues'); + expect(warning).toContain('GET /accounts/account/storage/kv/namespaces'); + expect(warning.split('management token you entered')).toHaveLength(2); + expect(warning).toContain('Check the token permissions and account/zone scope'); + }); + it('leaves no draft when management token input is cancelled', async () => { requireManagementToken(); mocks.password.mockResolvedValueOnce(Symbol('cancel')); diff --git a/packages/void/test/unit/platform-worker-reuse.test.ts b/packages/void/test/unit/platform-worker-reuse.test.ts index fe7c2baf0..10155821a 100644 --- a/packages/void/test/unit/platform-worker-reuse.test.ts +++ b/packages/void/test/unit/platform-worker-reuse.test.ts @@ -9,6 +9,24 @@ import { } from '@void/deploy-cloudflare'; import { expect, it, vi } from 'vite-plus/test'; import { PlatformWorkerRecoveryIncompleteError } from '../../src/cli/platform-installer-types.ts'; + +// Record the waits before deployment history is read again, and do not wait. +const wait = vi.hoisted(() => vi.fn(async (_milliseconds: number) => {})); +vi.mock('node:timers/promises', async (importOriginal) => ({ + ...(await importOriginal()), + setTimeout: wait, +})); + +// A new Worker is created through the conditional-create proxy. Send it to the +// test runner instead of starting the Cloudflare CLI. +const conditionalCreate = vi.hoisted(() => ({ + runner: null as null | ((operation: CloudflareOperation) => Promise), +})); +vi.mock('../../src/cli/platform-installer-process.ts', async (importOriginal) => ({ + ...(await importOriginal()), + runPlatformWranglerForConditionalCreate: async (operation: CloudflareOperation) => + conditionalCreate.runner!(operation), +})); import { deployPlatformWorker, platformWorkerUploadFingerprint, @@ -32,14 +50,18 @@ async function attemptReuse(input: { directProofDeploymentDrift?: boolean; directReadFailure?: boolean; liveTriggers?: boolean; - postActivationHeads?: ReadonlyArray<'previous' | 'activated' | 'competing' | Error>; + postActivationHeads?: ReadonlyArray< + 'previous' | 'activated' | 'competing' | 'empty' | 'older' | Error + >; reuseMatchingVersion?: boolean; secrets?: Record | null; triggerFailure?: boolean; lateVersionDrift?: boolean; versionDrift?: boolean; waitForPriorWorker?: () => Promise; + workerExisted?: boolean; }) { + const workerExisted = input.workerExisted !== false; const directory = mkdtempSync(join(tmpdir(), 'void-worker-reuse-')); const calls: Array = []; const onTrafficPublished = vi.fn(async () => {}); @@ -90,6 +112,13 @@ async function attemptReuse(input: { id: 'deployment-one', versions: [{ percentage: 100, version_id: VERSION }], }; + // An earlier restoration that runs the same version, listed under the + // previous deployment. A lagging read can return it at the head. + const olderDeployment = { + created_on: '2026-09-29T00:00:00.000Z', + id: 'deployment-older', + versions: [{ percentage: 100, version_id: VERSION }], + }; let directDeploymentReads = 0; const readCloudflare = vi.fn(async (path: string): Promise => { if (input.directReadFailure) { @@ -163,14 +192,22 @@ async function attemptReuse(input: { }; } if (args.type === 'deployments.list') { - const data = [deployment]; + let data = !workerExisted + ? [] + : input.postActivationHeads?.includes('older') + ? [deployment, olderDeployment] + : [deployment]; if (activated && input.postActivationHeads) { const head = input.postActivationHeads[postActivationReads++] ?? input.postActivationHeads.at(-1); if (head instanceof Error) { throw head; } - if (head && head !== 'previous') { + if (head === 'empty') { + data = []; + } else if (head === 'older') { + data = [olderDeployment]; + } else if (head && head !== 'previous') { visibleVersionId = head === 'activated' ? NEXT_VERSION : 'version-competing'; data.unshift({ created_on: '2026-09-30T00:01:00.000Z', @@ -213,8 +250,14 @@ async function attemptReuse(input: { activated = true; return { status: 0, stderr: '', stdout: '' }; } + if (args.type === 'worker.deploy' && input.postActivationHeads) { + activated = true; + return { status: 0, stderr: '', stdout: '', data: { versionId: NEXT_VERSION } }; + } throw new Error(`Unexpected command: ${args.type}`); }; + conditionalCreate.runner = runner; + wait.mockClear(); const operation = deployPlatformWorker(desiredConfig, input.secrets ?? null, { apiBaseUrl: 'https://api.cloudflare.com/client/v4', ...(input.acquireUploadSlot ? { acquireUploadSlot: input.acquireUploadSlot } : {}), @@ -236,7 +279,7 @@ async function attemptReuse(input: { publishRoutes: async () => {}, retainNewWorker, runner, - workerExisted: true, + workerExisted, }); const outcome = await operation.then( (value) => ({ error: null, value }), @@ -283,6 +326,84 @@ it('stops checking activation as soon as a competing deployment appears', async expect(run.onTrafficPublished).not.toHaveBeenCalled(); }); +it('waits through an empty deployment history after activation', async () => { + const run = await attemptReuse({ + changeConfig: true, + postActivationHeads: ['empty', 'activated'], + }); + + expect(run.error).toBeNull(); + expect(run.calls.filter((call) => call === 'versions.deploy')).toHaveLength(1); + expect(run.postActivationReads).toBe(2); + expect(run.onTrafficPublished).toHaveBeenCalledOnce(); +}); + +it('waits through an older known deployment at the head after activation', async () => { + const run = await attemptReuse({ + changeConfig: true, + postActivationHeads: ['older', 'activated'], + }); + + expect(run.error).toBeNull(); + expect(run.calls.filter((call) => call === 'versions.deploy')).toHaveLength(1); + expect(run.postActivationReads).toBe(2); + expect(run.onTrafficPublished).toHaveBeenCalledOnce(); +}); + +it('still refuses an unknown deployment after a stale empty history', async () => { + const run = await attemptReuse({ + changeConfig: true, + postActivationHeads: ['empty', 'competing', 'activated'], + }); + + expect(String(run.error)).toContain('changed concurrently after activation'); + expect(run.error).not.toBeInstanceOf(PlatformWorkerRecoveryIncompleteError); + expect(run.postActivationReads).toBe(2); + expect(run.onTrafficPublished).not.toHaveBeenCalled(); +}); + +it.each([ + { head: 'empty' as const, reason: 'lists no deployments for it yet' }, + { head: 'older' as const, reason: 'still reports older deployment deployment-older' }, +])('keeps the cutover unresolved when the history stays $head', async ({ head, reason }) => { + const run = await attemptReuse({ changeConfig: true, postActivationHeads: [head] }); + + expect(run.error).toBeInstanceOf(PlatformWorkerRecoveryIncompleteError); + expect(String(run.error)).toContain( + `could not confirm activation of void-test-api because Cloudflare ${reason}.`, + ); + expect(run.calls.filter((call) => call === 'versions.deploy')).toHaveLength(1); + expect(run.postActivationReads).toBe(4); + expect(wait.mock.calls.map(([milliseconds]) => milliseconds)).toEqual([250, 500, 1_000]); + expect(run.onTrafficPublished).not.toHaveBeenCalled(); +}); + +it('waits through an empty deployment history after creating a new Worker', async () => { + const run = await attemptReuse({ + changeConfig: true, + postActivationHeads: ['empty', 'activated'], + workerExisted: false, + }); + + expect(run.error).toBeNull(); + expect(run.calls.filter((call) => call === 'worker.deploy')).toHaveLength(1); + expect(run.calls).not.toContain('versions.deploy'); + expect(run.postActivationReads).toBe(2); + expect(run.onTrafficPublished).toHaveBeenCalledOnce(); +}); + +it('still refuses an unknown deployment on a new Worker', async () => { + const run = await attemptReuse({ + changeConfig: true, + postActivationHeads: ['competing'], + workerExisted: false, + }); + + expect(String(run.error)).toContain('changed concurrently after activation'); + expect(run.postActivationReads).toBe(1); + expect(run.onTrafficPublished).not.toHaveBeenCalled(); +}); + it('keeps the cutover unresolved when activation does not become visible', async () => { const run = await attemptReuse({ changeConfig: true, postActivationHeads: ['previous'] });