Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@ Safe to run without asking: `npm run validate`, `npm run audit`,
plain `npm run pull` (it never overwrites local edits), `npm run push -- <org>
--dry-run`, `npm run build`, `npm test`.

- `variable-undefined`: set squad members' `{{vars}}` in squad/member `variableValues`; `--strict` blocks missing values.
- `transfer-number-liquid`: transfer destination numbers must be literal; put per-squad numbers in an inline `transferCall` in `membersOverrides["tools:append"]`.

**2. Never handle API keys.** Don't ask the human to paste a key into chat,
don't print or read out `.env.*` files, never pass a key as a command-line
argument, and never commit `.env.*`. Keys live in `.env.<org>` (gitignored)
Expand Down Expand Up @@ -150,6 +153,8 @@ precisely.
never ask for a real key just to validate. Don't weaken the check or the
workflow to get past it, and never edit the state file, `.vapi-ignore` or
`state/<org>/org.json` to make a reference resolve: fix the name, or pull.
- `variable-undefined`: set squad members' `{{vars}}` in squad/member `variableValues`; `--strict` blocks this warning in validate and push.
- `transfer-number-liquid`: use literal transfer numbers; append a per-squad inline `transferCall` in the squad's `membersOverrides["tools:append"]`.
4. **Build PR checks offline** if `vapi-checks.yml` exists:
`npm run check -- --all --dry-run`. Fix anything it reports.
5. **Deploy only with a yes** (safety rule 1): `npm run apply -- <org>`, or
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ More detail: [How the engine works](docs/guides/how-it-works.md).
| Command | What it does |
| --- | --- |
| `npm run setup` | Connect an org: create `.env.<org>` and `resources/<org>/` |
| `npm run validate -- <org>` | Check resource files offline before deploying |
| `npm run validate -- <org> [--strict]` | Check resource files offline, including undefined squad variables (`variable-undefined`); `--strict` blocks on that warning. `transfer-number-liquid` always errors on Liquid in transfer destination numbers; use per-practice literal numbers (see `examples/nhs-shared-squads`). |
| `npm run apply -- <org>` | Deploy: pull, merge, then push (the default) |
| `npm run pull -- <org>` | Sync platform changes down without overwriting local edits |
| `npm run push -- <org>` | Push without pulling first (prefer `apply`) |
Expand Down
11 changes: 11 additions & 0 deletions docs/guides/blocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,17 @@ A block is a piece of prompt text or config that lives in one file and is
used by many resources. Change the block and every assistant and squad that
includes it changes with it, in every org.

`validate` checks Liquid variables after expanding blocks. For each squad member,
`variable-undefined` warns about root names missing from the member's
`assistantOverrides.variableValues` and the squad's `membersOverrides.variableValues`.
It checks system prompts, first/voicemail/end-call messages, and attached tools,
including `tools:append`. Vapi built-ins, squad tool/handoff extraction schema
properties and alias keys, and Liquid assign/capture/loop locals count as defined.
Raw blocks and tool code are ignored. Standalone assistants are not checked:
their variables can be supplied at call time. This is a static check, not proof
that an extraction tool runs before a variable is used. Use `--strict` with
validate or push to make missing squad variables blocking.

There are two kinds:

| Kind | File | Included with | Use it for |
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The other commands are direct only.
| Command | Usage | What it does |
| --- | --- | --- |
| `npm run setup` | `npm run setup [-- <org>]` | Connect an org: creates `.env.<org>` and `resources/<org>/`. |
| `npm run validate` | `npm run validate -- <org>` | Check resource files offline: API shape rules, and that every reference names a file or a state entry. `apply` runs it first. On GitHub Actions, findings are also shown on the files in the pull request. |
| `npm run validate` | `npm run validate -- <org> [--strict]` | Check resource files offline: API shape rules, references, and undefined squad variables. `variable-undefined` warns once per squad member, listing missing root names and where to define them; `--strict` makes this warning blocking in validate and push. `transfer-number-liquid` is always an error: the API rejects Liquid in transfer destination numbers, even with `numberE164CheckEnabled:false`; use per-practice literal numbers, e.g. inline in squad `membersOverrides['tools:append']` (see `examples/nhs-shared-squads`). Covers standalone tools, inline model tools and member/squad appended tools. `apply` runs validation first. On GitHub Actions, findings are also shown on the files in the pull request. |
| `npm run apply` | `npm run apply -- <org> [types or paths]` | **The default deploy:** pull, merge, then push. See [workflows](workflows.md). |
| `npm run pull` | `npm run pull -- <org> [--force] [--bootstrap]` | Sync platform changes down; never overwrites local edits unless `--force`. |
| `npm run push` | `npm run push -- <org> [--dry-run] [--strict]` | Push without pulling first. Prefer `apply`. `--strict` aborts before any API call if validation finds an error. |
Expand Down
12 changes: 10 additions & 2 deletions docs/learnings/squads.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,16 @@ For a one-off call using an existing `squadId`, the same shape can be passed as

## Override Merge Order

### Shared assistants speak unresolved practice variables

**Problem:** A shared assistant says literal `{{name}}` instead of a practice's details.

**Cause:** Undefined variables remain literal on the call; an explicit empty string renders as nothing. Override precedence is member → squad → call; later values win.

**Fix:** Keep assistants shared and set each practice's values in `membersOverrides.variableValues` (or a member's `assistantOverrides.variableValues`). `variable-undefined` warns per squad member; `--strict` blocks it in validate and push. Transfer numbers must stay literal: see [Liquid destination numbers](tools.md#liquid-destination-numbers-fail-api-validation).

### Layer precedence

The final assistant configuration is built by merging these layers in order:

1. Base assistant
Expand All @@ -169,8 +179,6 @@ Later layers win on conflicts. `variableValues` from all layers are merged separ

**Gotcha:** a `credentialIds` list anywhere in the squad narrows the whole call to the listed keys, for every member and every provider. A provider left off runs on Vapi's own key with no error. List every key the call needs, and let `npm run validate` check it: see [Credential pins](../guides/credential-pins.md).

**Gotcha:** a variable with no value is **left in the prompt as written**, not replaced and not an error. If a variable isn't in the merged `variableValues`, `{{ myVar }}` reaches the model as the literal text `{{ myVar }}`. A variable that is set to an empty string does render as nothing. So a squad that forgets a variable shows the braces to the model: check every `{{ … }}` in a prompt, and in any [block](../guides/blocks.md) it includes, has a value in each squad that uses it.

---

## Squad-level post-call extraction via `membersOverrides` (multi-member squads)
Expand Down
8 changes: 8 additions & 0 deletions docs/learnings/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,14 @@ If the schema doesn't meet these requirements, `strict` may be silently disabled

## transferCall Tools

### Liquid destination numbers fail API validation

**Problem:** A transfer destination such as `number: '{{handover_number}}'` is rejected even when the variable is defined.

**Cause:** The API requires E.164 by default; `numberE164CheckEnabled: false` still enforces `^\+?[a-zA-Z0-9]+$`, which rejects Liquid syntax.

**Fix:** Use literal numbers. For shared assistants, put per-squad numbers in an inline `transferCall` in the squad's `membersOverrides["tools:append"]`. `transfer-number-liquid` reports an error before deployment. See [shared squad variables](squads.md#shared-assistants-speak-unresolved-practice-variables) and the [NHS example (PR #100)](https://github.com/VapiAI/gitops/pull/100).

### Forwarding numbers from the assistant are auto-injected

**What you might expect:** Only the `destinations` array on the tool matters.
Expand Down
36 changes: 36 additions & 0 deletions improvements.md
Original file line number Diff line number Diff line change
Expand Up @@ -2334,6 +2334,42 @@ have a bootstrap pull leave existing baselines alone.

---

## 43. Missing practice variables reached live prompts literally

Shared assistants can run in many practice squads, each supplying practice
details through variableValues. A missing value stayed as `{{name}}` on a call.
`variable-undefined` now checks each squad member's expanded prompts and attached
tools against squad/member values, Vapi built-ins and runtime assignments. It
warns by default and blocks with `--strict`. Standalone assistants remain
unchecked because their values may arrive when a call is created.

### Status

**Implemented.** Covered by `tests/validate-variables.test.ts`.

---

## 44. Liquid transfer destination numbers passed offline validation

### Problem

The API rejects Liquid in transfer number destinations: the default E.164
validator rejects it, and disabling that check still requires alphanumeric
characters with an optional leading plus. Runtime template support does not
make these tool payloads API-valid. A push could therefore fail partway through.

### Current behavior and mitigation

`transfer-number-liquid` reports an error before deployment for standalone and
inline transfer tools, including member/squad appended tools. Use literal
per-practice numbers in inline squad overrides; see `examples/nhs-shared-squads`.

### Status

**Implemented.** Covered by `tests/validate-transfer-numbers.test.ts`.

---

## Out of scope (intentionally not improvements)

- **State file is identity-only and not git-ignored.** It's intentionally
Expand Down
6 changes: 5 additions & 1 deletion src/push.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1837,7 +1837,11 @@ async function main(): Promise<void> {
} else {
console.log(" ✅ No validation issues.");
}
const errorCount = findings.filter((f) => f.severity === "error").length;
const errorCount = findings.filter(
(f) =>
f.severity === "error" ||
(STRICT_VALIDATION && f.rule === "variable-undefined"),
).length;
if (errorCount > 0) {
if (STRICT_VALIDATION) {
console.error(
Expand Down
7 changes: 6 additions & 1 deletion src/validate-cmd.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import {
BASE_DIR,
loadIgnorePatterns,
RESOURCES_DIR,
STRICT_VALIDATION,
VAPI_ENV,
} from "./config.ts";
import {
Expand Down Expand Up @@ -96,7 +97,11 @@ async function main(): Promise<void> {
}
}

const errorCount = findings.filter((f) => f.severity === "error").length;
const errorCount = findings.filter(
(f) =>
f.severity === "error" ||
(STRICT_VALIDATION && f.rule === "variable-undefined"),
).length;
if (errorCount > 0) {
console.error(
`❌ Validation failed with ${errorCount} error(s). Fix the issues above before pushing.`,
Expand Down
44 changes: 44 additions & 0 deletions src/validate-transfer-numbers.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import type { LoadedResources } from "./types.ts";
import type { ValidationFinding } from "./validate.ts";

export function validateTransferNumbers(
resources: LoadedResources,
): ValidationFinding[] {
const findings: ValidationFinding[] = [];
for (const type of ["tools", "assistants", "squads"] as const) {
for (const resource of resources[type]) {
const visit = (value: unknown, path: string): void => {
if (!value || typeof value !== "object") return;
if (Array.isArray(value)) {
value.forEach((entry, index) => visit(entry, `${path}[${index}]`));
return;
}
const object = value as Record<string, unknown>;
if (
object.type === "transferCall" &&
Array.isArray(object.destinations)
) {
object.destinations.forEach((destination, index) => {
const number = destination?.number;
if (typeof number !== "string" || !/{{|{%/.test(number)) return;
// Vapi: libs/core/src/types/tool.types/transferCallTool.types.ts (TransferDestinationNumber).
findings.push({
severity: "error",
type,
resourceId: resource.resourceId,
rule: "transfer-number-liquid",
fieldPath: `${path ? `${path}.` : ""}destinations[${index}].number`,
message:
"The API rejects Liquid in transfer numbers: E.164 validation applies, and numberE164CheckEnabled:false still requires ^\\+?[a-zA-Z0-9]+$. Use a per-practice literal number, e.g. an inline tool in the squad's membersOverrides['tools:append']; see examples/nhs-shared-squads.",
});
});
}
for (const [key, child] of Object.entries(object)) {
visit(child, path ? `${path}.${key}` : key);
}
};
visit(resource.data, "");
}
}
return findings;
}
Loading
Loading