Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions fern/assistants/model-intelligence/understanding-cost.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ Vapi estimates 5 requests per turn. We use number of requests rather than conver

**Effective input rate** accounts for prompt caching. Cached input is stable prompt content that providers reuse across requests. Many providers offer a discounted rate for this input.

For OpenAI configuration options, see [Prompt caching](/providers/model/openai#prompt-caching).

When a model supports caching, Vapi blends its standard and cached rates based on an assumed 50% cache hit rate. Otherwise, Vapi uses the standard rate. The calculation uses the actual rates for each model and provider.

#### Output tokens per minute
Expand Down
24 changes: 24 additions & 0 deletions fern/providers/model/openai.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,30 @@ If you've added an [Azure OpenAI credential](/providers/model/azure-openai) for
| GPT 5 Mini | `gpt-5-mini:eastus2`<br />`gpt-5-mini:germanywestcentral`<br />`gpt-5-mini:polandcentral`<br />`gpt-5-mini:spaincentral`<br />`gpt-5-mini:swedencentral`<br />`gpt-5-mini:westeurope` |
| GPT 5 Nano | `gpt-5-nano:eastus2`<br />`gpt-5-nano:swedencentral` |

## Prompt caching

Prompt caching lets OpenAI reuse the unchanged beginning of a prompt, such as your assistant's instructions and tool definitions, instead of processing it again on every request. Cache hits can reduce input-token costs and latency, but are not guaranteed.

For supported GPT-4.1 and GPT-5.x models, open the assistant's model **Advanced settings** in the Dashboard to configure **Prompt cache retention** and **Prompt cache key**. Check the Dashboard or the [`OpenAIModel` fields](/api-reference/assistants/create#request.body.model.OpenAI) for the current supported model list.

- **`promptCacheKey`**: An optional hint to help OpenAI reuse matching prompt prefixes across requests. It does not store a prompt or guarantee a cache hit. In the standard assistant model configuration, Vapi leaves this unset unless you provide it; it does not generate a key automatically.
- **`promptCacheRetention`**: Choose `in_memory` for OpenAI's in-memory cache policy or `24h` to keep cached prefixes available for up to 24 hours on models that support extended caching.

Both settings are unset by default. For these models, omitting retention uses OpenAI's default `in_memory` policy; leaving the key unset does not disable automatic prompt caching.

For example, include these fields in your assistant's model configuration:

```json
{
"model": {
"provider": "openai",
"model": "gpt-4.1",
"promptCacheKey": "support-assistant-v1",
"promptCacheRetention": "24h"
}
}
```

## Reasoning effort

Reasoning models think before they answer. Set `model.reasoningEffort` to control how much. A lower effort reduces latency and cost, which suits most voice calls. A higher effort can help with harder tasks.
Expand Down
Loading