From 3653635876d748a3d397162c851205fdd5f3d0fa Mon Sep 17 00:00:00 2001 From: dhruva Date: Sat, 21 Nov 2026 00:58:13 +0000 Subject: [PATCH] docs: explain OpenAI prompt caching settings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --- .../model-intelligence/understanding-cost.mdx | 2 ++ fern/providers/model/openai.mdx | 24 +++++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/fern/assistants/model-intelligence/understanding-cost.mdx b/fern/assistants/model-intelligence/understanding-cost.mdx index 3bf4956ed..a030525bf 100644 --- a/fern/assistants/model-intelligence/understanding-cost.mdx +++ b/fern/assistants/model-intelligence/understanding-cost.mdx @@ -61,6 +61,8 @@ Vapi estimates 5 requests per turn. We use number of requests rather than conver **Effective input rate** accounts for prompt caching. Cached input is stable prompt content that providers reuse across requests. Many providers offer a discounted rate for this input. +For OpenAI configuration options, see [Prompt caching](/providers/model/openai#prompt-caching). + When a model supports caching, Vapi blends its standard and cached rates based on an assumed 50% cache hit rate. Otherwise, Vapi uses the standard rate. The calculation uses the actual rates for each model and provider. #### Output tokens per minute diff --git a/fern/providers/model/openai.mdx b/fern/providers/model/openai.mdx index 3b4a8371d..8b0cf1a46 100644 --- a/fern/providers/model/openai.mdx +++ b/fern/providers/model/openai.mdx @@ -82,6 +82,30 @@ If you've added an [Azure OpenAI credential](/providers/model/azure-openai) for | GPT 5 Mini | `gpt-5-mini:eastus2`
`gpt-5-mini:germanywestcentral`
`gpt-5-mini:polandcentral`
`gpt-5-mini:spaincentral`
`gpt-5-mini:swedencentral`
`gpt-5-mini:westeurope` | | GPT 5 Nano | `gpt-5-nano:eastus2`
`gpt-5-nano:swedencentral` | +## Prompt caching + +Prompt caching lets OpenAI reuse the unchanged beginning of a prompt, such as your assistant's instructions and tool definitions, instead of processing it again on every request. Cache hits can reduce input-token costs and latency, but are not guaranteed. + +For supported GPT-4.1 and GPT-5.x models, open the assistant's model **Advanced settings** in the Dashboard to configure **Prompt cache retention** and **Prompt cache key**. Check the Dashboard or the [`OpenAIModel` fields](/api-reference/assistants/create#request.body.model.OpenAI) for the current supported model list. + +- **`promptCacheKey`**: An optional hint to help OpenAI reuse matching prompt prefixes across requests. It does not store a prompt or guarantee a cache hit. In the standard assistant model configuration, Vapi leaves this unset unless you provide it; it does not generate a key automatically. +- **`promptCacheRetention`**: Choose `in_memory` for OpenAI's in-memory cache policy or `24h` to keep cached prefixes available for up to 24 hours on models that support extended caching. + +Both settings are unset by default. For these models, omitting retention uses OpenAI's default `in_memory` policy; leaving the key unset does not disable automatic prompt caching. + +For example, include these fields in your assistant's model configuration: + +```json +{ + "model": { + "provider": "openai", + "model": "gpt-4.1", + "promptCacheKey": "support-assistant-v1", + "promptCacheRetention": "24h" + } +} +``` + ## Reasoning effort Reasoning models think before they answer. Set `model.reasoningEffort` to control how much. A lower effort reduces latency and cost, which suits most voice calls. A higher effort can help with harder tasks.