Skip to content

docs: explain OpenAI prompt caching settings (2/4) - #1447

Open
dhruva-vapi wants to merge 1 commit into
devin/1795222693-docs-soniox-endpointingfrom
devin/1795222693-docs-openai-prompt-caching
Open

dhruva-vapi wants to merge 1 commit into
devin/1795222693-docs-soniox-endpointingfrom
devin/1795222693-docs-openai-prompt-caching

Conversation

@dhruva-vapi

@dhruva-vapi dhruva-vapi commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Stack 2/4. Based on the Soniox PR (1/4). Split from #1445.

Description

Touches:

  • fern/providers/model/openai.mdx: new "Prompt caching" section before "Reasoning effort".
  • fern/assistants/model-intelligence/understanding-cost.mdx: one added line under "Effective input rate" that links to that section.

No existing text changes.

What it documents: two OpenAI model settings, already shipped and shown ungated in the dashboard model Advanced settings for GPT-4.1 and GPT-5.x models.

  • model.promptCacheKey: an optional hint that helps OpenAI reuse a matching prompt prefix (instructions, tool definitions). It doesn't store anything or guarantee a cache hit. Vapi leaves it unset unless you provide one.
  • model.promptCacheRetention: in_memory (OpenAI's default) or 24h, which keeps cached prefixes for up to 24 hours on models that support extended caching.
  • Both are unset by default. Leaving the key unset doesn't turn caching off.
  • The page links to the OpenAIModel API reference for the current model list instead of copying it. The link target is verified in the preview.
  • Source: libs/core/src/types/model.types.ts, util/openAIUtil/openAIUtil.ts and apps/dashboard/.../openAiAdvanced/ in VapiAI/vapi.

Testing Steps


Technical Writer opened this PR for Dhruva
Implemented in session Update docs for recent product changes
Review with Devin · Open in Devin Desktop


Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access that start with 'Devin'.

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Devin Review

🟢 Devin Review found nothing to fix · View findings

@github-actions

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant