Skip to content

Stop rejecting code and choruses as repeated phrases - #304

Merged
iamwavecut merged 1 commit into
mainfrom
fix/dialog-repetition-false-positives
Nov 20, 2026
Merged

iamwavecut merged 1 commit into
mainfrom
fix/dialog-repetition-false-positives

Conversation

@iamwavecut

Copy link
Copy Markdown
Owner

Fixes #298.

The repetition check in pathological_final_answer_reason rejected any reply in which one normalized segment of two or more words occurred three times. It dropped operators and checked code, so a calculator (return a + b, return a - b, return a * b → return a b) and a song with a chorus failed on every model. The job retry budget then replayed the same rejection five times, about 20 LLM calls per turn, and the user got no answer.

Changes

  • Phrase check (openplotva-dialog): <pre>, <code> and fenced code are masked before the check. Operators stay as tokens, so different code lines outside code blocks stay different. A repeated phrase is rejected only when back-to-back copies cover at least half of the prose, or when all phrases seen three or more times cover at least 80% of it. The character-run, missing-space, paragraph-count and repeated-block checks are unchanged.
  • No requeue when every model loops (dialog_runtime, dialog_jobs/retry): when at least two models reject the same step as pathological and no attempt failed for another reason, the error carries OutputRejectedByEveryTarget. The retry handler then fails the turn with output_rejected_by_every_target instead of requeueing it. The job-level all_attempts_exhausted event has rejected_by_every_target.
  • Rejection kind in routing events (routed_attempts): run_described records what a validator rejected, for example pathological: repeated phrase, as rejection in the attempt_failed and model_output_retry detail. run keeps its signature and records nothing extra.

Tests

  • A calculator in <pre><code>, in a fence and without a fence passes.
  • A song with a chorus line repeated five times across verses passes.
  • A long reply that ends in a short repeated tail passes.
  • A sentence loop, a two-sentence cycle and a loop after a code block are still rejected. The earlier rejection tests pass unchanged.
  • Two models that both reject a step as pathological mark the requeue futile, and their events carry rejection. A failure of another kind keeps the requeue.
  • The retry handler fails a futile step on its first attempt without a requeue.

The phrase check dropped operators and scanned code, so a calculator
and a song with a chorus failed on every model, and the job retry
budget replayed the rejection five times. Code spans are now masked,
operators stay tokens, and a repeated phrase is rejected only when it
fills most of the prose. A step that every model answers with a loop
fails without a requeue, and routing events record the rejection kind.
@github-actions

Copy link
Copy Markdown
Messages
📖 Danger checked 6 changed file(s).

Generated by 🚫 dangerJS against 365c4ae

@iamwavecut
iamwavecut merged commit 18336ee into main Nov 20, 2026
12 checks passed
@iamwavecut
iamwavecut deleted the fix/dialog-repetition-false-positives branch November 20, 2026 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant