fix(llm): ensure json keyword in messages for Alibaba/Qwen compatibility

[kevin-ho]

Closes #663

When json_mode=True is set, Alibaba/Qwen providers strictly enforce the OpenAI spec requirement that the word "json" must appear in messages. Most providers are lenient about this check but Qwen rejects the call with a 400.

Adds _ensure_json_keyword() helper that appends a brief instruction to the system message when json_mode is active and no message contains the keyword. Gated on json_mode specifically to avoid noise for structured output calls to providers that don't need it.

Changes:

• New module-level constant _JSON_KEYWORD_HINT and helper _ensure_json_keyword() in src/llm/backends/openai.py
• Called in both complete() and stream() when extra_params["json_mode"] is set
• Early return if any message already contains "json" (case-insensitive) — no unnecessary injection
• If a system message exists, the hint is appended to it; otherwise a new system message is prepended

Testing:
Verified against opencode-go/qwen3.6-plus via OmniRoute gateway — previously 400'd immediately, now succeeds on first attempt.

Summary by CodeRabbit

• Bug Fixes
  • Improved compatibility for structured JSON output across OpenAI-compatible providers by ensuring chat input reliably triggers JSON-formatted responses when structured-output mode is enabled.
  • Enhances reliability of returned JSON without changing public APIs or altering existing response formatting behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 39880fbf-c9e5-48fd-8252-9d7bb336e82a

📥 Commits

Reviewing files that changed from the base of the PR and between 04d76ec and e43cd0b.

📒 Files selected for processing (1)

• src/llm/backends/openai.py

---

Walkthrough

Adds an internal helper that ensures the case-insensitive token "json" appears in chat messages when extra_params["json_mode"] is enabled, and invokes this helper in both OpenAIBackend.complete() and OpenAIBackend.stream() before sending requests.

Changes

JSON Keyword Enforcement for OpenAI-Compatible Providers

Layer / File(s)	Summary
JSON Keyword Helper Function src/llm/backends/openai.py	New _JSON_KEYWORD_HINT constant and _ensure_json_keyword(messages) helper that detects case-insensitive "json" in messages; if missing, appends the hint to the first system message (supports string and list content shapes) or prepends a new system message.
Integration in Complete and Stream src/llm/backends/openai.py	OpenAIBackend.complete() and OpenAIBackend.stream() call _ensure_json_keyword() after _build_params() when extra_params["json_mode"] is enabled, rewriting params["messages"] before the OpenAI API call/streaming request.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

A rabbit nudges messages, light and spry,
"Say 'json' please" beneath the system sky,
A tiny hint, no drama, no alarm,
Now strict and lenient providers share the charm,
Hoppity fixes, tidy and spry. 🐇✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding JSON keyword compatibility for Alibaba/Qwen providers, which is the primary objective of this PR.
Linked Issues check	✅ Passed	The code changes fully implement the proposed solution from issue #663: adding _ensure_json_keyword() helper that checks for 'json' in messages when json_mode=True and appends hint to system message or creates new one if absent.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to fixing the Qwen compatibility issue by adding the JSON keyword hint logic to OpenAI backend; no unrelated modifications are present.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

• Generate code and open pull requests
• Plan features and break down work
• Investigate incidents and troubleshoot customer tickets together
• Automate recurring tasks and respond to alerts with triggers
• Summarize progress and report instantly

Built for teams:

• Shared memory across your entire org—no repeating context
• Per-thread sandboxes to safely plan and execute work
• Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/llm/backends/openai.py`:
- Around line 41-46: The loop that mutates system messages assumes
updated["content"] is a string and calls rstrip(), causing AttributeError for
multimodal content lists; update the mutation in the for-loop (where result,
msg, updated and _JSON_KEYWORD_HINT are used) to safely coerce or normalize
content to a string before calling rstrip() (e.g., convert list/other types via
str(...) or serialize text blocks) and then append f"\n\n{_JSON_KEYWORD_HINT}"
so the mutation path mirrors the earlier detection path and never calls rstrip()
on a non-str.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 96498046-60ad-45a5-bb6c-d6b88f063ac3

📥 Commits

Reviewing files that changed from the base of the PR and between a4ae372 and 04d76ec.

📒 Files selected for processing (1)

• src/llm/backends/openai.py

[kevin-ho]

Thanks for the review @coderabbitai[bot] — I looked into the AttributeError concern on the .rstrip() call.

This is a false positive. The isinstance branches are mutually exclusive:

• Line 49 (if isinstance(content, str)): .rstrip() is only called here, inside the string branch.
• Line 52 (elif isinstance(content, list)): Appends a new {"type": "text", "text": ...} block — no .rstrip() anywhere.

The detection path (line 36) uses str() as a catch-all for the keyword check, but the mutation path already gates on type before touching string methods. A list-typed content will never reach .rstrip().

No changes needed.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Read the relevant file to verify the user's explanation
cat src/llm/backends/openai.py

Length of output: 19068

---

@kevin-ho You're absolutely right — after reviewing the code, your analysis is correct.

• Line 36 (detection): str(m.get("content", "")) is a safe cast used only for the keyword check — no string methods are called on the original value.
• Line 49 (if isinstance(content, str)): .rstrip() is strictly scoped inside this branch.
• Line 52 (elif isinstance(content, list)): only appends a new {"type": "text", "text": ...} dict — no .rstrip() anywhere.
• Line 55 (else: continue): gracefully skips unrecognised shapes and falls through to the prepend path.

The branches are mutually exclusive, so a list-typed content can never reach .rstrip(). The AttributeError concern was a false positive. The logic is sound as-is.