From a2d1a285ae8a9e15fe2c1929f6d49a94740c19d7 Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Thu, 23 Apr 2026 08:49:36 +0200 Subject: [PATCH 01/29] feat(openai-codex): add provider module skeleton Adds the headless `_core/openai_codex/` module with Codex endpoint detection, Cloudflare-compatible header helpers, and the module contract documenting the ChatGPT Plus OAuth transport. - request.js defines CODEX_BASE_URL plus prefix-match detection so `/responses`, `/models`, and future sub-endpoints share one matcher - applyCodexHeaders() sets the required User-Agent plus originator headers; the Cloudflare layer in front of the Codex endpoint rejects requests without these regardless of token validity - extractChatGPTAccountId() tolerantly parses the OAuth JWT claim `https://api.openai.com/auth.chatgpt_account_id` using atob() for the browser request-mutation path; malformed tokens silently return "" so the header is just omitted - AGENTS.md documents the Cloudflare header requirement, the Chat- Completions-to-Responses request-shape conversion rules, the SSE event mapping tables, the persisted token shape, and the OAuth URL constants - root AGENTS.md index gains the new module path --- AGENTS.md | 1 + app/L0/_all/mod/_core/openai_codex/AGENTS.md | 122 ++++++++++++++++++ app/L0/_all/mod/_core/openai_codex/request.js | 78 +++++++++++ 3 files changed, 201 insertions(+) create mode 100644 app/L0/_all/mod/_core/openai_codex/AGENTS.md create mode 100644 app/L0/_all/mod/_core/openai_codex/request.js diff --git a/AGENTS.md b/AGENTS.md index 9f334057..fbe84ba1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -95,6 +95,7 @@ App docs: - `/app/L0/_all/mod/_core/onscreen_agent/prompts/AGENTS.md` - `/app/L0/_all/mod/_core/onscreen_menu/AGENTS.md` - `/app/L0/_all/mod/_core/open_router/AGENTS.md` +- `/app/L0/_all/mod/_core/openai_codex/AGENTS.md` - `/app/L0/_all/mod/_core/panels/AGENTS.md` - `/app/L0/_all/mod/_core/promptinclude/AGENTS.md` - `/app/L0/_all/mod/_core/router/AGENTS.md` diff --git a/app/L0/_all/mod/_core/openai_codex/AGENTS.md b/app/L0/_all/mod/_core/openai_codex/AGENTS.md new file mode 100644 index 00000000..d6c88b51 --- /dev/null +++ b/app/L0/_all/mod/_core/openai_codex/AGENTS.md @@ -0,0 +1,122 @@ +# AGENTS + +## Purpose + +`_core/openai_codex/` owns OpenAI-Codex-specific frontend request customization for the ChatGPT Plus subscription transport. + +It is a headless helper module. It does not own chat UI or prompt assembly. It owns reusable Codex endpoint detection, Cloudflare-compatible header application, OAuth token helpers, Chat-Completions-to-Responses request-shape conversion, and Responses-API SSE event mapping for the first-party chat surfaces. + +Documentation is top priority for this module. After any change under this subtree, update this file and any affected parent or consumer docs in the same session. + +## Ownership + +This module owns: + +- `request.js`: shared Codex endpoint detection, Cloudflare-compatible request-header application, JWT account-id extraction, and OAuth device-flow URL constants +- `ext/js/_core/onscreen_agent/api.js/prepareOnscreenAgentApiRequest/end/openai-codex.js`: overlay-chat API request customization (body shape + headers) +- `ext/js/_core/admin/views/agent/api.js/prepareAdminAgentApiRequest/end/openai-codex.js`: admin-chat API request customization (body shape + headers) +- `request_shape.js`: pure stateless converters between OpenAI Chat-Completions request bodies and Codex Responses-API request bodies +- `sse_adapter.js`: pure stateless mapper from Codex Responses-API SSE events into the existing Chat-Completions-shaped delta frames that `_core/onscreen_agent/api.js` and `_core/admin/views/agent/api.js` already parse + +## Local Contracts + +- this module contributes behavior only through JS extension hooks, shared helpers, and the dedicated LLM-client subclass; it must not fork or duplicate the admin or onscreen chat runtimes +- Codex endpoint detection uses a prefix match against `CODEX_BASE_URL` (`https://chatgpt.com/backend-api/codex`) so `/responses`, `/models`, and any future sub-endpoints share the same matcher +- detection uses the configured upstream API endpoint, not the proxied fetch URL, because frontend fetches may be rerouted through `/api/proxy` +- the two shipped extension hooks may mutate the prepared API request object, including headers, body, URL, method, or extra fetch-init fields, but they must leave non-Codex requests untouched +- provider-specific HTTP policy belongs here or in similar headless provider modules, not hard-coded into `_core/onscreen_agent/llm.js` or `_core/admin/views/agent/api.js` +- the Codex `/responses` endpoint rejects `max_output_tokens` and `temperature` with HTTP 400; `request_shape.js` strips both before producing the outbound body +- the Codex `/responses` endpoint streams its own SSE event family, not the Chat-Completions `data: {choices:[...]}` stream; the SSE adapter here is the single source of truth for translating between the two formats +- refresh-token rotation is single-use and must not run concurrently from multiple browser tabs; the browser-side module must route refreshes through the server endpoint `/api/openai_codex_token_refresh` rather than posting to `auth.openai.com` directly (see `/server/api/AGENTS.md` for that endpoint contract) + +## Known Cloudflare Requirement + +The Codex endpoint sits behind Cloudflare, which denies requests from non-residential IPs that do not advertise a first-party originator. + +Required headers on every outbound request to `https://chatgpt.com/backend-api/codex/...`: + +- `User-Agent: codex_cli_rs/0.0.0 (space-agent)` — must begin with `codex_cli_rs/` +- `originator: codex_cli_rs` — lowercase header name, canonical casing from the `codex-rs` client +- `ChatGPT-Account-ID: ` — canonical casing, extracted once from the OAuth access-token JWT claim `https://api.openai.com/auth.chatgpt_account_id` and persisted alongside the tokens in user config; omit the header when extraction fails +- `Authorization: Bearer ` +- `Accept: text/event-stream` +- `Content-Type: application/json` + +Without `User-Agent` plus `originator` the endpoint returns HTTP 403 with response header `cf-mitigated: challenge` regardless of token validity. Do not remove these headers as part of a code-cleanup pass; they are load-bearing infrastructure, not stylistic choices. + +## Request Shape Conversion Contract + +`request_shape.js` exposes a pure `chatToResponsesRequest(chatBody)` converter. Input is the OpenAI Chat-Completions body produced by `createApiRequestBody(...)` in `_core/onscreen_agent/api.js` or `createRequestBody(...)` in `_core/admin/views/agent/api.js`. Output is the Codex Responses-API body. + +Conversion rules: + +- the first `role: "system"` message becomes the top-level `instructions` string and is removed from `input` +- remaining `role: "user"` and `role: "assistant"` messages become `input[]` entries with `content: [{ type: "input_text", text }]` +- multimodal `{ type: "text", text }` content parts stay as `{ type: "input_text", text }` +- multimodal `{ type: "image_url", image_url: { url, detail } }` content parts become `{ type: "input_image", image_url, detail }` +- `model` is passed through unchanged +- `stream: true` is preserved when present; the Responses endpoint streams SSE on its own regardless, but sending it keeps the contract explicit +- `store: false` is added unconditionally so Codex does not retain completions +- `max_output_tokens` is dropped; the Codex endpoint rejects it with HTTP 400 +- `temperature` is dropped; the Codex endpoint rejects it with HTTP 400 +- all other Chat-Completions-specific fields (`n`, `frequency_penalty`, `presence_penalty`, `logit_bias`, `response_format`, `tools`, `tool_choice`, `stop`) are dropped; the Codex Responses-API accepts a narrow body and any unknown field may cause HTTP 400 + +## SSE Adapter Contract + +`sse_adapter.js` exposes a pure `mapCodexEventToChatFrames(event)` mapper. Input is one parsed Codex Responses-API SSE event object. Output is an array of zero or more Chat-Completions-shaped frames plus optional `[DONE]` marker. + +### Supported events (produce frames) + +| Event type | Output | +|---|---| +| `response.output_text.delta` | `{ choices: [{ delta: { content: event.delta }, index: 0 }] }` | +| `response.refusal.delta` | `{ choices: [{ delta: { content: event.delta }, index: 0 }] }` | +| `response.completed` | `{ choices: [{ finish_reason: "stop", delta: {}, index: 0 }], usage: { prompt_tokens, completion_tokens, total_tokens } }` plus `[DONE]` | +| `response.incomplete` | `{ choices: [{ finish_reason: event.response.incomplete_details.reason \|\| "length", delta: {}, index: 0 }] }` plus `[DONE]` | +| `response.failed` | thrown Error with the upstream error message | +| `error` (standalone error event) | thrown Error with the upstream error message | + +### Ignored events (skipped silently to avoid unknown-event log noise) + +`response.created`, `response.in_progress`, `response.output_item.added`, `response.output_item.done`, `response.content_part.added`, `response.content_part.done`, `response.output_text.done`, `response.refusal.done`, `response.function_call_arguments.delta`, `response.function_call_arguments.done`, `response.reasoning_text.delta`, `response.reasoning_text.done`, `response.reasoning_summary_text.delta`, `response.reasoning_summary_text.done`, `response.audio_*`, `response.code_interpreter_*`, `response.file_search_call_*`, `response.web_search_call_*`, `response.image_gen_call_*`, `response.mcp_*`, `response.queued`, `response.output_text_annotation.added`, `response.custom_tool_call_input_*` + +### Output accumulation rule + +Text output must be accumulated live from `response.output_text.delta` events. Do not read the final reply from `response.completed.response.output` — Codex has been observed returning an empty `output` array in the final event even when deltas streamed correctly. Use `response.completed` only for `usage` and `finish_reason`. + +### End-of-stream marker + +The Codex Responses-API does not emit a `data: [DONE]` line. The adapter synthesizes `[DONE]` after `response.completed`, `response.incomplete`, `response.failed`, or a standalone `error` event so the existing Chat-Completions SSE parser in `_core/onscreen_agent/api.js` and `_core/admin/views/agent/api.js` sees the expected terminator. + +## OAuth Device Flow Reference + +The module exports four URL constants used by the server-owned OAuth endpoints and by the frontend settings UI: + +- `CODEX_OAUTH_CLIENT_ID` — `app_EMoamEEZ73f0CkXaXp7hrann` +- `CODEX_OAUTH_AUTHORIZE_URL` — `https://auth.openai.com/codex/device` — where the user pastes the displayed `user_code` +- `CODEX_OAUTH_DEVICE_CODE_URL` — `https://auth.openai.com/api/accounts/deviceauth/usercode` — server POSTs here to start a device flow +- `CODEX_OAUTH_DEVICE_TOKEN_URL` — `https://auth.openai.com/api/accounts/deviceauth/token` — server polls here until the user authorizes +- `CODEX_OAUTH_TOKEN_URL` — `https://auth.openai.com/oauth/token` — server POSTs here to exchange the authorization code for tokens and to refresh +- `CODEX_OAUTH_REDIRECT_URI` — `https://auth.openai.com/deviceauth/callback` — fixed redirect URI sent during the code-exchange step + +## Persisted Token Shape + +Codex tokens are persisted inside each chat surface's existing configuration file (`~/conf/onscreen-agent.yaml` for the overlay, `~/conf/admin-chat.yaml` for the admin agent) as a nested encrypted structure. + +When the current session is unlocked, the value stored at `openai_codex` is a `userCrypto:`-prefixed ciphertext whose plaintext is a YAML-serialized object with these fields: + +- `access_token` — current JWT access token +- `refresh_token` — current refresh token (single-use, rotates on every refresh) +- `expires_at` — Unix timestamp in seconds when `access_token` expires +- `obtained_at` — Unix timestamp in seconds when `access_token` was issued, for telemetry only +- `account_id` — ChatGPT account id extracted once from the `access_token` JWT claim `https://api.openai.com/auth.chatgpt_account_id`; empty when the token is not a JWT or the claim is missing + +The server OAuth endpoints own extraction of `account_id` and return it alongside the tokens; the frontend never re-parses the JWT during normal requests. + +## Development Guidance + +- keep provider detection small and explicit +- prefer one shared helper for endpoint matching, header mutation, and body-shape conversion so the admin and onscreen hooks stay in sync +- if additional Codex request shaping is needed later, extend the prepared request object here instead of reintroducing per-surface hard-coded branches +- keep the Cloudflare header block in `request.js` even if it looks like boilerplate; removing it breaks the endpoint with a confusing 403 +- update this file when the Codex endpoint adds new SSE event families, when Codex rejects another request-body field, or when OAuth URLs change diff --git a/app/L0/_all/mod/_core/openai_codex/request.js b/app/L0/_all/mod/_core/openai_codex/request.js new file mode 100644 index 00000000..5c576f55 --- /dev/null +++ b/app/L0/_all/mod/_core/openai_codex/request.js @@ -0,0 +1,78 @@ +export const CODEX_BASE_URL = "https://chatgpt.com/backend-api/codex"; +export const CODEX_RESPONSES_ENDPOINT = `${CODEX_BASE_URL}/responses`; +export const CODEX_MODELS_ENDPOINT = `${CODEX_BASE_URL}/models`; +export const CODEX_OAUTH_CLIENT_ID = "app_EMoamEEZ73f0CkXaXp7hrann"; +export const CODEX_OAUTH_AUTHORIZE_URL = "https://auth.openai.com/codex/device"; +export const CODEX_OAUTH_DEVICE_CODE_URL = "https://auth.openai.com/api/accounts/deviceauth/usercode"; +export const CODEX_OAUTH_DEVICE_TOKEN_URL = "https://auth.openai.com/api/accounts/deviceauth/token"; +export const CODEX_OAUTH_TOKEN_URL = "https://auth.openai.com/oauth/token"; +export const CODEX_OAUTH_REDIRECT_URI = "https://auth.openai.com/deviceauth/callback"; + +const CODEX_ORIGINATOR = "codex_cli_rs"; +const CODEX_USER_AGENT = `${CODEX_ORIGINATOR}/0.0.0 (space-agent)`; + +export function isCodexEndpoint(endpoint = "") { + const normalizedEndpoint = String(endpoint || "").trim(); + + if (!normalizedEndpoint) { + return false; + } + + return normalizedEndpoint.startsWith(CODEX_BASE_URL); +} + +export function extractChatGPTAccountId(accessToken = "") { + const token = String(accessToken || "").trim(); + + if (!token) { + return ""; + } + + try { + const parts = token.split("."); + + if (parts.length < 2) { + return ""; + } + + const payloadSegment = parts[1].replace(/-/gu, "+").replace(/_/gu, "/"); + const paddedSegment = payloadSegment + "=".repeat((4 - (payloadSegment.length % 4)) % 4); + const payload = JSON.parse(atob(paddedSegment)); + const accountId = payload?.["https://api.openai.com/auth"]?.chatgpt_account_id; + + return typeof accountId === "string" ? accountId : ""; + } catch { + return ""; + } +} + +export function applyCodexHeaders(apiRequest = {}, options = {}) { + const headers = + apiRequest?.headers && typeof apiRequest.headers === "object" + ? { ...apiRequest.headers } + : {}; + const accessToken = String(options?.accessToken || "").trim(); + const explicitAccountId = String(options?.chatGPTAccountId || "").trim(); + + // Cloudflare in front of the Codex endpoint blocks non-residential traffic unless the request + // advertises a first-party originator. Without these two headers the server returns HTTP 403 + // with `cf-mitigated: challenge` regardless of token validity. + headers["User-Agent"] = CODEX_USER_AGENT; + headers.originator = CODEX_ORIGINATOR; + headers.Accept = "text/event-stream"; + + if (accessToken) { + headers.Authorization = `Bearer ${accessToken}`; + } + + const chatGPTAccountId = explicitAccountId || extractChatGPTAccountId(accessToken); + + if (chatGPTAccountId) { + headers["ChatGPT-Account-ID"] = chatGPTAccountId; + } + + return { + ...apiRequest, + headers + }; +} From 808ed6687ccc099c9810e152087b5524e02a62a4 Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Thu, 23 Apr 2026 08:51:09 +0200 Subject: [PATCH 02/29] feat(openai-codex): request format conversion (chat to responses) Pure stateless converter from OpenAI Chat-Completions bodies into Codex Responses-API bodies, with 11 unit tests covering the shape rules documented in the module AGENTS.md. - the first system message lifts into the top-level `instructions` string and drops from `input` - remaining user/assistant messages become `input[]` entries with `content: [{ type: "input_text", text }]` - multimodal `text` parts stay as `input_text`, `image_url` parts become `input_image` - `max_output_tokens`, `temperature`, and other Chat-Completions-only fields are stripped because Codex rejects them with HTTP 400 - `store: false` is always forced so Codex does not retain completions --- .../mod/_core/openai_codex/request_shape.js | 173 +++++++++++++++ tests/openai_codex_request_shape_test.mjs | 205 ++++++++++++++++++ 2 files changed, 378 insertions(+) create mode 100644 app/L0/_all/mod/_core/openai_codex/request_shape.js create mode 100644 tests/openai_codex_request_shape_test.mjs diff --git a/app/L0/_all/mod/_core/openai_codex/request_shape.js b/app/L0/_all/mod/_core/openai_codex/request_shape.js new file mode 100644 index 00000000..dc2d4d3f --- /dev/null +++ b/app/L0/_all/mod/_core/openai_codex/request_shape.js @@ -0,0 +1,173 @@ +const CHAT_COMPLETIONS_FIELDS_TO_DROP = new Set([ + "frequency_penalty", + "logit_bias", + "max_output_tokens", + "max_tokens", + "n", + "presence_penalty", + "response_format", + "stop", + "temperature", + "tool_choice", + "tools", + "top_logprobs", + "top_p" +]); + +function isObject(value) { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +function convertContentParts(content) { + if (typeof content === "string") { + return [ + { + text: content, + type: "input_text" + } + ]; + } + + if (!Array.isArray(content)) { + return []; + } + + return content + .map((part) => { + if (typeof part === "string") { + return { + text: part, + type: "input_text" + }; + } + + if (!isObject(part)) { + return null; + } + + if (part.type === "text" || part.type === "input_text") { + return { + text: typeof part.text === "string" ? part.text : "", + type: "input_text" + }; + } + + if (part.type === "image_url") { + const imageUrl = + typeof part.image_url === "string" + ? part.image_url + : isObject(part.image_url) + ? part.image_url.url + : ""; + const detail = isObject(part.image_url) ? part.image_url.detail : undefined; + + const converted = { + image_url: imageUrl, + type: "input_image" + }; + + if (typeof detail === "string" && detail) { + converted.detail = detail; + } + + return converted; + } + + if (part.type === "input_image") { + return { ...part }; + } + + return null; + }) + .filter(Boolean); +} + +function extractInstructionsAndInput(messages) { + if (!Array.isArray(messages)) { + return { + input: [], + instructions: "" + }; + } + + let instructions = ""; + let systemConsumed = false; + const input = []; + + for (const message of messages) { + if (!isObject(message)) { + continue; + } + + const role = message.role; + + if (role === "system") { + if (!systemConsumed) { + instructions = typeof message.content === "string" + ? message.content + : convertContentParts(message.content) + .map((part) => part.text || "") + .join(""); + systemConsumed = true; + } + + continue; + } + + if (role !== "user" && role !== "assistant") { + continue; + } + + const contentParts = convertContentParts(message.content); + const hasMeaningfulPart = contentParts.some( + (part) => part.type !== "input_text" || (typeof part.text === "string" && part.text.length > 0) + ); + + if (!hasMeaningfulPart) { + continue; + } + + input.push({ + content: contentParts, + role + }); + } + + return { + input, + instructions + }; +} + +export function chatToResponsesRequest(chatBody = {}) { + const body = isObject(chatBody) ? chatBody : {}; + const { input, instructions } = extractInstructionsAndInput(body.messages); + + const responsesBody = { + input, + model: typeof body.model === "string" ? body.model : "", + store: false + }; + + if (instructions) { + responsesBody.instructions = instructions; + } + + if (body.stream === true) { + responsesBody.stream = true; + } + + for (const [key, value] of Object.entries(body)) { + if (key === "messages" || key === "model" || key === "stream" || key === "store") { + continue; + } + + if (CHAT_COMPLETIONS_FIELDS_TO_DROP.has(key)) { + continue; + } + + responsesBody[key] = value; + } + + return responsesBody; +} diff --git a/tests/openai_codex_request_shape_test.mjs b/tests/openai_codex_request_shape_test.mjs new file mode 100644 index 00000000..57dc48bc --- /dev/null +++ b/tests/openai_codex_request_shape_test.mjs @@ -0,0 +1,205 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { chatToResponsesRequest } from "../app/L0/_all/mod/_core/openai_codex/request_shape.js"; + +test("chatToResponsesRequest lifts the first system message into instructions", () => { + const body = chatToResponsesRequest({ + messages: [ + { role: "system", content: "You are helpful." }, + { role: "user", content: "Hello." } + ], + model: "gpt-5.4-mini" + }); + + assert.equal(body.instructions, "You are helpful."); + assert.equal(body.model, "gpt-5.4-mini"); + assert.deepEqual(body.input, [ + { + content: [{ text: "Hello.", type: "input_text" }], + role: "user" + } + ]); + assert.equal(body.store, false); +}); + +test("chatToResponsesRequest wraps string content into input_text parts", () => { + const body = chatToResponsesRequest({ + messages: [ + { role: "user", content: "Hi" }, + { role: "assistant", content: "Hey" }, + { role: "user", content: "How are you?" } + ], + model: "gpt-5.4" + }); + + assert.deepEqual(body.input, [ + { + content: [{ text: "Hi", type: "input_text" }], + role: "user" + }, + { + content: [{ text: "Hey", type: "input_text" }], + role: "assistant" + }, + { + content: [{ text: "How are you?", type: "input_text" }], + role: "user" + } + ]); +}); + +test("chatToResponsesRequest converts multimodal text parts to input_text", () => { + const body = chatToResponsesRequest({ + messages: [ + { + role: "user", + content: [ + { type: "text", text: "Describe" }, + { type: "text", text: "this" } + ] + } + ], + model: "gpt-5.4" + }); + + assert.deepEqual(body.input[0].content, [ + { text: "Describe", type: "input_text" }, + { text: "this", type: "input_text" } + ]); +}); + +test("chatToResponsesRequest converts image_url parts to input_image", () => { + const body = chatToResponsesRequest({ + messages: [ + { + role: "user", + content: [ + { type: "text", text: "See this:" }, + { + type: "image_url", + image_url: { url: "https://example.invalid/a.png", detail: "high" } + } + ] + } + ], + model: "gpt-5.4" + }); + + assert.deepEqual(body.input[0].content, [ + { text: "See this:", type: "input_text" }, + { + detail: "high", + image_url: "https://example.invalid/a.png", + type: "input_image" + } + ]); +}); + +test("chatToResponsesRequest drops fields the Codex endpoint rejects", () => { + const body = chatToResponsesRequest({ + frequency_penalty: 0.1, + max_output_tokens: 1000, + max_tokens: 2000, + messages: [{ role: "user", content: "ping" }], + model: "gpt-5.4-mini", + presence_penalty: 0.2, + response_format: { type: "json_object" }, + stop: ["\n\n"], + temperature: 0.5, + tool_choice: "none", + tools: [{ type: "function", function: { name: "foo" } }], + top_p: 0.9 + }); + + assert.ok(!("frequency_penalty" in body)); + assert.ok(!("max_output_tokens" in body)); + assert.ok(!("max_tokens" in body)); + assert.ok(!("presence_penalty" in body)); + assert.ok(!("response_format" in body)); + assert.ok(!("stop" in body)); + assert.ok(!("temperature" in body)); + assert.ok(!("tool_choice" in body)); + assert.ok(!("tools" in body)); + assert.ok(!("top_p" in body)); +}); + +test("chatToResponsesRequest preserves stream flag when set", () => { + const streaming = chatToResponsesRequest({ + messages: [{ role: "user", content: "ping" }], + model: "gpt-5.4", + stream: true + }); + const nonStreaming = chatToResponsesRequest({ + messages: [{ role: "user", content: "ping" }], + model: "gpt-5.4" + }); + + assert.equal(streaming.stream, true); + assert.ok(!("stream" in nonStreaming)); +}); + +test("chatToResponsesRequest always sets store:false", () => { + const body = chatToResponsesRequest({ + messages: [{ role: "user", content: "ping" }], + model: "gpt-5.4", + store: true + }); + + assert.equal(body.store, false); +}); + +test("chatToResponsesRequest keeps only the first system message as instructions", () => { + const body = chatToResponsesRequest({ + messages: [ + { role: "system", content: "First system rule." }, + { role: "user", content: "Hi" }, + { role: "system", content: "Second system rule should be ignored." }, + { role: "assistant", content: "Hey" } + ], + model: "gpt-5.4" + }); + + assert.equal(body.instructions, "First system rule."); + assert.deepEqual( + body.input.map((entry) => entry.role), + ["user", "assistant"] + ); +}); + +test("chatToResponsesRequest skips messages with empty or unsupported content", () => { + const body = chatToResponsesRequest({ + messages: [ + { role: "user", content: "" }, + { role: "tool", content: "ignored" }, + { role: "user", content: [] }, + { role: "user", content: "keep me" } + ], + model: "gpt-5.4" + }); + + assert.equal(body.input.length, 1); + assert.equal(body.input[0].content[0].text, "keep me"); +}); + +test("chatToResponsesRequest omits instructions when no system message exists", () => { + const body = chatToResponsesRequest({ + messages: [{ role: "user", content: "Hi" }], + model: "gpt-5.4" + }); + + assert.ok(!("instructions" in body)); +}); + +test("chatToResponsesRequest handles malformed body gracefully", () => { + assert.deepEqual(chatToResponsesRequest(null), { + input: [], + model: "", + store: false + }); + assert.deepEqual(chatToResponsesRequest(), { + input: [], + model: "", + store: false + }); +}); From 1928490bdf44a16a32d97e37ab8683092806e85a Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Thu, 23 Apr 2026 08:53:22 +0200 Subject: [PATCH 03/29] feat(openai-codex): SSE response event decoder Pure stateless mapper from Codex Responses-API SSE events into the Chat-Completions-shaped delta frames the existing space-agent SSE parsers already understand, with 15 unit tests covering single-event mapping plus realistic multi-event sequences. - `response.output_text.delta` and `response.refusal.delta` emit `{ choices: [{ delta: { content: delta }, index: 0 }] }` frames - `response.completed` synthesizes a finish frame with mapped usage tokens plus a `[DONE]` marker so the existing Chat-Completions stream reader terminates cleanly; the Responses-API does not emit a native `[DONE]` line - `response.incomplete` maps the Codex reason onto the closest Chat- Completions finish_reason (`max_output_tokens` -> `length`, `content_filter` passes through) - `response.failed` and standalone `error` events throw with the upstream message so the transport layer surfaces the error - all other events (content_part.added/done, output_item.added/done, reasoning, audio, tool-calls, code_interpreter, file_search, web_search, image_gen, mcp, queued, annotations, custom_tool_call input, future unknowns) are skipped silently to avoid unknown-event log noise - text is accumulated live from delta events rather than from the final `response.completed.response.output` because the Codex endpoint has been observed returning an empty `output` array even when deltas streamed correctly --- .../mod/_core/openai_codex/sse_adapter.js | 188 +++++++++++ tests/openai_codex_sse_adapter_test.mjs | 304 ++++++++++++++++++ 2 files changed, 492 insertions(+) create mode 100644 app/L0/_all/mod/_core/openai_codex/sse_adapter.js create mode 100644 tests/openai_codex_sse_adapter_test.mjs diff --git a/app/L0/_all/mod/_core/openai_codex/sse_adapter.js b/app/L0/_all/mod/_core/openai_codex/sse_adapter.js new file mode 100644 index 00000000..35417118 --- /dev/null +++ b/app/L0/_all/mod/_core/openai_codex/sse_adapter.js @@ -0,0 +1,188 @@ +export const CODEX_STREAM_DONE_MARKER = "[DONE]"; + +const IGNORED_EVENT_TYPES = new Set([ + "response.audio.delta", + "response.audio.done", + "response.audio_transcript.delta", + "response.audio_transcript.done", + "response.code_interpreter_call_code.delta", + "response.code_interpreter_call_code.done", + "response.code_interpreter_call.completed", + "response.code_interpreter_call.in_progress", + "response.code_interpreter_call.interpreting", + "response.content_part.added", + "response.content_part.done", + "response.created", + "response.custom_tool_call_input.delta", + "response.custom_tool_call_input.done", + "response.file_search_call.completed", + "response.file_search_call.in_progress", + "response.file_search_call.searching", + "response.function_call_arguments.delta", + "response.function_call_arguments.done", + "response.image_gen_call.completed", + "response.image_gen_call.generating", + "response.image_gen_call.in_progress", + "response.image_gen_call.partial_image", + "response.in_progress", + "response.mcp_call.completed", + "response.mcp_call.failed", + "response.mcp_call.in_progress", + "response.mcp_call_arguments.delta", + "response.mcp_call_arguments.done", + "response.mcp_list_tools.completed", + "response.mcp_list_tools.failed", + "response.mcp_list_tools.in_progress", + "response.output_item.added", + "response.output_item.done", + "response.output_text.done", + "response.output_text_annotation.added", + "response.queued", + "response.reasoning_summary_part.added", + "response.reasoning_summary_part.done", + "response.reasoning_summary_text.delta", + "response.reasoning_summary_text.done", + "response.reasoning_text.delta", + "response.reasoning_text.done", + "response.refusal.done", + "response.web_search_call.completed", + "response.web_search_call.in_progress", + "response.web_search_call.searching" +]); + +function isObject(value) { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +function buildTextDeltaFrame(delta) { + return { + choices: [ + { + delta: { content: delta }, + index: 0 + } + ] + }; +} + +function buildFinishFrame(finishReason, usage) { + const frame = { + choices: [ + { + delta: {}, + finish_reason: finishReason, + index: 0 + } + ] + }; + + if (usage && isObject(usage)) { + frame.usage = { + completion_tokens: Number.isFinite(usage.output_tokens) ? usage.output_tokens : 0, + prompt_tokens: Number.isFinite(usage.input_tokens) ? usage.input_tokens : 0, + total_tokens: Number.isFinite(usage.total_tokens) ? usage.total_tokens : 0 + }; + } + + return frame; +} + +function readEventType(event) { + if (!isObject(event)) { + return ""; + } + + return typeof event.type === "string" ? event.type : ""; +} + +function readErrorMessage(event) { + if (!isObject(event)) { + return "Codex stream failed with an unknown error."; + } + + if (typeof event.message === "string" && event.message) { + return event.message; + } + + const nestedError = event.response?.error; + + if (isObject(nestedError) && typeof nestedError.message === "string" && nestedError.message) { + return nestedError.message; + } + + return "Codex stream failed without an error message."; +} + +export function mapCodexEventToChatFrames(event) { + const eventType = readEventType(event); + + if (!eventType) { + return []; + } + + if (IGNORED_EVENT_TYPES.has(eventType)) { + return []; + } + + switch (eventType) { + case "response.output_text.delta": + case "response.refusal.delta": { + const delta = typeof event.delta === "string" ? event.delta : ""; + + if (!delta) { + return []; + } + + return [buildTextDeltaFrame(delta)]; + } + + case "response.completed": { + const usage = event.response?.usage; + return [buildFinishFrame("stop", usage), CODEX_STREAM_DONE_MARKER]; + } + + case "response.incomplete": { + const reason = event.response?.incomplete_details?.reason; + const finishReason = + reason === "max_output_tokens" + ? "length" + : reason === "content_filter" + ? "content_filter" + : reason === "max_tokens" + ? "length" + : typeof reason === "string" && reason + ? reason + : "length"; + const usage = event.response?.usage; + + return [buildFinishFrame(finishReason, usage), CODEX_STREAM_DONE_MARKER]; + } + + case "response.failed": + case "error": { + throw new Error(readErrorMessage(event)); + } + + default: + // Unknown event type - skip silently so future Codex additions do not break streaming. + return []; + } +} + +export function mapCodexEventSequenceToChatFrames(events) { + if (!Array.isArray(events)) { + return []; + } + + const frames = []; + + for (const event of events) { + const eventFrames = mapCodexEventToChatFrames(event); + + for (const frame of eventFrames) { + frames.push(frame); + } + } + + return frames; +} diff --git a/tests/openai_codex_sse_adapter_test.mjs b/tests/openai_codex_sse_adapter_test.mjs new file mode 100644 index 00000000..42478aa5 --- /dev/null +++ b/tests/openai_codex_sse_adapter_test.mjs @@ -0,0 +1,304 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + CODEX_STREAM_DONE_MARKER, + mapCodexEventSequenceToChatFrames, + mapCodexEventToChatFrames +} from "../app/L0/_all/mod/_core/openai_codex/sse_adapter.js"; + +test("mapCodexEventToChatFrames emits content frames for output_text deltas", () => { + const frames = mapCodexEventToChatFrames({ + delta: "Hel", + item_id: "msg_1", + output_index: 0, + sequence_number: 5, + type: "response.output_text.delta" + }); + + assert.deepEqual(frames, [ + { + choices: [ + { + delta: { content: "Hel" }, + index: 0 + } + ] + } + ]); +}); + +test("mapCodexEventToChatFrames emits content frames for refusal deltas", () => { + const frames = mapCodexEventToChatFrames({ + delta: "I cannot", + type: "response.refusal.delta" + }); + + assert.equal(frames.length, 1); + assert.equal(frames[0].choices[0].delta.content, "I cannot"); +}); + +test("mapCodexEventToChatFrames skips empty text deltas", () => { + const frames = mapCodexEventToChatFrames({ + delta: "", + type: "response.output_text.delta" + }); + + assert.deepEqual(frames, []); +}); + +test("mapCodexEventToChatFrames synthesizes finish + [DONE] on response.completed", () => { + const frames = mapCodexEventToChatFrames({ + response: { + usage: { + input_tokens: 12, + output_tokens: 3, + total_tokens: 15 + } + }, + sequence_number: 99, + type: "response.completed" + }); + + assert.equal(frames.length, 2); + assert.deepEqual(frames[0], { + choices: [ + { + delta: {}, + finish_reason: "stop", + index: 0 + } + ], + usage: { + completion_tokens: 3, + prompt_tokens: 12, + total_tokens: 15 + } + }); + assert.equal(frames[1], CODEX_STREAM_DONE_MARKER); +}); + +test("mapCodexEventToChatFrames handles response.completed without usage", () => { + const frames = mapCodexEventToChatFrames({ + response: {}, + type: "response.completed" + }); + + assert.equal(frames.length, 2); + assert.equal(frames[0].choices[0].finish_reason, "stop"); + assert.ok(!("usage" in frames[0])); + assert.equal(frames[1], CODEX_STREAM_DONE_MARKER); +}); + +test("mapCodexEventToChatFrames maps incomplete reasons onto chat finish reasons", () => { + const lengthFrames = mapCodexEventToChatFrames({ + response: { + incomplete_details: { reason: "max_output_tokens" } + }, + type: "response.incomplete" + }); + assert.equal(lengthFrames[0].choices[0].finish_reason, "length"); + assert.equal(lengthFrames[1], CODEX_STREAM_DONE_MARKER); + + const filterFrames = mapCodexEventToChatFrames({ + response: { + incomplete_details: { reason: "content_filter" } + }, + type: "response.incomplete" + }); + assert.equal(filterFrames[0].choices[0].finish_reason, "content_filter"); + + const unknownFrames = mapCodexEventToChatFrames({ + response: {}, + type: "response.incomplete" + }); + assert.equal(unknownFrames[0].choices[0].finish_reason, "length"); +}); + +test("mapCodexEventToChatFrames throws on response.failed", () => { + assert.throws( + () => + mapCodexEventToChatFrames({ + response: { + error: { message: "upstream boom" } + }, + type: "response.failed" + }), + /upstream boom/u + ); +}); + +test("mapCodexEventToChatFrames throws on standalone error event", () => { + assert.throws( + () => + mapCodexEventToChatFrames({ + message: "rate limited", + type: "error" + }), + /rate limited/u + ); +}); + +test("mapCodexEventToChatFrames silently ignores documented low-level events", () => { + const ignoredTypes = [ + "response.created", + "response.in_progress", + "response.output_item.added", + "response.output_item.done", + "response.content_part.added", + "response.content_part.done", + "response.output_text.done", + "response.refusal.done", + "response.function_call_arguments.delta", + "response.function_call_arguments.done", + "response.reasoning_text.delta", + "response.reasoning_summary_text.delta", + "response.queued", + "response.output_text_annotation.added", + "response.audio.delta", + "response.code_interpreter_call.in_progress", + "response.file_search_call.searching", + "response.web_search_call.completed", + "response.image_gen_call.generating", + "response.mcp_call.in_progress", + "response.custom_tool_call_input.delta" + ]; + + for (const type of ignoredTypes) { + const frames = mapCodexEventToChatFrames({ type }); + assert.deepEqual(frames, [], `expected ${type} to produce no frames`); + } +}); + +test("mapCodexEventToChatFrames ignores unknown future events without throwing", () => { + assert.deepEqual( + mapCodexEventToChatFrames({ type: "response.some_future_event" }), + [] + ); +}); + +test("mapCodexEventToChatFrames handles malformed input defensively", () => { + assert.deepEqual(mapCodexEventToChatFrames(null), []); + assert.deepEqual(mapCodexEventToChatFrames(undefined), []); + assert.deepEqual(mapCodexEventToChatFrames("string"), []); + assert.deepEqual(mapCodexEventToChatFrames({}), []); + assert.deepEqual(mapCodexEventToChatFrames({ type: 42 }), []); +}); + +test("mapCodexEventSequenceToChatFrames processes a realistic happy-path stream", () => { + const events = [ + { response: {}, sequence_number: 0, type: "response.created" }, + { sequence_number: 1, type: "response.in_progress" }, + { item: {}, output_index: 0, sequence_number: 2, type: "response.output_item.added" }, + { + content_index: 0, + item_id: "msg_1", + output_index: 0, + part: {}, + sequence_number: 3, + type: "response.content_part.added" + }, + { + content_index: 0, + delta: "Hel", + item_id: "msg_1", + output_index: 0, + sequence_number: 4, + type: "response.output_text.delta" + }, + { + content_index: 0, + delta: "lo", + item_id: "msg_1", + output_index: 0, + sequence_number: 5, + type: "response.output_text.delta" + }, + { + content_index: 0, + delta: "!", + item_id: "msg_1", + output_index: 0, + sequence_number: 6, + type: "response.output_text.delta" + }, + { + content_index: 0, + item_id: "msg_1", + output_index: 0, + sequence_number: 7, + text: "Hello!", + type: "response.output_text.done" + }, + { item: {}, output_index: 0, sequence_number: 8, type: "response.output_item.done" }, + { + response: { + usage: { input_tokens: 8, output_tokens: 3, total_tokens: 11 } + }, + sequence_number: 9, + type: "response.completed" + } + ]; + + const frames = mapCodexEventSequenceToChatFrames(events); + + // 3 text deltas + 1 finish frame + 1 [DONE] marker + assert.equal(frames.length, 5); + assert.equal(frames[0].choices[0].delta.content, "Hel"); + assert.equal(frames[1].choices[0].delta.content, "lo"); + assert.equal(frames[2].choices[0].delta.content, "!"); + assert.equal(frames[3].choices[0].finish_reason, "stop"); + assert.deepEqual(frames[3].usage, { + completion_tokens: 3, + prompt_tokens: 8, + total_tokens: 11 + }); + assert.equal(frames[4], CODEX_STREAM_DONE_MARKER); +}); + +test("mapCodexEventSequenceToChatFrames processes a refusal sequence", () => { + const events = [ + { response: {}, type: "response.created" }, + { + delta: "I cannot ", + type: "response.refusal.delta" + }, + { + delta: "help with that.", + type: "response.refusal.delta" + }, + { + text: "I cannot help with that.", + type: "response.refusal.done" + }, + { + response: { + usage: { input_tokens: 5, output_tokens: 8, total_tokens: 13 } + }, + type: "response.completed" + } + ]; + + const frames = mapCodexEventSequenceToChatFrames(events); + + assert.equal(frames.length, 4); + assert.equal(frames[0].choices[0].delta.content, "I cannot "); + assert.equal(frames[1].choices[0].delta.content, "help with that."); + assert.equal(frames[2].choices[0].finish_reason, "stop"); + assert.equal(frames[3], CODEX_STREAM_DONE_MARKER); +}); + +test("mapCodexEventSequenceToChatFrames passes errors through the middle of a stream", () => { + const events = [ + { type: "response.created" }, + { delta: "Start", type: "response.output_text.delta" }, + { message: "mid-stream failure", type: "error" } + ]; + + assert.throws(() => mapCodexEventSequenceToChatFrames(events), /mid-stream failure/u); +}); + +test("mapCodexEventSequenceToChatFrames tolerates non-array input", () => { + assert.deepEqual(mapCodexEventSequenceToChatFrames(null), []); + assert.deepEqual(mapCodexEventSequenceToChatFrames("not-an-array"), []); +}); From b26519132bc35126e5a015e7283a948b5ad95453 Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Thu, 23 Apr 2026 08:56:49 +0200 Subject: [PATCH 04/29] feat(openai-codex): OAuth device-code flow on the server Adds three authenticated endpoints that own the OAuth device-code flow and refresh-token rotation against `auth.openai.com`, backed by a new `server/lib/openai_codex/` helper subsystem. Endpoints: - `POST /api/openai_codex_auth_start` -> returns `{ deviceAuthId, userCode, verificationUrl, interval, expiresIn }` - `POST /api/openai_codex_auth_poll` -> returns `{ status: "pending" }` until the user authorizes, then `{ status: "complete", tokens }` - `POST /api/openai_codex_token_refresh` -> returns a refreshed token payload; maps OAuth `invalid_grant` onto HTTP 401 so the frontend can prompt a fresh login when the refresh token was consumed elsewhere Why this lives on the server: OpenAI refresh tokens use single-use rotation. A frontend-only implementation cannot serialize concurrent tab refreshes safely: both calls post the same single-use token, one succeeds, one returns `invalid_grant`, and the only valid refresh token is lost. Full re-authentication becomes the only recovery. This is a shared-data integrity concern under the rule in `/server/AGENTS.md`. The server layer provides: - `oauth_client.js` pure transport functions for the three OAuth calls with defensive JSON body parsing, JWT account-id extraction from the `access_token` (never `id_token`), and `502` mapping for upstream failures - `refresh_lock.js` in-process single-writer coalescer keyed by the refresh-token string; documented limitation: in clustered runtime (WORKERS>1) different workers still race, which is acceptable for the single-user single-browser-profile scenario and documented in `server/lib/openai_codex/AGENTS.md` Token persistence stays on the frontend under `userCrypto:`-prefixed encryption; the server never reads or writes tokens from the app tree. No revoke endpoint is exposed because logout clears the encrypted config entry and access tokens expire within about an hour. Docs: - `server/lib/openai_codex/AGENTS.md` new contract doc - `server/api/AGENTS.md` documents the new endpoint family and its backend-ownership rationale - `server/AGENTS.md` index and structure updated - root `AGENTS.md` index updated --- AGENTS.md | 1 + server/AGENTS.md | 2 + server/api/AGENTS.md | 17 ++ server/api/openai_codex_auth_poll.js | 26 +++ server/api/openai_codex_auth_start.js | 18 ++ server/api/openai_codex_token_refresh.js | 29 ++++ server/lib/openai_codex/AGENTS.md | 50 ++++++ server/lib/openai_codex/oauth_client.js | 200 +++++++++++++++++++++++ server/lib/openai_codex/refresh_lock.js | 29 ++++ 9 files changed, 372 insertions(+) create mode 100644 server/api/openai_codex_auth_poll.js create mode 100644 server/api/openai_codex_auth_start.js create mode 100644 server/api/openai_codex_token_refresh.js create mode 100644 server/lib/openai_codex/AGENTS.md create mode 100644 server/lib/openai_codex/oauth_client.js create mode 100644 server/lib/openai_codex/refresh_lock.js diff --git a/AGENTS.md b/AGENTS.md index fbe84ba1..1a7891d8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -124,6 +124,7 @@ Server docs: - `/server/lib/customware/AGENTS.md` - `/server/lib/file_watch/AGENTS.md` - `/server/lib/git/AGENTS.md` +- `/server/lib/openai_codex/AGENTS.md` - `/server/lib/share/AGENTS.md` - `/server/lib/tmp/AGENTS.md` - `/server/pages/AGENTS.md` diff --git a/server/AGENTS.md b/server/AGENTS.md index 9828c8ff..31c09ada 100644 --- a/server/AGENTS.md +++ b/server/AGENTS.md @@ -34,6 +34,7 @@ Current subsystem-local docs in the server tree: - `server/lib/share/AGENTS.md` - `server/lib/tmp/AGENTS.md` - `server/lib/git/AGENTS.md` +- `server/lib/openai_codex/AGENTS.md` Update rules: @@ -121,6 +122,7 @@ Current server layout: - `server/lib/share/`: backend-owned hosted-share archive storage, ZIP validation, authenticated import, and anonymous guest-clone helpers - `server/lib/tmp/`: `server/tmp/` lifecycle, stale-entry cleanup, and low-RAM ZIP archive creation for attachment-style downloads - `server/lib/git/`: Git backend abstraction used by update flows and Git-backed module installs +- `server/lib/openai_codex/`: backend helpers for the OpenAI Codex (ChatGPT Plus) OAuth device-code flow and single-writer refresh-token rotation used by the `openai_codex_auth_*` and `openai_codex_token_refresh` endpoints - `server/tmp/`: transient disk-backed artifacts such as folder-download ZIP files ## Request Flow And Runtime Contracts diff --git a/server/api/AGENTS.md b/server/api/AGENTS.md index 0475d294..2a2df1ef 100644 --- a/server/api/AGENTS.md +++ b/server/api/AGENTS.md @@ -139,6 +139,23 @@ Important notes: - `user_self_info` returns the authenticated user's derived identity plus browser-bootstrap crypto metadata: `{ username, fullName, groups, managedGroups, sessionId, userCryptoKeyId, userCryptoState }` - `password_generate` is an authenticated utility endpoint that returns the backend-sealed `password.json` payload and should stay narrow +OpenAI Codex OAuth endpoints: + +- `openai_codex_auth_start` +- `openai_codex_auth_poll` +- `openai_codex_token_refresh` + +Current rules: + +- these endpoints back the ChatGPT Plus subscription transport for the first-party chat surfaces and authenticate through the normal `space_session` cookie; anonymous access is not allowed +- they are backend-owned because OpenAI refresh tokens use single-use rotation, which frontend-only code cannot serialize safely between concurrent browser tabs without losing the token and forcing a full re-login; this is a shared-data integrity concern under the rule in `/server/AGENTS.md` +- they delegate all OAuth traffic to `server/lib/openai_codex/oauth_client.js` and use the in-process mutex in `server/lib/openai_codex/refresh_lock.js` to coalesce concurrent refresh calls that share the same refresh token +- `openai_codex_auth_start` accepts `POST` with no body; it returns `{ deviceAuthId, userCode, verificationUrl, interval, expiresIn }` so the frontend can show the user the code and poll +- `openai_codex_auth_poll` accepts `POST` with `{ deviceAuthId, userCode }`; it returns `{ status: "pending" }` until the user authorizes, then `{ status: "complete", tokens: { accessToken, refreshToken, idToken, expiresAt, obtainedAt, accountId } }` +- `openai_codex_token_refresh` accepts `POST` with `{ refreshToken }`; it returns a fresh token payload in the same shape, or HTTP 401 when the refresh token has already been consumed (e.g. by a parallel Codex client) +- token storage stays on the frontend as `userCrypto:`-prefixed ciphertext in `~/conf/onscreen-agent.yaml` and `~/conf/admin-chat.yaml`; these endpoints never read or persist tokens themselves +- there is no revoke endpoint; logout clears the encrypted config entry on the frontend because access tokens expire within about an hour and OpenAI's device-flow issues only one refresh token per device + ## Handler Contract Handlers receive the request context assembled by `server/router/router.js`, including: diff --git a/server/api/openai_codex_auth_poll.js b/server/api/openai_codex_auth_poll.js new file mode 100644 index 00000000..15b78bf7 --- /dev/null +++ b/server/api/openai_codex_auth_poll.js @@ -0,0 +1,26 @@ +import { pollDeviceAuthorization } from "../lib/openai_codex/oauth_client.js"; + +function createHttpError(message, statusCode) { + const error = new Error(message); + error.statusCode = statusCode; + return error; +} + +export async function post(context) { + const payload = + context.body && typeof context.body === "object" && !Buffer.isBuffer(context.body) + ? context.body + : {}; + + try { + return await pollDeviceAuthorization({ + deviceAuthId: payload.deviceAuthId, + userCode: payload.userCode + }); + } catch (error) { + throw createHttpError( + error.message || "Failed to poll Codex device authorization.", + Number(error.statusCode) || 502 + ); + } +} diff --git a/server/api/openai_codex_auth_start.js b/server/api/openai_codex_auth_start.js new file mode 100644 index 00000000..bc4d2151 --- /dev/null +++ b/server/api/openai_codex_auth_start.js @@ -0,0 +1,18 @@ +import { startDeviceAuthorization } from "../lib/openai_codex/oauth_client.js"; + +function createHttpError(message, statusCode) { + const error = new Error(message); + error.statusCode = statusCode; + return error; +} + +export async function post() { + try { + return await startDeviceAuthorization(); + } catch (error) { + throw createHttpError( + error.message || "Failed to start Codex device authorization.", + Number(error.statusCode) || 502 + ); + } +} diff --git a/server/api/openai_codex_token_refresh.js b/server/api/openai_codex_token_refresh.js new file mode 100644 index 00000000..88ae7327 --- /dev/null +++ b/server/api/openai_codex_token_refresh.js @@ -0,0 +1,29 @@ +import { refreshAccessToken } from "../lib/openai_codex/oauth_client.js"; +import { runSingleWriterRefresh } from "../lib/openai_codex/refresh_lock.js"; + +function createHttpError(message, statusCode) { + const error = new Error(message); + error.statusCode = statusCode; + return error; +} + +export async function post(context) { + const payload = + context.body && typeof context.body === "object" && !Buffer.isBuffer(context.body) + ? context.body + : {}; + const refreshToken = String(payload.refreshToken || "").trim(); + + if (!refreshToken) { + throw createHttpError("refreshToken is required.", 400); + } + + try { + return await runSingleWriterRefresh(refreshToken, () => refreshAccessToken({ refreshToken })); + } catch (error) { + throw createHttpError( + error.message || "Failed to refresh Codex access token.", + Number(error.statusCode) || 502 + ); + } +} diff --git a/server/lib/openai_codex/AGENTS.md b/server/lib/openai_codex/AGENTS.md new file mode 100644 index 00000000..2473bf04 --- /dev/null +++ b/server/lib/openai_codex/AGENTS.md @@ -0,0 +1,50 @@ +# AGENTS + +## Purpose + +`server/lib/openai_codex/` owns the backend helpers for the OpenAI Codex (ChatGPT Plus) OAuth device-code flow and refresh-token rotation. + +It is a thin helper layer: endpoint modules under `server/api/` delegate to these helpers for the OAuth HTTP traffic and for the single-writer refresh mutex. These helpers must not read or write app files, must not touch sessions, and must not depend on the frontend runtime. + +## Why This Lives On The Server + +The root `server/AGENTS.md` reserves backend code for "security, shared-data integrity, multi-user isolation, or runtime-stability" concerns that the frontend cannot own safely. The Codex OAuth refresh path meets the *shared-data integrity* bar: OpenAI refresh tokens use single-use rotation — if two browser tabs refresh concurrently, exactly one call succeeds and the other returns `invalid_grant`, discarding the only valid refresh token the user has. Full re-authentication becomes the only recovery. A frontend-only implementation cannot provide the serialization guarantee needed to prevent that loss. + +The device-code and token-exchange calls additionally leverage server-side fetch so the flow keeps working even when browsers block cross-origin POSTs to `auth.openai.com`. + +## Ownership + +This subsystem owns: + +- `oauth_client.js`: pure OAuth transport functions (`startDeviceAuthorization`, `pollDeviceAuthorization`, `refreshAccessToken`), OAuth URL constants, and JWT account-id extraction +- `refresh_lock.js`: in-process single-writer coalescer (`runSingleWriterRefresh`) that prevents two concurrent callers from consuming the same refresh token + +## Contracts + +- `startDeviceAuthorization()` returns `{ deviceAuthId, expiresIn, interval, userCode, verificationUrl }` on success or throws an error with a `statusCode` property +- `pollDeviceAuthorization({ deviceAuthId, userCode })` returns `{ status: "pending" }` while the user has not yet entered the code, or `{ status: "complete", tokens }` once the token exchange succeeds +- `refreshAccessToken({ refreshToken })` returns the full token payload described below; it throws a `401` error with `invalid_grant` mapped to a human-readable message when the refresh token has already been consumed, and `502` for other upstream failures +- the returned `tokens` payload shape is: + ``` + { + accessToken: string, + refreshToken: string, + idToken: string, + expiresAt: number, // unix seconds + obtainedAt: number, // unix seconds + accountId: string // empty when the JWT cannot be parsed or the claim is missing + } + ``` +- `runSingleWriterRefresh(refreshToken, worker)` coalesces concurrent calls with the same `refreshToken` string into one inflight promise so the single-use token is never posted twice at the same time; it does not persist across process restarts and does not span worker processes in clustered runtime + +## Known Limitations + +- the refresh mutex is in-process only; when `WORKERS>1`, concurrent refreshes on different worker processes can still race. This is acceptable for the typical single-user browser scenario because one user's tokens are stored in one browser profile and only one Codex client uses them at a time; clustered deployments that expect multiple workers to refresh the same token concurrently must elevate the lock into `server/runtime/state_system.js` named locks +- this subsystem does not read or persist tokens; persistence stays on the frontend under `userCrypto:`-prefixed encryption in `~/conf/onscreen-agent.yaml` and `~/conf/admin-chat.yaml` +- there is no revoke endpoint; logout clears the encrypted config entry on the frontend, which is sufficient because access tokens expire within about an hour and OpenAI's device-flow issues one refresh token per device + +## Development Guidance + +- keep these helpers pure and side-effect-free apart from the network calls and the inflight refresh map +- never import these helpers from other layers than `server/api/openai_codex_*.js` endpoints +- when OpenAI changes the OAuth URLs, client id, or device-flow response shape, update this file, `oauth_client.js`, and the matching `/app/L0/_all/mod/_core/openai_codex/` constants in the same session diff --git a/server/lib/openai_codex/oauth_client.js b/server/lib/openai_codex/oauth_client.js new file mode 100644 index 00000000..648eb692 --- /dev/null +++ b/server/lib/openai_codex/oauth_client.js @@ -0,0 +1,200 @@ +export const CODEX_OAUTH_CLIENT_ID = "app_EMoamEEZ73f0CkXaXp7hrann"; +export const CODEX_OAUTH_DEVICE_CODE_URL = "https://auth.openai.com/api/accounts/deviceauth/usercode"; +export const CODEX_OAUTH_DEVICE_TOKEN_URL = "https://auth.openai.com/api/accounts/deviceauth/token"; +export const CODEX_OAUTH_TOKEN_URL = "https://auth.openai.com/oauth/token"; +export const CODEX_OAUTH_REDIRECT_URI = "https://auth.openai.com/deviceauth/callback"; +export const CODEX_OAUTH_VERIFICATION_URL = "https://auth.openai.com/codex/device"; + +function createHttpError(message, statusCode) { + const error = new Error(message); + error.statusCode = statusCode; + return error; +} + +function extractChatGPTAccountId(accessToken) { + const token = typeof accessToken === "string" ? accessToken.trim() : ""; + + if (!token) { + return ""; + } + + try { + const parts = token.split("."); + + if (parts.length < 2) { + return ""; + } + + const payload = JSON.parse(Buffer.from(parts[1], "base64url").toString("utf-8")); + const accountId = payload?.["https://api.openai.com/auth"]?.chatgpt_account_id; + + return typeof accountId === "string" ? accountId : ""; + } catch { + return ""; + } +} + +async function readJsonBody(response) { + const text = await response.text(); + + if (!text) { + return {}; + } + + try { + return JSON.parse(text); + } catch { + return { raw: text }; + } +} + +export async function startDeviceAuthorization() { + const response = await fetch(CODEX_OAUTH_DEVICE_CODE_URL, { + body: JSON.stringify({ client_id: CODEX_OAUTH_CLIENT_ID }), + headers: { "Content-Type": "application/json" }, + method: "POST" + }); + + const payload = await readJsonBody(response); + + if (!response.ok) { + throw createHttpError( + payload.error_description || payload.error || `Device authorization failed with HTTP ${response.status}.`, + response.status === 429 ? 429 : 502 + ); + } + + return { + deviceAuthId: String(payload.device_auth_id || "").trim(), + expiresIn: Number.isFinite(payload.expires_in) ? payload.expires_in : 900, + interval: Number.isFinite(payload.interval) && payload.interval >= 3 ? payload.interval : 3, + userCode: String(payload.user_code || "").trim(), + verificationUrl: CODEX_OAUTH_VERIFICATION_URL + }; +} + +function buildTokenPayload(accessToken, refreshToken, expiresIn, idToken) { + const normalizedExpiresIn = Number.isFinite(expiresIn) && expiresIn > 0 ? expiresIn : 3600; + const obtainedAt = Math.floor(Date.now() / 1000); + const accessTokenValue = typeof accessToken === "string" ? accessToken : ""; + const refreshTokenValue = typeof refreshToken === "string" ? refreshToken : ""; + + return { + accessToken: accessTokenValue, + accountId: extractChatGPTAccountId(accessTokenValue), + expiresAt: obtainedAt + normalizedExpiresIn, + idToken: typeof idToken === "string" ? idToken : "", + obtainedAt, + refreshToken: refreshTokenValue + }; +} + +export async function pollDeviceAuthorization({ deviceAuthId, userCode }) { + const normalizedDeviceAuthId = String(deviceAuthId || "").trim(); + const normalizedUserCode = String(userCode || "").trim(); + + if (!normalizedDeviceAuthId || !normalizedUserCode) { + throw createHttpError("deviceAuthId and userCode are required.", 400); + } + + const pollResponse = await fetch(CODEX_OAUTH_DEVICE_TOKEN_URL, { + body: JSON.stringify({ + device_auth_id: normalizedDeviceAuthId, + user_code: normalizedUserCode + }), + headers: { "Content-Type": "application/json" }, + method: "POST" + }); + + if (pollResponse.status === 403 || pollResponse.status === 404) { + return { status: "pending" }; + } + + const pollPayload = await readJsonBody(pollResponse); + + if (!pollResponse.ok) { + throw createHttpError( + pollPayload.error_description || pollPayload.error || `Device poll failed with HTTP ${pollResponse.status}.`, + pollResponse.status === 429 ? 429 : 502 + ); + } + + const authorizationCode = String(pollPayload.authorization_code || "").trim(); + const codeVerifier = String(pollPayload.code_verifier || "").trim(); + + if (!authorizationCode || !codeVerifier) { + return { status: "pending" }; + } + + const formBody = new URLSearchParams({ + client_id: CODEX_OAUTH_CLIENT_ID, + code: authorizationCode, + code_verifier: codeVerifier, + grant_type: "authorization_code", + redirect_uri: CODEX_OAUTH_REDIRECT_URI + }); + + const tokenResponse = await fetch(CODEX_OAUTH_TOKEN_URL, { + body: formBody.toString(), + headers: { "Content-Type": "application/x-www-form-urlencoded" }, + method: "POST" + }); + + const tokenPayload = await readJsonBody(tokenResponse); + + if (!tokenResponse.ok) { + throw createHttpError( + tokenPayload.error_description || tokenPayload.error || `Token exchange failed with HTTP ${tokenResponse.status}.`, + 502 + ); + } + + return { + status: "complete", + tokens: buildTokenPayload( + tokenPayload.access_token, + tokenPayload.refresh_token, + tokenPayload.expires_in, + tokenPayload.id_token + ) + }; +} + +export async function refreshAccessToken({ refreshToken }) { + const normalizedRefreshToken = String(refreshToken || "").trim(); + + if (!normalizedRefreshToken) { + throw createHttpError("refreshToken is required.", 400); + } + + const formBody = new URLSearchParams({ + client_id: CODEX_OAUTH_CLIENT_ID, + grant_type: "refresh_token", + refresh_token: normalizedRefreshToken + }); + + const response = await fetch(CODEX_OAUTH_TOKEN_URL, { + body: formBody.toString(), + headers: { "Content-Type": "application/x-www-form-urlencoded" }, + method: "POST" + }); + + const payload = await readJsonBody(response); + + if (!response.ok) { + const errorCode = String(payload.error || "").trim(); + // Bubble up `invalid_grant` as 401 so the frontend knows the refresh token + // has already been consumed (e.g. by another Codex client) and a full + // re-login is required. + if (errorCode === "invalid_grant") { + throw createHttpError("Refresh token is no longer valid. Please log in again.", 401); + } + + throw createHttpError( + payload.error_description || errorCode || `Token refresh failed with HTTP ${response.status}.`, + 502 + ); + } + + return buildTokenPayload(payload.access_token, payload.refresh_token, payload.expires_in, payload.id_token); +} diff --git a/server/lib/openai_codex/refresh_lock.js b/server/lib/openai_codex/refresh_lock.js new file mode 100644 index 00000000..91bce38e --- /dev/null +++ b/server/lib/openai_codex/refresh_lock.js @@ -0,0 +1,29 @@ +const inFlightRefreshes = new Map(); + +export async function runSingleWriterRefresh(refreshToken, worker) { + const key = typeof refreshToken === "string" ? refreshToken : ""; + + if (!key) { + return worker(); + } + + const existing = inFlightRefreshes.get(key); + + if (existing) { + // Coalesce concurrent refreshes for the same refresh token so we never + // post the same single-use token twice at the same moment; that would + // consume it twice and leave one caller with invalid_grant. + return existing; + } + + const pending = (async () => { + try { + return await worker(); + } finally { + inFlightRefreshes.delete(key); + } + })(); + + inFlightRefreshes.set(key, pending); + return pending; +} From ec44e6eab331b1897e358aded26d78e74acf2da1 Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Thu, 23 Apr 2026 08:59:40 +0200 Subject: [PATCH 05/29] feat(openai-codex): frontend token manager with always-fresh refresh Browser-side helper that wraps the three OAuth backend endpoints and guarantees the locally persisted refresh token is the one actually used at refresh time, avoiding single-use-rotation loss across tabs. - `ensureFreshCodexAccessToken({ loadTokens, saveTokens, ... })` re-reads persisted tokens on every call rather than trusting an in-memory copy; another tab or process may have rotated the refresh token, and the single-use rotation rule means a stale in-memory refresh_token would fail with invalid_grant on next refresh - refresh is triggered when the access token is within the default 300s safety margin of expiry; the persisted `expires_at` timestamp is computed server-side from the OAuth `expires_in` response so both chat surfaces share one source of truth - concurrent refresh calls that observe the same stale refresh_token are coalesced into one network request via an in-module map of in-flight promises, on top of the separate server-side mutex in `server/lib/openai_codex/refresh_lock.js` - `saveTokens` failures are logged via console.warn but do not block the active request; the refreshed tokens are still returned so the LLM call can proceed - thin wrappers `startCodexDeviceAuthorization` and `pollCodexDeviceAuthorization` expose the first two OAuth endpoints for the upcoming settings UI - 11 unit tests cover not-expiring, refreshing, always-fresh read, concurrent coalesce, missing tokens, invalid_grant propagation, and save-failure resilience --- .../mod/_core/openai_codex/token_manager.js | 185 ++++++++++ tests/openai_codex_token_manager_test.mjs | 316 ++++++++++++++++++ 2 files changed, 501 insertions(+) create mode 100644 app/L0/_all/mod/_core/openai_codex/token_manager.js create mode 100644 tests/openai_codex_token_manager_test.mjs diff --git a/app/L0/_all/mod/_core/openai_codex/token_manager.js b/app/L0/_all/mod/_core/openai_codex/token_manager.js new file mode 100644 index 00000000..9729041e --- /dev/null +++ b/app/L0/_all/mod/_core/openai_codex/token_manager.js @@ -0,0 +1,185 @@ +export const DEFAULT_CODEX_REFRESH_MARGIN_SECONDS = 300; +export const CODEX_AUTH_START_ENDPOINT = "/api/openai_codex_auth_start"; +export const CODEX_AUTH_POLL_ENDPOINT = "/api/openai_codex_auth_poll"; +export const CODEX_TOKEN_REFRESH_ENDPOINT = "/api/openai_codex_token_refresh"; + +const inFlightRefreshes = new Map(); + +function isObject(value) { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +export function normalizeCodexTokens(value) { + if (!isObject(value)) { + return null; + } + + const accessToken = String(value.accessToken || "").trim(); + const refreshToken = String(value.refreshToken || "").trim(); + + if (!accessToken || !refreshToken) { + return null; + } + + const expiresAt = Number.isFinite(value.expiresAt) ? Number(value.expiresAt) : 0; + const obtainedAt = Number.isFinite(value.obtainedAt) ? Number(value.obtainedAt) : 0; + const idToken = String(value.idToken || "").trim(); + const accountId = String(value.accountId || "").trim(); + + return { + accessToken, + accountId, + expiresAt, + idToken, + obtainedAt, + refreshToken + }; +} + +export function isCodexAccessTokenExpiring(tokens, marginSeconds = DEFAULT_CODEX_REFRESH_MARGIN_SECONDS) { + const normalized = normalizeCodexTokens(tokens); + + if (!normalized) { + return true; + } + + const nowSeconds = Math.floor(Date.now() / 1000); + const margin = Number.isFinite(marginSeconds) && marginSeconds >= 0 ? marginSeconds : DEFAULT_CODEX_REFRESH_MARGIN_SECONDS; + + return normalized.expiresAt <= nowSeconds + margin; +} + +async function postJson(url, body, fetchImpl) { + const fetchFn = typeof fetchImpl === "function" ? fetchImpl : globalThis.fetch; + + if (typeof fetchFn !== "function") { + throw new Error("No fetch implementation available."); + } + + const response = await fetchFn(url, { + body: JSON.stringify(body || {}), + credentials: "same-origin", + headers: { "Content-Type": "application/json" }, + method: "POST" + }); + + let payload = null; + const contentType = response.headers?.get?.("content-type") || ""; + + if (contentType.includes("application/json")) { + try { + payload = await response.json(); + } catch { + payload = null; + } + } else { + try { + payload = await response.text(); + } catch { + payload = null; + } + } + + if (!response.ok) { + const message = isObject(payload) && typeof payload.error === "string" ? payload.error : `HTTP ${response.status}`; + const error = new Error(message); + error.statusCode = response.status; + error.payload = payload; + throw error; + } + + return payload; +} + +export async function startCodexDeviceAuthorization({ fetchImpl } = {}) { + return postJson(CODEX_AUTH_START_ENDPOINT, {}, fetchImpl); +} + +export async function pollCodexDeviceAuthorization({ deviceAuthId, userCode, fetchImpl } = {}) { + return postJson( + CODEX_AUTH_POLL_ENDPOINT, + { + deviceAuthId: String(deviceAuthId || "").trim(), + userCode: String(userCode || "").trim() + }, + fetchImpl + ); +} + +async function refreshOnce(refreshToken, fetchImpl) { + const response = await postJson(CODEX_TOKEN_REFRESH_ENDPOINT, { refreshToken }, fetchImpl); + const normalized = normalizeCodexTokens(response); + + if (!normalized) { + throw new Error("Codex refresh response did not include valid tokens."); + } + + return normalized; +} + +async function runSingleFlightRefresh(refreshToken, fetchImpl) { + const existing = inFlightRefreshes.get(refreshToken); + + if (existing) { + return existing; + } + + const pending = (async () => { + try { + return await refreshOnce(refreshToken, fetchImpl); + } finally { + inFlightRefreshes.delete(refreshToken); + } + })(); + + inFlightRefreshes.set(refreshToken, pending); + return pending; +} + +export async function ensureFreshCodexAccessToken({ + fetchImpl, + loadTokens, + marginSeconds = DEFAULT_CODEX_REFRESH_MARGIN_SECONDS, + saveTokens +} = {}) { + if (typeof loadTokens !== "function") { + throw new Error("loadTokens is required."); + } + + // Always re-read persisted tokens instead of trusting an in-memory copy. + // Other tabs or processes may have rotated the refresh token, and the + // single-use rotation rule means a stale in-memory refresh_token will + // fail with invalid_grant and force a full re-login. + const loadedTokens = normalizeCodexTokens(await loadTokens()); + + if (!loadedTokens) { + const error = new Error("Codex tokens are missing. Please log in with ChatGPT."); + error.statusCode = 401; + throw error; + } + + if (!isCodexAccessTokenExpiring(loadedTokens, marginSeconds)) { + return loadedTokens; + } + + const refreshed = await runSingleFlightRefresh(loadedTokens.refreshToken, fetchImpl); + + if (typeof saveTokens === "function") { + try { + await saveTokens(refreshed); + } catch (error) { + // Saving the refreshed tokens back into user config is best-effort from + // this module's perspective; the caller should log the failure. The + // refreshed tokens are still returned so the active request can proceed. + if (typeof globalThis.console?.warn === "function") { + globalThis.console.warn("Failed to persist refreshed Codex tokens:", error); + } + } + } + + return refreshed; +} + +export function resetCodexTokenManagerForTesting() { + inFlightRefreshes.clear(); +} diff --git a/tests/openai_codex_token_manager_test.mjs b/tests/openai_codex_token_manager_test.mjs new file mode 100644 index 00000000..f0b14e95 --- /dev/null +++ b/tests/openai_codex_token_manager_test.mjs @@ -0,0 +1,316 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + DEFAULT_CODEX_REFRESH_MARGIN_SECONDS, + ensureFreshCodexAccessToken, + isCodexAccessTokenExpiring, + normalizeCodexTokens, + resetCodexTokenManagerForTesting +} from "../app/L0/_all/mod/_core/openai_codex/token_manager.js"; + +function createFetchStub(handler) { + return async function stubbedFetch(url, init) { + const callInfo = { + body: init && typeof init.body === "string" ? JSON.parse(init.body) : null, + credentials: init?.credentials, + method: init?.method, + url + }; + const { status = 200, body = {}, contentType = "application/json" } = await handler(callInfo); + const bodyText = contentType.includes("application/json") ? JSON.stringify(body) : String(body); + + return { + headers: { + get: (name) => (String(name).toLowerCase() === "content-type" ? contentType : null) + }, + json: async () => JSON.parse(bodyText), + ok: status >= 200 && status < 300, + status, + text: async () => bodyText + }; + }; +} + +function nowSeconds() { + return Math.floor(Date.now() / 1000); +} + +test("normalizeCodexTokens returns null for missing fields", () => { + assert.equal(normalizeCodexTokens(null), null); + assert.equal(normalizeCodexTokens({}), null); + assert.equal(normalizeCodexTokens({ accessToken: "a" }), null); + assert.equal(normalizeCodexTokens({ refreshToken: "r" }), null); +}); + +test("normalizeCodexTokens fills numeric defaults", () => { + const normalized = normalizeCodexTokens({ + accessToken: "a", + refreshToken: "r" + }); + + assert.equal(normalized.accessToken, "a"); + assert.equal(normalized.refreshToken, "r"); + assert.equal(normalized.expiresAt, 0); + assert.equal(normalized.obtainedAt, 0); + assert.equal(normalized.accountId, ""); + assert.equal(normalized.idToken, ""); +}); + +test("isCodexAccessTokenExpiring returns true when token has no expiry", () => { + assert.equal( + isCodexAccessTokenExpiring({ accessToken: "a", refreshToken: "r" }), + true + ); +}); + +test("isCodexAccessTokenExpiring respects safety margin", () => { + const farFuture = nowSeconds() + DEFAULT_CODEX_REFRESH_MARGIN_SECONDS + 600; + const nearFuture = nowSeconds() + DEFAULT_CODEX_REFRESH_MARGIN_SECONDS - 60; + + assert.equal( + isCodexAccessTokenExpiring({ + accessToken: "a", + expiresAt: farFuture, + refreshToken: "r" + }), + false + ); + + assert.equal( + isCodexAccessTokenExpiring({ + accessToken: "a", + expiresAt: nearFuture, + refreshToken: "r" + }), + true + ); +}); + +test("ensureFreshCodexAccessToken returns existing token when not expiring", async () => { + resetCodexTokenManagerForTesting(); + const tokens = { + accessToken: "live", + accountId: "", + expiresAt: nowSeconds() + 3600, + idToken: "", + obtainedAt: nowSeconds(), + refreshToken: "rt1" + }; + + let loadCalls = 0; + let saveCalls = 0; + const fetchImpl = createFetchStub(() => { + throw new Error("fetch should not be called when token is still valid"); + }); + + const result = await ensureFreshCodexAccessToken({ + fetchImpl, + loadTokens: async () => { + loadCalls += 1; + return tokens; + }, + saveTokens: async () => { + saveCalls += 1; + } + }); + + assert.equal(result.accessToken, "live"); + assert.equal(loadCalls, 1); + assert.equal(saveCalls, 0); +}); + +test("ensureFreshCodexAccessToken refreshes when token is expiring", async () => { + resetCodexTokenManagerForTesting(); + const stale = { + accessToken: "stale", + accountId: "", + expiresAt: nowSeconds() + 30, + idToken: "", + obtainedAt: nowSeconds() - 3600, + refreshToken: "rt-old" + }; + const refreshedPayload = { + accessToken: "fresh", + accountId: "acc-1", + expiresAt: nowSeconds() + 3600, + idToken: "id1", + obtainedAt: nowSeconds(), + refreshToken: "rt-new" + }; + + let postedBody = null; + let savedTokens = null; + + const fetchImpl = createFetchStub((callInfo) => { + assert.equal(callInfo.url, "/api/openai_codex_token_refresh"); + postedBody = callInfo.body; + return { body: refreshedPayload }; + }); + + const result = await ensureFreshCodexAccessToken({ + fetchImpl, + loadTokens: async () => stale, + saveTokens: async (tokens) => { + savedTokens = tokens; + } + }); + + assert.deepEqual(postedBody, { refreshToken: "rt-old" }); + assert.equal(result.accessToken, "fresh"); + assert.equal(result.refreshToken, "rt-new"); + assert.equal(savedTokens?.accessToken, "fresh"); + assert.equal(savedTokens?.refreshToken, "rt-new"); +}); + +test("ensureFreshCodexAccessToken re-reads tokens on every call (no in-memory cache)", async () => { + resetCodexTokenManagerForTesting(); + let loadCalls = 0; + const fetchImpl = createFetchStub(() => { + throw new Error("fetch should not be called when token is still valid"); + }); + + const stillValid = { + accessToken: "a", + accountId: "", + expiresAt: nowSeconds() + 3600, + idToken: "", + obtainedAt: nowSeconds(), + refreshToken: "r" + }; + + const loadTokens = async () => { + loadCalls += 1; + return stillValid; + }; + + await ensureFreshCodexAccessToken({ fetchImpl, loadTokens }); + await ensureFreshCodexAccessToken({ fetchImpl, loadTokens }); + await ensureFreshCodexAccessToken({ fetchImpl, loadTokens }); + + assert.equal(loadCalls, 3); +}); + +test("ensureFreshCodexAccessToken coalesces concurrent refresh calls for the same refresh token", async () => { + resetCodexTokenManagerForTesting(); + const stale = { + accessToken: "stale", + accountId: "", + expiresAt: nowSeconds() - 60, + idToken: "", + obtainedAt: nowSeconds() - 3600, + refreshToken: "rt-shared" + }; + + let refreshCallCount = 0; + let resolveRefresh; + const refreshPayload = { + accessToken: "fresh", + accountId: "", + expiresAt: nowSeconds() + 3600, + idToken: "", + obtainedAt: nowSeconds(), + refreshToken: "rt-new" + }; + + const fetchImpl = createFetchStub(async () => { + refreshCallCount += 1; + await new Promise((resolve) => { + resolveRefresh = resolve; + }); + return { body: refreshPayload }; + }); + + const loadTokens = async () => stale; + const promiseA = ensureFreshCodexAccessToken({ fetchImpl, loadTokens }); + const promiseB = ensureFreshCodexAccessToken({ fetchImpl, loadTokens }); + const promiseC = ensureFreshCodexAccessToken({ fetchImpl, loadTokens }); + + // Let microtasks run so all three calls reach the single-flight coalesce path. + await Promise.resolve(); + await Promise.resolve(); + + resolveRefresh({ body: refreshPayload }); + + const [a, b, c] = await Promise.all([promiseA, promiseB, promiseC]); + + assert.equal(refreshCallCount, 1); + assert.equal(a.accessToken, "fresh"); + assert.equal(b.accessToken, "fresh"); + assert.equal(c.accessToken, "fresh"); +}); + +test("ensureFreshCodexAccessToken throws when no tokens are stored", async () => { + resetCodexTokenManagerForTesting(); + await assert.rejects( + () => + ensureFreshCodexAccessToken({ + fetchImpl: createFetchStub(() => { + throw new Error("should not be called"); + }), + loadTokens: async () => null + }), + /Codex tokens are missing/u + ); +}); + +test("ensureFreshCodexAccessToken propagates invalid_grant from refresh endpoint", async () => { + resetCodexTokenManagerForTesting(); + const stale = { + accessToken: "stale", + expiresAt: nowSeconds() - 60, + obtainedAt: nowSeconds() - 3600, + refreshToken: "rt-revoked" + }; + + const fetchImpl = createFetchStub(() => ({ + body: { error: "Refresh token is no longer valid. Please log in again." }, + status: 401 + })); + + await assert.rejects( + () => + ensureFreshCodexAccessToken({ + fetchImpl, + loadTokens: async () => stale + }), + (error) => error.statusCode === 401 && /Refresh token is no longer valid/u.test(error.message) + ); +}); + +test("ensureFreshCodexAccessToken returns refreshed tokens even when save fails", async () => { + resetCodexTokenManagerForTesting(); + const stale = { + accessToken: "stale", + expiresAt: nowSeconds() - 60, + obtainedAt: nowSeconds() - 3600, + refreshToken: "rt1" + }; + const refreshed = { + accessToken: "fresh", + expiresAt: nowSeconds() + 3600, + obtainedAt: nowSeconds(), + refreshToken: "rt2" + }; + + const fetchImpl = createFetchStub(() => ({ body: refreshed })); + + // Silence expected console.warn during this test so it does not pollute + // the overall test output. + const originalWarn = globalThis.console.warn; + globalThis.console.warn = () => {}; + + try { + const result = await ensureFreshCodexAccessToken({ + fetchImpl, + loadTokens: async () => stale, + saveTokens: async () => { + throw new Error("disk error"); + } + }); + + assert.equal(result.accessToken, "fresh"); + } finally { + globalThis.console.warn = originalWarn; + } +}); From ddcebff9f18f13c1bef49e2befc5693d1d059e25 Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Thu, 23 Apr 2026 09:03:48 +0200 Subject: [PATCH 06/29] feat(openai-codex): model catalog and device-code login flow Adds the shipped Codex model catalog used by the settings UI and the stateful controller that drives the device-code login UX against the three OAuth backend endpoints. Also extends the overlay config enum with the `openai-codex` provider variant so upcoming UI wiring has a third tab to bind to. - `models.js` ships the 6 Codex models observable from ChatGPT Plus subscriptions and exports `CODEX_DEFAULT_MODEL_ID` (`gpt-5.4-mini`) as the recommended default: cheapest, fastest, and the most quota-friendly option. Live discovery via `GET /backend-api/codex/models?client_version=1.0.0` is left as follow-up rather than MVP because it would add a second network call to the login UX. - `auth_flow.js` emits a small finite state machine (`STARTING` -> `PENDING` -> `COMPLETE` | `FAILED`) so the settings UI can display the verification URL and user code while polling runs, and aborts cleanly via `AbortSignal` when the user cancels. It uses the poll interval returned by the OAuth server but enforces a 3-second floor so we never hammer the endpoint if the server sends something unusable. - `onscreen_agent/config.js` gains the `openai-codex` provider enum value plus new settings fields (`codexModel`, `codexTokens`), and `normalizeOnscreenAgentLlmProvider` now recognizes the third variant. UI and transport wiring land in the next two commits. --- .../_all/mod/_core/onscreen_agent/config.js | 16 +- app/L0/_all/mod/_core/openai_codex/AGENTS.md | 3 + .../_all/mod/_core/openai_codex/auth_flow.js | 142 ++++++++++++++++++ app/L0/_all/mod/_core/openai_codex/models.js | 44 ++++++ tests/openai_codex_models_test.mjs | 47 ++++++ 5 files changed, 249 insertions(+), 3 deletions(-) create mode 100644 app/L0/_all/mod/_core/openai_codex/auth_flow.js create mode 100644 app/L0/_all/mod/_core/openai_codex/models.js create mode 100644 tests/openai_codex_models_test.mjs diff --git a/app/L0/_all/mod/_core/onscreen_agent/config.js b/app/L0/_all/mod/_core/onscreen_agent/config.js index bcca393a..72fd2885 100644 --- a/app/L0/_all/mod/_core/onscreen_agent/config.js +++ b/app/L0/_all/mod/_core/onscreen_agent/config.js @@ -1,4 +1,5 @@ import { DEFAULT_PROMPT_BUDGET_RATIOS, normalizePromptBudgetRatios } from "/mod/_core/agent_prompt/prompt-items.js"; +import { CODEX_DEFAULT_MODEL_ID } from "/mod/_core/openai_codex/models.js"; export const ONSCREEN_AGENT_CONFIG_PATH = "~/conf/onscreen-agent.yaml"; export const ONSCREEN_AGENT_HISTORY_PATH = "~/hist/onscreen-agent.json"; @@ -6,6 +7,7 @@ export const ONSCREEN_AGENT_UI_STATE_STORAGE_KEY = "space.onscreenAgent.uiState" export const DEFAULT_ONSCREEN_AGENT_MAX_TOKENS = 120_000; export const ONSCREEN_AGENT_LLM_PROVIDER = Object.freeze({ API: "api", + CODEX: "openai-codex", LOCAL: "local" }); export const ONSCREEN_AGENT_LOCAL_PROVIDER = Object.freeze({ @@ -21,6 +23,8 @@ export const ONSCREEN_AGENT_HIDDEN_EDGE = Object.freeze({ export const DEFAULT_ONSCREEN_AGENT_SETTINGS = { apiEndpoint: "https://openrouter.ai/api/v1/chat/completions", apiKey: "", + codexModel: CODEX_DEFAULT_MODEL_ID, + codexTokens: "", huggingfaceDtype: "q4", huggingfaceModel: "", localProvider: ONSCREEN_AGENT_LOCAL_PROVIDER.HUGGINGFACE, @@ -36,9 +40,15 @@ function normalizeOnscreenAgentSettingText(value) { } export function normalizeOnscreenAgentLlmProvider(value) { - return value === ONSCREEN_AGENT_LLM_PROVIDER.LOCAL - ? ONSCREEN_AGENT_LLM_PROVIDER.LOCAL - : ONSCREEN_AGENT_LLM_PROVIDER.API; + if (value === ONSCREEN_AGENT_LLM_PROVIDER.LOCAL) { + return ONSCREEN_AGENT_LLM_PROVIDER.LOCAL; + } + + if (value === ONSCREEN_AGENT_LLM_PROVIDER.CODEX) { + return ONSCREEN_AGENT_LLM_PROVIDER.CODEX; + } + + return ONSCREEN_AGENT_LLM_PROVIDER.API; } export function normalizeOnscreenAgentLocalProvider(value) { diff --git a/app/L0/_all/mod/_core/openai_codex/AGENTS.md b/app/L0/_all/mod/_core/openai_codex/AGENTS.md index d6c88b51..337b8d23 100644 --- a/app/L0/_all/mod/_core/openai_codex/AGENTS.md +++ b/app/L0/_all/mod/_core/openai_codex/AGENTS.md @@ -17,6 +17,9 @@ This module owns: - `ext/js/_core/admin/views/agent/api.js/prepareAdminAgentApiRequest/end/openai-codex.js`: admin-chat API request customization (body shape + headers) - `request_shape.js`: pure stateless converters between OpenAI Chat-Completions request bodies and Codex Responses-API request bodies - `sse_adapter.js`: pure stateless mapper from Codex Responses-API SSE events into the existing Chat-Completions-shaped delta frames that `_core/onscreen_agent/api.js` and `_core/admin/views/agent/api.js` already parse +- `token_manager.js`: browser-side helper that wraps the three OAuth backend endpoints (`/api/openai_codex_auth_start`, `/api/openai_codex_auth_poll`, `/api/openai_codex_token_refresh`) and enforces always-fresh-read refresh semantics with a single-flight coalescer per refresh token +- `auth_flow.js`: stateful controller that drives the end-to-end device-code login UX from the settings dialog, emitting `STARTING` / `PENDING` / `COMPLETE` / `FAILED` status events and polling the backend at the interval the OAuth server returns +- `models.js`: shipped Codex model catalog used by the settings UI, with `CODEX_DEFAULT_MODEL_ID` pointing at the cheapest and fastest option suitable for a ChatGPT Plus subscription ## Local Contracts diff --git a/app/L0/_all/mod/_core/openai_codex/auth_flow.js b/app/L0/_all/mod/_core/openai_codex/auth_flow.js new file mode 100644 index 00000000..ff80ce17 --- /dev/null +++ b/app/L0/_all/mod/_core/openai_codex/auth_flow.js @@ -0,0 +1,142 @@ +import { + pollCodexDeviceAuthorization, + startCodexDeviceAuthorization, + normalizeCodexTokens +} from "/mod/_core/openai_codex/token_manager.js"; + +export const CODEX_AUTH_FLOW_STATUS = Object.freeze({ + COMPLETE: "complete", + FAILED: "failed", + IDLE: "idle", + PENDING: "pending", + STARTING: "starting" +}); + +const DEFAULT_POLL_INTERVAL_SECONDS = 3; +const MIN_POLL_INTERVAL_SECONDS = 3; +const DEFAULT_FLOW_TIMEOUT_SECONDS = 15 * 60; + +function wait(ms, signal) { + return new Promise((resolve, reject) => { + if (signal?.aborted) { + reject(new DOMException("Device authorization cancelled.", "AbortError")); + return; + } + + const timer = setTimeout(() => { + cleanup(); + resolve(); + }, ms); + + const onAbort = () => { + clearTimeout(timer); + cleanup(); + reject(new DOMException("Device authorization cancelled.", "AbortError")); + }; + + function cleanup() { + signal?.removeEventListener?.("abort", onAbort); + } + + signal?.addEventListener?.("abort", onAbort, { once: true }); + }); +} + +export async function runCodexDeviceAuthorizationFlow({ + fetchImpl, + onStatusChange, + signal +} = {}) { + const emit = typeof onStatusChange === "function" ? onStatusChange : () => {}; + + emit({ status: CODEX_AUTH_FLOW_STATUS.STARTING }); + + let deviceAuth; + try { + deviceAuth = await startCodexDeviceAuthorization({ fetchImpl }); + } catch (error) { + emit({ + error: error instanceof Error ? error.message : String(error), + status: CODEX_AUTH_FLOW_STATUS.FAILED + }); + throw error; + } + + const deviceAuthId = String(deviceAuth?.deviceAuthId || "").trim(); + const userCode = String(deviceAuth?.userCode || "").trim(); + const verificationUrl = String(deviceAuth?.verificationUrl || "").trim(); + const serverInterval = Number.isFinite(deviceAuth?.interval) ? Number(deviceAuth.interval) : DEFAULT_POLL_INTERVAL_SECONDS; + const pollIntervalSeconds = Math.max(MIN_POLL_INTERVAL_SECONDS, serverInterval); + const timeoutSeconds = Number.isFinite(deviceAuth?.expiresIn) ? Number(deviceAuth.expiresIn) : DEFAULT_FLOW_TIMEOUT_SECONDS; + + if (!deviceAuthId || !userCode || !verificationUrl) { + const error = new Error("ChatGPT returned an invalid device authorization payload."); + emit({ + error: error.message, + status: CODEX_AUTH_FLOW_STATUS.FAILED + }); + throw error; + } + + emit({ + deviceAuthId, + pollIntervalSeconds, + status: CODEX_AUTH_FLOW_STATUS.PENDING, + userCode, + verificationUrl + }); + + const startedAt = Date.now(); + + while (true) { + if (signal?.aborted) { + const error = new DOMException("Device authorization cancelled.", "AbortError"); + emit({ error: error.message, status: CODEX_AUTH_FLOW_STATUS.FAILED }); + throw error; + } + + if ((Date.now() - startedAt) / 1000 > timeoutSeconds) { + const error = new Error("Timed out waiting for ChatGPT authorization."); + emit({ error: error.message, status: CODEX_AUTH_FLOW_STATUS.FAILED }); + throw error; + } + + await wait(pollIntervalSeconds * 1000, signal); + + let pollResult; + try { + pollResult = await pollCodexDeviceAuthorization({ + deviceAuthId, + fetchImpl, + userCode + }); + } catch (error) { + emit({ + error: error instanceof Error ? error.message : String(error), + status: CODEX_AUTH_FLOW_STATUS.FAILED + }); + throw error; + } + + if (pollResult?.status === "complete" && pollResult.tokens) { + const tokens = normalizeCodexTokens(pollResult.tokens); + + if (!tokens) { + const error = new Error("ChatGPT returned an invalid token payload."); + emit({ error: error.message, status: CODEX_AUTH_FLOW_STATUS.FAILED }); + throw error; + } + + emit({ status: CODEX_AUTH_FLOW_STATUS.COMPLETE, tokens }); + return tokens; + } + + emit({ + deviceAuthId, + pollIntervalSeconds, + status: CODEX_AUTH_FLOW_STATUS.PENDING, + userCode, + verificationUrl + }); + } +} diff --git a/app/L0/_all/mod/_core/openai_codex/models.js b/app/L0/_all/mod/_core/openai_codex/models.js new file mode 100644 index 00000000..df95e35b --- /dev/null +++ b/app/L0/_all/mod/_core/openai_codex/models.js @@ -0,0 +1,44 @@ +export const CODEX_DEFAULT_MODEL_ID = "gpt-5.4-mini"; + +export const CODEX_MODEL_CATALOG = Object.freeze([ + { + description: "Cheapest and fastest. Recommended default for ChatGPT Plus.", + id: "gpt-5.4-mini" + }, + { + description: "Flagship general-purpose model.", + id: "gpt-5.4" + }, + { + description: "Code-optimized, latest codex variant.", + id: "gpt-5.3-codex" + }, + { + description: "Code-optimized, previous generation.", + id: "gpt-5.2-codex" + }, + { + description: "Maximum code-optimized performance.", + id: "gpt-5.1-codex-max" + }, + { + description: "Smallest code-optimized variant.", + id: "gpt-5.1-codex-mini" + } +]); + +export function normalizeCodexModelId(value) { + const normalized = String(value || "").trim(); + + if (!normalized) { + return CODEX_DEFAULT_MODEL_ID; + } + + return normalized; +} + +export function isKnownCodexModelId(value) { + const normalized = String(value || "").trim(); + + return CODEX_MODEL_CATALOG.some((entry) => entry.id === normalized); +} diff --git a/tests/openai_codex_models_test.mjs b/tests/openai_codex_models_test.mjs new file mode 100644 index 00000000..68213fc5 --- /dev/null +++ b/tests/openai_codex_models_test.mjs @@ -0,0 +1,47 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + CODEX_DEFAULT_MODEL_ID, + CODEX_MODEL_CATALOG, + isKnownCodexModelId, + normalizeCodexModelId +} from "../app/L0/_all/mod/_core/openai_codex/models.js"; + +test("CODEX_MODEL_CATALOG exposes the documented 6 entries", () => { + assert.equal(CODEX_MODEL_CATALOG.length, 6); + for (const entry of CODEX_MODEL_CATALOG) { + assert.ok(typeof entry.id === "string" && entry.id.startsWith("gpt-")); + assert.ok(typeof entry.description === "string" && entry.description.length > 0); + } +}); + +test("CODEX_DEFAULT_MODEL_ID appears in the catalog", () => { + assert.ok(isKnownCodexModelId(CODEX_DEFAULT_MODEL_ID)); +}); + +test("normalizeCodexModelId falls back to the default for empty input", () => { + assert.equal(normalizeCodexModelId(""), CODEX_DEFAULT_MODEL_ID); + assert.equal(normalizeCodexModelId(null), CODEX_DEFAULT_MODEL_ID); + assert.equal(normalizeCodexModelId(undefined), CODEX_DEFAULT_MODEL_ID); + assert.equal(normalizeCodexModelId(" "), CODEX_DEFAULT_MODEL_ID); +}); + +test("normalizeCodexModelId preserves any trimmed non-empty string", () => { + assert.equal(normalizeCodexModelId(" gpt-5.4 "), "gpt-5.4"); + assert.equal(normalizeCodexModelId("future-model"), "future-model"); +}); + +test("isKnownCodexModelId returns true only for catalog entries", () => { + assert.equal(isKnownCodexModelId("gpt-5.4"), true); + assert.equal(isKnownCodexModelId("gpt-5.4-mini"), true); + assert.equal(isKnownCodexModelId("gpt-5.3-codex"), true); + assert.equal(isKnownCodexModelId("gpt-does-not-exist"), false); + assert.equal(isKnownCodexModelId(""), false); +}); + +test("CODEX_MODEL_CATALOG is frozen and cannot be mutated", () => { + assert.throws(() => { + CODEX_MODEL_CATALOG.push({ description: "x", id: "x" }); + }, TypeError); +}); From 757d81120e3507dc354323c06464217fab78bf36 Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Thu, 23 Apr 2026 09:08:34 +0200 Subject: [PATCH 07/29] feat(openai-codex): wire provider into onscreen agent client Integrates the `openai-codex` provider into the overlay chat surface: a new LLM-client subclass with a Codex-aware SSE reader, the request- mutation hook that swaps Chat-Completions shape for Responses-API shape and adds the Cloudflare-required headers, and the settings UI that drives the device-code login. Transport: - `OnscreenAgentCodexLlmClient` extends the shared base client and uses a dedicated `readCodexStreamingResponse` that feeds raw SSE event blocks through `mapCodexEventToChatFrames` so the upstream per-delta callback contract stays identical for all providers - the OpenRouter-style Chat-Completions SSE reader is left untouched; Codex is an additive subclass rather than a core refactor - `createOnscreenAgentLlmClient` now dispatches on three provider variants (API / CODEX / LOCAL) Request hook (`ext/js/_core/onscreen_agent/api.js/prepareOnscreen AgentApiRequest/end/openai-codex.js`): - detects Codex-provider settings and rewrites the prepared request in place: `requestUrl` becomes the Codex `/responses` endpoint, `requestBody` goes through `chatToResponsesRequest`, headers add the Cloudflare originator plus extracted `ChatGPT-Account-ID` - ensures a fresh access token through the always-fresh-read `ensureFreshCodexAccessToken` helper, loading persisted tokens from the `userCrypto:`-encrypted `codex_tokens` entry in `~/conf/onscreen-agent.yaml` and saving refreshed tokens back into the same file so other tabs pick up the rotation Settings UI: - third segmented-control tab labeled `ChatGPT` next to the existing `API` and `Local` tabs - three UI states: logged-out (explainer + Sign in button), login-pending (verification URL + user code + Cancel), and signed- in (account summary + model dropdown + Sign out) - login flow runs through `runCodexDeviceAuthorizationFlow` with an AbortController so Cancel stops polling immediately - model dropdown is populated from `CODEX_MODEL_CATALOG` and defaults to `gpt-5.4-mini` Docs: - onscreen_agent AGENTS.md now documents the three-tab contract and references the Codex flow + token-storage rule --- .../_all/mod/_core/onscreen_agent/AGENTS.md | 3 +- app/L0/_all/mod/_core/onscreen_agent/api.js | 169 ++++++++++++++++++ .../_all/mod/_core/onscreen_agent/panel.html | 54 ++++++ app/L0/_all/mod/_core/onscreen_agent/store.js | 147 +++++++++++++++ .../end/openai-codex.js | 146 +++++++++++++++ 5 files changed, 518 insertions(+), 1 deletion(-) create mode 100644 app/L0/_all/mod/_core/openai_codex/ext/js/_core/onscreen_agent/api.js/prepareOnscreenAgentApiRequest/end/openai-codex.js diff --git a/app/L0/_all/mod/_core/onscreen_agent/AGENTS.md b/app/L0/_all/mod/_core/onscreen_agent/AGENTS.md index 91e3cbe9..6d2073f9 100644 --- a/app/L0/_all/mod/_core/onscreen_agent/AGENTS.md +++ b/app/L0/_all/mod/_core/onscreen_agent/AGENTS.md @@ -279,7 +279,8 @@ Current overlay behavior: - `store.js` should hold one prompt-instance object per chat surface, rebuild full prompt input when the overlay boots or the thread is reset or a new LLM turn is about to start, and reuse the cached system or examples or transient sections without recomputing the full prepared payload or token counts on every streamed delta - prompt-history previews and token counts are derived from the prepared outbound request payload so request-prep extensions stay visible in the context window instead of only affecting the final fetch call, but exact recomputation should happen only at stable boundaries such as request preparation, stop handling, or settled assistant completion, never on every streamed delta - the settings modal and prompt-history modal must both use the shared fixed-chrome dialog shell from `_core/visual/forms/dialog.css` so their header and footer rows stay static while only the settings form body or prompt-history frame scrolls -- the settings modal keeps a provider switch with exactly two tabs named `API` and `Local`; API settings show endpoint, model, and API key fields, while local settings mount the shared `_core/huggingface/config-sidebar.html` component in `onscreen` mode +- the settings modal keeps a provider switch with three tabs named `API`, `ChatGPT`, and `Local`; API settings show endpoint, model, and API key fields, `ChatGPT` settings own the OpenAI Codex OAuth device-code login plus a model dropdown sourced from `/mod/_core/openai_codex/models.js`, and local settings mount the shared `_core/huggingface/config-sidebar.html` component in `onscreen` mode +- the `ChatGPT` tab drives the login UX through `_core/openai_codex/auth_flow.js` and stores refreshed tokens in the overlay config under the same `userCrypto:`-prefixed encryption rule as `api_key` - local provider settings are limited to the shared Hugging Face browser runtime for now; the overlay subscribes to `_core/huggingface/manager.js`, reads the same saved-model list and live worker state as the routed Local LLM page, and should not boot the worker just because the modal opened - when no Hugging Face model is selected and the shared saved-model list has entries, the overlay local-provider panel should preselect the browser-wide last successfully loaded saved model from `_core/huggingface/manager.js`, falling back to the first saved entry if that last-used entry was discarded - when no Hugging Face model is selected, no model is loaded, and the shared saved-model list is empty, the overlay local-provider panel should prefill the model field with the same default used by the routed testing page: `onnx-community/gemma-4-E4B-it-ONNX` diff --git a/app/L0/_all/mod/_core/onscreen_agent/api.js b/app/L0/_all/mod/_core/onscreen_agent/api.js index 510d23a6..0ca34229 100644 --- a/app/L0/_all/mod/_core/onscreen_agent/api.js +++ b/app/L0/_all/mod/_core/onscreen_agent/api.js @@ -2,6 +2,12 @@ import * as config from "/mod/_core/onscreen_agent/config.js"; import * as llmParams from "/mod/_core/onscreen_agent/llm-params.js"; import { prepareOnscreenAgentCompletionRequest } from "/mod/_core/onscreen_agent/llm.js"; import { getHuggingFaceManager } from "/mod/_core/huggingface/manager.js"; +import { CODEX_RESPONSES_ENDPOINT, applyCodexHeaders } from "/mod/_core/openai_codex/request.js"; +import { chatToResponsesRequest } from "/mod/_core/openai_codex/request_shape.js"; +import { + CODEX_STREAM_DONE_MARKER, + mapCodexEventToChatFrames +} from "/mod/_core/openai_codex/sse_adapter.js"; function extractTextContent(value) { if (typeof value === "string") { @@ -186,6 +192,87 @@ async function readStreamingResponse(response, onDelta) { } } +function parseCodexEventBlock(eventBlock, onDelta, meta) { + const lines = eventBlock.split(/\r?\n/u); + + for (const line of lines) { + if (!line.startsWith("data:")) { + continue; + } + + const value = line.slice(5).trim(); + + if (!value) { + continue; + } + + let event; + try { + event = JSON.parse(value); + } catch { + continue; + } + + // `mapCodexEventToChatFrames` throws on response.failed / error events, + // which bubbles up to the caller as a request failure. That is the + // intended behavior for terminal upstream errors mid-stream. + const frames = mapCodexEventToChatFrames(event); + + for (const frame of frames) { + if (frame === CODEX_STREAM_DONE_MARKER) { + meta.sawDoneMarker = true; + return true; + } + + const delta = extractStreamingDelta(frame); + noteCompletionPayload(meta, frame, delta); + + if (delta) { + onDelta(delta); + } + } + } + + return false; +} + +async function readCodexStreamingResponse(response, onDelta) { + const meta = createCompletionResponseMeta("stream"); + const reader = response.body.getReader(); + const decoder = new TextDecoder(); + let buffer = ""; + + while (true) { + const { done, value } = await reader.read(); + buffer += decoder.decode(value || new Uint8Array(), { + stream: !done + }); + + let boundary = buffer.indexOf("\n\n"); + + while (boundary !== -1) { + const eventBlock = buffer.slice(0, boundary).trim(); + buffer = buffer.slice(boundary + 2); + + if (eventBlock && parseCodexEventBlock(eventBlock, onDelta, meta)) { + return finalizeCompletionResponseMeta(meta); + } + + boundary = buffer.indexOf("\n\n"); + } + + if (done) { + const remaining = buffer.trim(); + + if (remaining) { + parseCodexEventBlock(remaining, onDelta, meta); + } + + return finalizeCompletionResponseMeta(meta); + } + } +} + function normalizeCompletionMessagesForLocal(messages) { return (Array.isArray(messages) ? messages : []) .map((message) => { @@ -357,6 +444,82 @@ export class OnscreenAgentApiLlmClient extends OnscreenAgentLlmClient { } } +export class OnscreenAgentCodexLlmClient extends OnscreenAgentLlmClient { + validateSettings(settings = this.settings) { + if (!settings?.model?.trim()) { + throw new Error("Choose a Codex model before sending a message."); + } + + const tokens = parseCodexTokensFromSettings(settings); + + if (!tokens?.accessToken) { + const error = new Error("Sign in with ChatGPT before sending a message."); + error.requiresCodexLogin = true; + throw error; + } + } + + async resolveApiRequest(options = {}) { + const preparedRequest = await this.resolvePreparedRequest(options); + const effectiveSettings = + preparedRequest?.settings && typeof preparedRequest.settings === "object" + ? preparedRequest.settings + : this.settings; + + return prepareOnscreenAgentApiRequest({ + preparedRequest, + settings: effectiveSettings + }); + } + + async streamCompletion(options = {}) { + const onDelta = typeof options.onDelta === "function" ? options.onDelta : () => {}; + const effectiveRequest = await this.resolveApiRequest(options); + const effectiveSettings = + effectiveRequest?.settings && typeof effectiveRequest.settings === "object" + ? effectiveRequest.settings + : this.settings; + + this.validateSettings(effectiveSettings); + + const response = await fetch(effectiveRequest.requestUrl, { + ...buildFetchRequestInit(effectiveRequest, options.signal) + }); + + if (!response.ok) { + await throwResponseError(response); + } + + if (!response.body) { + throw new Error("Streaming response body is not available."); + } + + return readCodexStreamingResponse(response, onDelta); + } +} + +function parseCodexTokensFromSettings(settings = {}) { + const raw = settings?.codexTokens; + + if (!raw) { + return null; + } + + if (typeof raw === "object") { + return raw; + } + + if (typeof raw !== "string") { + return null; + } + + try { + return JSON.parse(raw); + } catch { + return null; + } +} + export class OnscreenAgentLocalLlmClient extends OnscreenAgentLlmClient { validateSettings(settings = this.settings) { const selection = config.getOnscreenAgentLocalModelSelection(settings); @@ -450,6 +613,12 @@ export function createOnscreenAgentLlmClient(settings = config.DEFAULT_ONSCREEN_ }); } + if (provider === config.ONSCREEN_AGENT_LLM_PROVIDER.CODEX) { + return new OnscreenAgentCodexLlmClient({ + settings + }); + } + return new OnscreenAgentApiLlmClient({ settings }); diff --git a/app/L0/_all/mod/_core/onscreen_agent/panel.html b/app/L0/_all/mod/_core/onscreen_agent/panel.html index a4510bdc..8a3990c1 100644 --- a/app/L0/_all/mod/_core/onscreen_agent/panel.html +++ b/app/L0/_all/mod/_core/onscreen_agent/panel.html @@ -338,6 +338,14 @@

Model, credentials, params, and instructions

> API + + From 74824c191465368b31fc2d387e7fa60227b8c00e Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Thu, 23 Apr 2026 12:00:55 +0200 Subject: [PATCH 26/29] fix(openai-codex): poll immediately on first iteration of device flow The poll loop previously awaited `pollIntervalSeconds * 1000` at the top of every iteration, including the first. A user who entered the code into the ChatGPT browser before the space-agent UI had painted the pending panel still waited the full interval (default 5 s) for the first server poll. Skip the wait on the first iteration and keep it on every subsequent iteration so fast humans see completion near- instantly without changing the poll cadence for the common case. --- app/L0/_all/mod/_core/openai_codex/auth_flow.js | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/app/L0/_all/mod/_core/openai_codex/auth_flow.js b/app/L0/_all/mod/_core/openai_codex/auth_flow.js index 84f061b6..5765130c 100644 --- a/app/L0/_all/mod/_core/openai_codex/auth_flow.js +++ b/app/L0/_all/mod/_core/openai_codex/auth_flow.js @@ -87,6 +87,7 @@ export async function runCodexDeviceAuthorizationFlow({ }); const startedAt = Date.now(); + let firstIteration = true; while (true) { if (signal?.aborted) { @@ -101,7 +102,15 @@ export async function runCodexDeviceAuthorizationFlow({ throw error; } - await wait(pollIntervalSeconds * 1000, signal); + // Poll immediately on the first iteration so a user who enters the code + // before the browser renders the pending panel still sees the login + // complete near-instantly. Subsequent iterations wait for the server- + // advertised poll interval to avoid hammering the endpoint. + if (!firstIteration) { + await wait(pollIntervalSeconds * 1000, signal); + } + + firstIteration = false; let pollResult; try { From 05f3b135a43d4a042a63b3476417a1c5fe391e8a Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Thu, 23 Apr 2026 12:01:04 +0200 Subject: [PATCH 27/29] fix(openai-codex): explicit codex_tokens deletion on sign-out Both storage layers built the YAML payload with a truthy-guarded `codex_tokens` assignment: when the user signed out the key was simply omitted from the new payload. Because `fileWrite` replaces the file contents outright this was sufficient today, but it leaves the guarantee implicit; a future change that starts merging payloads, or an accidental in-place patch helper, could leave the previous ciphertext in place after sign-out. Make the clearing explicit by `delete`-ing the key in the sign-out branch so the contract "empty in memory means empty on disk" is visible to future maintainers. --- app/L0/_all/mod/_core/admin/views/agent/storage.js | 6 ++++++ app/L0/_all/mod/_core/onscreen_agent/storage.js | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/app/L0/_all/mod/_core/admin/views/agent/storage.js b/app/L0/_all/mod/_core/admin/views/agent/storage.js index e71d3f2d..55525414 100644 --- a/app/L0/_all/mod/_core/admin/views/agent/storage.js +++ b/app/L0/_all/mod/_core/admin/views/agent/storage.js @@ -203,8 +203,14 @@ async function buildStoredConfigPayload(runtime, { settings, systemPrompt }) { } }; + // When the user signs out `encodedCodexTokens` is an empty string; make the + // absence explicit in the YAML payload so sign-out deterministically clears + // the stored entry on disk instead of relying on the full-rewrite behaviour + // of `fileWrite` alone. if (encodedCodexTokens) { payload.codex_tokens = encodedCodexTokens; + } else { + delete payload.codex_tokens; } if (normalizedSystemPrompt) { diff --git a/app/L0/_all/mod/_core/onscreen_agent/storage.js b/app/L0/_all/mod/_core/onscreen_agent/storage.js index f642cb36..c47e97fd 100644 --- a/app/L0/_all/mod/_core/onscreen_agent/storage.js +++ b/app/L0/_all/mod/_core/onscreen_agent/storage.js @@ -308,8 +308,14 @@ async function buildStoredConfigPayload(runtime, { settings, systemPrompt }) { } }; + // When the user signs out `encodedCodexTokens` is an empty string; make the + // absence explicit in the YAML payload so sign-out deterministically clears + // the stored entry on disk instead of relying on the full-rewrite behaviour + // of `fileWrite` alone. if (encodedCodexTokens) { payload.codex_tokens = encodedCodexTokens; + } else { + delete payload.codex_tokens; } if (normalizedSystemPrompt) { From bba17097de8650624b0c9fc99e6b135d609cdd89 Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Mon, 27 Apr 2026 05:11:12 +0200 Subject: [PATCH 28/29] fix(openai-codex): route chat-completion request through space.proxy The standard prepareOnscreenAgentCompletionRequest / prepareAdminAgentApiRequest flow already routes its API URL through space.proxy.buildUrl(...) for proxyable external endpoints. The Codex hooks override `requestUrl` with the bare CODEX_RESPONSES_ENDPOINT and therefore re-introduce the proxy bypass: every Codex chat call paid a failed-direct-fetch roundtrip rescued by installFetchProxy(...)'s fallback retry, and emitted a red `Access-Control-Allow-Origin` CORS error in the DevTools console on the first call of every page load. Mirror the pattern already used by models_discovery.js: route CODEX_RESPONSES_ENDPOINT through space.proxy.buildUrl(...) explicitly and set requestInit.credentials = "same-origin" so the proxy endpoint receives the browser session cookie. The proxy strips the cookie header before forwarding upstream, so this does not leak space_session to chatgpt.com. The existing applyCodexHeaders() Cloudflare originator plus User-Agent block continues to run with the proxied URL. Document the proxy routing in openai_codex/AGENTS.md alongside the existing Cloudflare-header anti-refactor warning so a future cleanup pass does not silently revert the routing. --- app/L0/_all/mod/_core/openai_codex/AGENTS.md | 1 + .../end/openai-codex.js | 37 ++++++++++++++++++- .../end/openai-codex.js | 37 ++++++++++++++++++- 3 files changed, 71 insertions(+), 4 deletions(-) diff --git a/app/L0/_all/mod/_core/openai_codex/AGENTS.md b/app/L0/_all/mod/_core/openai_codex/AGENTS.md index 5b64a3de..342ec8d0 100644 --- a/app/L0/_all/mod/_core/openai_codex/AGENTS.md +++ b/app/L0/_all/mod/_core/openai_codex/AGENTS.md @@ -140,4 +140,5 @@ What requires a ChatGPT Plus subscription to verify: - prefer one shared helper for endpoint matching, header mutation, and body-shape conversion so the admin and onscreen hooks stay in sync - if additional Codex request shaping is needed later, extend the prepared request object here instead of reintroducing per-surface hard-coded branches - keep the Cloudflare header block in `request.js` even if it looks like boilerplate; removing it breaks the endpoint with a confusing 403 +- keep the `space.proxy.buildUrl(CODEX_RESPONSES_ENDPOINT)` routing plus `requestInit.credentials = "same-origin"` in both chat-completion hooks (onscreen and admin) — `chatgpt.com` advertises no `Access-Control-Allow-Origin` header so a direct browser fetch is always blocked. The `installFetchProxy(...)` wrapper provides a transparent fallback retry, but every page-load pays one failed-direct-fetch roundtrip and emits a red CORS error in the DevTools console until the wrapper caches the origin. Routing through the proxy explicitly eliminates both costs. - update this file when the Codex endpoint adds new SSE event families, when Codex rejects another request-body field, or when OAuth URLs change diff --git a/app/L0/_all/mod/_core/openai_codex/ext/js/_core/admin/views/agent/api.js/prepareAdminAgentApiRequest/end/openai-codex.js b/app/L0/_all/mod/_core/openai_codex/ext/js/_core/admin/views/agent/api.js/prepareAdminAgentApiRequest/end/openai-codex.js index 10e4ffee..1a54383b 100644 --- a/app/L0/_all/mod/_core/openai_codex/ext/js/_core/admin/views/agent/api.js/prepareAdminAgentApiRequest/end/openai-codex.js +++ b/app/L0/_all/mod/_core/openai_codex/ext/js/_core/admin/views/agent/api.js/prepareAdminAgentApiRequest/end/openai-codex.js @@ -38,6 +38,29 @@ function serializeTokens(tokens) { return tokens ? JSON.stringify(tokens) : ""; } +// chatgpt.com does not advertise CORS, so a direct browser fetch from the +// page origin is always blocked by the browser's preflight. The standard +// `prepareAdminAgentApiRequest` flow already routes its API URL through +// `space.proxy.buildUrl(...)` for proxyable external endpoints; the Codex +// hook overrides `requestUrl` and must therefore re-apply the same proxy +// routing itself. Without this, every Codex chat call pays an extra +// failed-direct-fetch roundtrip (rescued by `installFetchProxy(...)`'s +// fallback retry) and emits a red CORS error in the DevTools console on +// the first call of every page load. +function resolveCodexProxyRequestUrl(targetUrl) { + const runtimeProxy = globalThis.space?.proxy; + + if (!runtimeProxy || typeof runtimeProxy.buildUrl !== "function") { + return String(targetUrl || ""); + } + + try { + return runtimeProxy.buildUrl(targetUrl); + } catch { + return String(targetUrl || ""); + } +} + // Token persistence lives on the store/storage side. When a refresh rotates // the refresh token the hook must hand the new payload back to the store so // that encoding, userCrypto, and YAML writing stay owned by `storage.js`. @@ -76,11 +99,12 @@ export default async function openAiCodexAdminRequestHook(hookContext) { ? settings.codexModel.trim() : chatBody.model; const codexBody = chatToResponsesRequest({ ...chatBody, model }); + const proxiedRequestUrl = resolveCodexProxyRequestUrl(CODEX_RESPONSES_ENDPOINT); const withHeaders = applyCodexHeaders( { ...apiRequest, requestBody: codexBody, - requestUrl: CODEX_RESPONSES_ENDPOINT + requestUrl: proxiedRequestUrl }, { accessToken: freshTokens.accessToken, @@ -88,9 +112,18 @@ export default async function openAiCodexAdminRequestHook(hookContext) { } ); + // The proxy endpoint is itself an authenticated space-agent route that + // needs the browser session cookie. It strips the cookie header before + // forwarding upstream so this does not leak `space_session` to chatgpt.com. + const requestInit = { + ...(withHeaders.requestInit && typeof withHeaders.requestInit === "object" ? withHeaders.requestInit : {}), + credentials: "same-origin" + }; + hookContext.result = { ...withHeaders, - apiEndpoint: CODEX_RESPONSES_ENDPOINT, + apiEndpoint: proxiedRequestUrl, + requestInit, settings: { ...settings, codexTokens: serializeTokens(freshTokens) diff --git a/app/L0/_all/mod/_core/openai_codex/ext/js/_core/onscreen_agent/api.js/prepareOnscreenAgentApiRequest/end/openai-codex.js b/app/L0/_all/mod/_core/openai_codex/ext/js/_core/onscreen_agent/api.js/prepareOnscreenAgentApiRequest/end/openai-codex.js index fa7599fb..76829f2f 100644 --- a/app/L0/_all/mod/_core/openai_codex/ext/js/_core/onscreen_agent/api.js/prepareOnscreenAgentApiRequest/end/openai-codex.js +++ b/app/L0/_all/mod/_core/openai_codex/ext/js/_core/onscreen_agent/api.js/prepareOnscreenAgentApiRequest/end/openai-codex.js @@ -38,6 +38,29 @@ function serializeTokens(tokens) { return tokens ? JSON.stringify(tokens) : ""; } +// chatgpt.com does not advertise CORS, so a direct browser fetch from the +// page origin is always blocked by the browser's preflight. The standard +// `prepareOnscreenAgentCompletionRequest` flow already routes its API URL +// through `space.proxy.buildUrl(...)` for proxyable external endpoints; the +// Codex hook overrides `requestUrl` and must therefore re-apply the same +// proxy routing itself. Without this, every Codex chat call pays an extra +// failed-direct-fetch roundtrip (rescued by `installFetchProxy(...)`'s +// fallback retry) and emits a red CORS error in the DevTools console on +// the first call of every page load. +function resolveCodexProxyRequestUrl(targetUrl) { + const runtimeProxy = globalThis.space?.proxy; + + if (!runtimeProxy || typeof runtimeProxy.buildUrl !== "function") { + return String(targetUrl || ""); + } + + try { + return runtimeProxy.buildUrl(targetUrl); + } catch { + return String(targetUrl || ""); + } +} + // Token persistence lives on the store/storage side. When a refresh rotates // the refresh token the hook must hand the new payload back to the store so // that encoding, userCrypto, and YAML writing stay owned by `storage.js`. @@ -78,11 +101,12 @@ export default async function openAiCodexOnscreenRequestHook(hookContext) { ? settings.codexModel.trim() : chatBody.model; const codexBody = chatToResponsesRequest({ ...chatBody, model }); + const proxiedRequestUrl = resolveCodexProxyRequestUrl(CODEX_RESPONSES_ENDPOINT); const withHeaders = applyCodexHeaders( { ...apiRequest, requestBody: codexBody, - requestUrl: CODEX_RESPONSES_ENDPOINT + requestUrl: proxiedRequestUrl }, { accessToken: freshTokens.accessToken, @@ -90,9 +114,18 @@ export default async function openAiCodexOnscreenRequestHook(hookContext) { } ); + // The proxy endpoint is itself an authenticated space-agent route that + // needs the browser session cookie. It strips the cookie header before + // forwarding upstream so this does not leak `space_session` to chatgpt.com. + const requestInit = { + ...(withHeaders.requestInit && typeof withHeaders.requestInit === "object" ? withHeaders.requestInit : {}), + credentials: "same-origin" + }; + hookContext.result = { ...withHeaders, - apiEndpoint: CODEX_RESPONSES_ENDPOINT, + apiEndpoint: proxiedRequestUrl, + requestInit, settings: { ...settings, codexTokens: serializeTokens(freshTokens) From 3974f622846ee4448be8d01270d534ec3f67010e Mon Sep 17 00:00:00 2001 From: "Syring, Nikolas" Date: Tue, 12 May 2026 11:53:46 +0200 Subject: [PATCH 29/29] fix(openai-codex): read SINGLE_USER_APP from runtime config, not runtime.params `token_envelope.js` checked `runtime?.params?.SINGLE_USER_APP` for the single-user-mode branch, but frontend runtime configuration is exposed through `runtime.config.get("SINGLE_USER_APP", false)` in this codebase (see `_core/admin/views/agent/storage.js`, `_core/onscreen_agent/storage.js`, `_core/user/storage.js`, and `_core/framework/js/context.js` for the established read path). With the legacy path the helper silently fell through to the multi-user branch under SINGLE_USER_APP=true, so a `userCrypto:`-wrapped legacy ciphertext loaded into Codex token storage would be reported as `locked: false` and reach the runtime as an empty plaintext value instead of being preserved as locked metadata. Switch the check to the same `runtime?.config?.get?.(...)` pattern the sibling storage modules use, so SINGLE_USER_APP single-user mode is detected uniformly across the chat storage path and the Codex token envelope. Add `tests/openai_codex_token_envelope_test.mjs` to cover both the correct single-user-mode branch and an explicit regression guard that the helper no longer reads `runtime.params.SINGLE_USER_APP`. Tests cover serializeCodexTokens, parseCodexTokens, decodeStoredCodexTokens across single-user, multi-user-with-crypto, and missing-crypto runtimes, plus encodeStoredCodexTokens for the locked-preserve, single-user, multi-user, and cleared-tokens branches. docs(openai-codex): list the module from app/AGENTS.md, modules-and-extensions.md, and the admin and onscreen agent runtime docs `app/AGENTS.md` and `docs/app/modules-and-extensions.md` enumerate every documented frontend module, and the admin-agent and onscreen-agent runtime docs describe the provider switch surfaced in their settings modals. Both were missing entries for `_core/openai_codex` and listed the provider switch as two tabs (API and Local) instead of three (API, Codex, and Local). Add the module to both indexes, summarize the Codex provider in the modules-and-extensions JS extension hooks list next to the existing OpenRouter entry, and extend both runtime docs to describe the third provider tab with a pointer to the module's own AGENTS.md. Co-Authored-By: Claude Opus 4.7 (1M context) --- app/AGENTS.md | 1 + .../docs/agent/onscreen-agent-runtime.md | 2 +- .../docs/app/admin-agent-runtime.md | 1 + .../docs/app/modules-and-extensions.md | 2 + .../mod/_core/openai_codex/token_envelope.js | 2 +- tests/openai_codex_token_envelope_test.mjs | 175 ++++++++++++++++++ 6 files changed, 181 insertions(+), 2 deletions(-) create mode 100644 tests/openai_codex_token_envelope_test.mjs diff --git a/app/AGENTS.md b/app/AGENTS.md index 02229fae..0aeb9938 100644 --- a/app/AGENTS.md +++ b/app/AGENTS.md @@ -53,6 +53,7 @@ Current module-local docs in the app tree: - `app/L0/_all/mod/_core/onscreen_agent/AGENTS.md` - `app/L0/_all/mod/_core/onscreen_menu/AGENTS.md` - `app/L0/_all/mod/_core/open_router/AGENTS.md` +- `app/L0/_all/mod/_core/openai_codex/AGENTS.md` - `app/L1/_all/mod/metrics/posthog/AGENTS.md` - `app/L0/_admin/mod/_core/overlay_agent/AGENTS.md` diff --git a/app/L0/_all/mod/_core/documentation/docs/agent/onscreen-agent-runtime.md b/app/L0/_all/mod/_core/documentation/docs/agent/onscreen-agent-runtime.md index e0eb2309..0bdf1fd7 100644 --- a/app/L0/_all/mod/_core/documentation/docs/agent/onscreen-agent-runtime.md +++ b/app/L0/_all/mod/_core/documentation/docs/agent/onscreen-agent-runtime.md @@ -110,7 +110,7 @@ The settings and prompt-history dialogs reuse the shared `_core/visual/forms/dia Caught overlay runtime errors are logged through `console.error` and shown through the shared toast stack from `_core/visual/chrome/toast.js`. The composer placeholder still belongs to ready-state and lightweight status guidance, so raw exception text should not be pushed into the textarea placeholder. Overlay execution transcripts now use the shared YAML-first formatter for both console logs and returned values, emitting block headers such as `log↓`, `warn↓`, `error↓`, and `result↓` so structured telemetry stays complete across the thread and execution cards. Immediately before those execution results are serialized back into the `execution-output` follow-up turn, the overlay also runs the shared assistant-message evaluation seam; the current first-party hook from `_core/agent-chat` prepends synthetic loop warnings when the exact same assistant message reappears, using `info` on the 2nd send, `warn` on the 3rd send, and `error` on the 4th send onward. -The settings dialog now has two provider tabs named `API` and `Local`. `API` keeps the OpenAI-compatible endpoint, model, and key fields. `Local` mounts the shared Hugging Face config sidebar in onscreen mode, so the overlay reads the same saved-model list and live WebGPU worker state as the routed Local LLM page and the admin chat. Opening the Local tab should refresh saved-model shortcuts without booting the worker; saving local settings persists the selected repo id and dtype, then starts background model preparation. When no local model is selected and saved models exist, the Local panel preselects the browser-wide last successfully loaded saved model from `_core/huggingface/manager.js`, falling back to the first saved entry if that last-used entry was discarded. When no local model is selected, no local model is loaded, and the shared saved-model list is empty, the Local panel prefills the Hugging Face model field with the same testing-page default: `onnx-community/gemma-4-E4B-it-ONNX`. +The settings dialog now has three provider tabs named `API`, `Codex`, and `Local`. `API` keeps the OpenAI-compatible endpoint, model, and key fields. `Codex` uses an existing OpenAI ChatGPT Plus subscription through the OpenAI Codex OAuth flow instead of an API key, with curated model selection and silent server-side token refresh; the request rewrite, token storage, and proxy-routing contract live in `app/L0/_all/mod/_core/openai_codex/AGENTS.md`. `Local` mounts the shared Hugging Face config sidebar in onscreen mode, so the overlay reads the same saved-model list and live WebGPU worker state as the routed Local LLM page and the admin chat. Opening the Local tab should refresh saved-model shortcuts without booting the worker; saving local settings persists the selected repo id and dtype, then starts background model preparation. When no local model is selected and saved models exist, the Local panel preselects the browser-wide last successfully loaded saved model from `_core/huggingface/manager.js`, falling back to the first saved entry if that last-used entry was discarded. When no local model is selected, no local model is loaded, and the shared saved-model list is empty, the Local panel prefills the Hugging Face model field with the same testing-page default: `onnx-community/gemma-4-E4B-it-ONNX`. The API-key composer blocker applies only to the default API-provider configuration with no API key, where the composer shows a centered `Set LLM API key` action over the disabled textarea. Local Hugging Face mode can send without an API key and falls back to loading the selected local model on the first message if background preparation has not finished. diff --git a/app/L0/_all/mod/_core/documentation/docs/app/admin-agent-runtime.md b/app/L0/_all/mod/_core/documentation/docs/app/admin-agent-runtime.md index 458c5902..575ecb57 100644 --- a/app/L0/_all/mod/_core/documentation/docs/app/admin-agent-runtime.md +++ b/app/L0/_all/mod/_core/documentation/docs/app/admin-agent-runtime.md @@ -60,6 +60,7 @@ Manual loads follow the same placement rule as the onscreen agent: `history` pla The admin settings modal now starts with a provider switch: - `API`: the existing endpoint, model, API key, params, and max-token settings +- `Codex`: an OAuth-authenticated path that uses an existing OpenAI ChatGPT Plus subscription instead of an API key. See `app/L0/_all/mod/_core/openai_codex/AGENTS.md` for the request rewrite, token storage, and proxy-routing contract - `Local`: a browser-local Hugging Face path that uses Transformers.js on WebGPU Below those provider-specific sections, the shared settings area also exposes `max_tokens`, prompt-budget ratios for `system`, `history`, and `transient`, plus the separate single-history-message ratio used by the shared trimming path. Those values are persisted in `prompt_budget_ratios` and feed the same prompt-budget builder used by the onscreen agent: prepared entries and prompt items reuse cached token counts, single live history messages are capped first, contributor-level trims must each be at least `250` tokens, and `system` or `transient` falls back to one combined section-body trim when smaller contributor cuts would otherwise be required. diff --git a/app/L0/_all/mod/_core/documentation/docs/app/modules-and-extensions.md b/app/L0/_all/mod/_core/documentation/docs/app/modules-and-extensions.md index 9a844806..cc077c9e 100644 --- a/app/L0/_all/mod/_core/documentation/docs/app/modules-and-extensions.md +++ b/app/L0/_all/mod/_core/documentation/docs/app/modules-and-extensions.md @@ -14,6 +14,7 @@ This doc covers how browser code is delivered and composed. - `app/L0/_all/mod/_core/visual/AGENTS.md` - `app/L0/_all/mod/_core/login_hooks/AGENTS.md` - `app/L0/_all/mod/_core/open_router/AGENTS.md` +- `app/L0/_all/mod/_core/openai_codex/AGENTS.md` - `app/L0/_all/mod/_core/onscreen_menu/AGENTS.md` - `app/L0/_all/mod/_core/promptinclude/AGENTS.md` - `app/L0/_all/mod/_core/router/AGENTS.md` @@ -128,6 +129,7 @@ Rules: - `_core/login_hooks` is another headless helper module: it extends `_core/framework/initializer.js/initialize/end`, checks for the client-owned `~/meta/login_hooks.json` marker, dispatches `_core/login_hooks/first_login` once when that marker is absent, and dispatches `_core/login_hooks/any_login` when the authenticated shell was reached directly from `/login`; `_core/spaces` currently consumes `_core/login_hooks/first_login` through `ext/js/_core/login_hooks/first_login/big-bang-space.js` to copy or reuse the module-owned `Big Bang` onboarding space and rewrite the root-shell default route before dashboard loads - `_core/user_crypto` is a headless runtime helper module: it extends `_core/framework/initializer.js/initialize/end`, reads `space.api.userSelfInfo()` for the current backend `sessionId` plus `userCrypto` state, restores the unlocked browser key from per-tab session storage when available and otherwise from the encrypted `localStorage` blob through `/api/user_crypto_session_key`, uses that same endpoint to backfill the persisted local blob from an already-unlocked tab, stores that blob under one fixed key, logs concise `console.warn(...)` messages when cache or bootstrap restore paths fail but the module can still continue fail-soft, and signs the browser out when the backend reports that the user still needs a first-login `userCrypto` provisioning run at `/login` or when the persisted blob is stale for the current session - `_core/open_router` is a headless provider-policy module: it extends `_core/onscreen_agent/api.js/prepareOnscreenAgentApiRequest/end` and `_core/admin/views/agent/api.js/prepareAdminAgentApiRequest/end`, detects when API mode targets an OpenRouter upstream endpoint, and applies the OpenRouter-specific request headers there instead of hardcoding them inside the chat runtimes +- `_core/openai_codex` is a headless provider module for the OpenAI Codex (ChatGPT Plus) subscription. It extends the same `prepareOnscreenAgentApiRequest/end` and `prepareAdminAgentApiRequest/end` seams, detects when the active settings select the Codex provider, ensures a fresh access token through the server-side refresh endpoint, rewrites the chat-completions body into the Codex Responses API shape, and routes the call through `space.proxy.buildUrl(...)` because `chatgpt.com` does not advertise CORS for direct browser fetches. Token persistence stays on the storage side via the shared `token_envelope` helpers, and a small `token_manager` coalesces concurrent refresh attempts Uncached HTML `` lookups are grouped before they hit `/api/extensions_load`: diff --git a/app/L0/_all/mod/_core/openai_codex/token_envelope.js b/app/L0/_all/mod/_core/openai_codex/token_envelope.js index cd7645ec..6a5ae16e 100644 --- a/app/L0/_all/mod/_core/openai_codex/token_envelope.js +++ b/app/L0/_all/mod/_core/openai_codex/token_envelope.js @@ -5,7 +5,7 @@ function getUserCryptoRuntime(runtime) { } function isSingleUserAppRuntime(runtime) { - return Boolean(runtime?.params?.SINGLE_USER_APP); + return Boolean(runtime?.config?.get?.("SINGLE_USER_APP", false)); } // Serialize an in-memory Codex tokens object into the JSON string used as the diff --git a/tests/openai_codex_token_envelope_test.mjs b/tests/openai_codex_token_envelope_test.mjs new file mode 100644 index 00000000..980cb709 --- /dev/null +++ b/tests/openai_codex_token_envelope_test.mjs @@ -0,0 +1,175 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + decodeStoredCodexTokens, + encodeStoredCodexTokens, + parseCodexTokens, + serializeCodexTokens +} from "../app/L0/_all/mod/_core/openai_codex/token_envelope.js"; + +function createConfigStub(values = {}) { + return { + get: (key, fallback) => (key in values ? values[key] : fallback), + values + }; +} + +function createUserCryptoStub({ decrypt, encrypt } = {}) { + return { + decryptText: async (value) => (typeof decrypt === "function" ? decrypt(value) : ""), + encryptText: async (value) => (typeof encrypt === "function" ? encrypt(value) : "") + }; +} + +function createRuntime({ singleUserApp = false, userCrypto = null, useParamsForLegacyBugRepro = false } = {}) { + const runtime = { + config: createConfigStub({ SINGLE_USER_APP: singleUserApp }), + utils: userCrypto ? { userCrypto } : {} + }; + + if (useParamsForLegacyBugRepro) { + runtime.params = { SINGLE_USER_APP: singleUserApp }; + } + + return runtime; +} + +test("serializeCodexTokens returns JSON string for object input", () => { + assert.equal(serializeCodexTokens({ access_token: "abc" }), '{"access_token":"abc"}'); +}); + +test("serializeCodexTokens returns '' for falsy or non-object input", () => { + assert.equal(serializeCodexTokens(null), ""); + assert.equal(serializeCodexTokens(undefined), ""); + assert.equal(serializeCodexTokens("string"), ""); + assert.equal(serializeCodexTokens(42), ""); +}); + +test("parseCodexTokens accepts an object directly", () => { + const value = { access_token: "abc" }; + assert.deepEqual(parseCodexTokens(value), value); +}); + +test("parseCodexTokens parses a JSON string into an object", () => { + assert.deepEqual(parseCodexTokens('{"access_token":"abc"}'), { access_token: "abc" }); +}); + +test("parseCodexTokens returns null for empty, malformed or non-object payloads", () => { + assert.equal(parseCodexTokens(""), null); + assert.equal(parseCodexTokens(" "), null); + assert.equal(parseCodexTokens("not-json"), null); + assert.equal(parseCodexTokens("[1,2,3]"), null); + assert.equal(parseCodexTokens(null), null); +}); + +test("decodeStoredCodexTokens returns a blank result for empty stored value", async () => { + const runtime = createRuntime(); + const result = await decodeStoredCodexTokens(runtime, ""); + assert.deepEqual(result, { locked: false, storedValue: "", value: "" }); +}); + +test("decodeStoredCodexTokens returns a locked result in SINGLE_USER_APP for legacy ciphertext", async () => { + const runtime = createRuntime({ singleUserApp: true }); + const ciphertext = "userCrypto:legacy-payload"; + + const result = await decodeStoredCodexTokens(runtime, ciphertext); + + assert.deepEqual(result, { + locked: true, + storedValue: ciphertext, + value: "" + }); +}); + +test("decodeStoredCodexTokens ignores runtime.params.SINGLE_USER_APP — the read path is runtime.config.get", async () => { + // Regression guard: previously the single-user check read from + // `runtime.params.SINGLE_USER_APP`, which is not how frontend runtime + // config is exposed. With the legacy path set but the correct config + // path absent, the helper must NOT take the single-user branch. + const runtime = { + params: { SINGLE_USER_APP: true }, + config: createConfigStub({}), + utils: { userCrypto: createUserCryptoStub({ decrypt: () => "" }) } + }; + const ciphertext = "userCrypto:legacy-payload"; + + const result = await decodeStoredCodexTokens(runtime, ciphertext); + + assert.equal(result.locked, true); + assert.equal(result.value, ""); + assert.equal(result.storedValue, ciphertext); +}); + +test("decodeStoredCodexTokens decrypts in multi-user runtime when userCrypto is available", async () => { + const runtime = createRuntime({ + userCrypto: createUserCryptoStub({ decrypt: () => '{"access_token":"abc"}' }) + }); + + const result = await decodeStoredCodexTokens(runtime, "userCrypto:wrapped"); + + assert.equal(result.locked, false); + assert.equal(result.value, '{"access_token":"abc"}'); +}); + +test("decodeStoredCodexTokens reports locked when userCrypto is missing and value is wrapped", async () => { + const runtime = createRuntime(); + const result = await decodeStoredCodexTokens(runtime, "userCrypto:wrapped"); + + assert.equal(result.locked, true); + assert.equal(result.value, ""); +}); + +test("encodeStoredCodexTokens preserves locked stored ciphertext when caller did not replace tokens", async () => { + const runtime = createRuntime(); + const result = await encodeStoredCodexTokens(runtime, { + codexTokens: "", + storedCodexTokensLocked: true, + storedCodexTokensValue: "userCrypto:previous" + }); + + assert.equal(result, "userCrypto:previous"); +}); + +test("encodeStoredCodexTokens returns plaintext in SINGLE_USER_APP", async () => { + const runtime = createRuntime({ singleUserApp: true }); + + const result = await encodeStoredCodexTokens(runtime, { + codexTokens: '{"access_token":"abc"}' + }); + + assert.equal(result, '{"access_token":"abc"}'); +}); + +test("encodeStoredCodexTokens encrypts in multi-user runtime when userCrypto is available", async () => { + const runtime = createRuntime({ + userCrypto: createUserCryptoStub({ encrypt: (value) => `userCrypto:${value}` }) + }); + + const result = await encodeStoredCodexTokens(runtime, { + codexTokens: '{"access_token":"abc"}' + }); + + assert.equal(result, 'userCrypto:{"access_token":"abc"}'); +}); + +test("encodeStoredCodexTokens throws when userCrypto is unavailable in multi-user runtime", async () => { + const runtime = createRuntime(); + + await assert.rejects( + () => encodeStoredCodexTokens(runtime, { codexTokens: '{"access_token":"abc"}' }), + /userCrypto is unavailable/u + ); +}); + +test("encodeStoredCodexTokens returns '' when caller cleared tokens and no locked ciphertext is held", async () => { + const runtime = createRuntime({ singleUserApp: true }); + + const result = await encodeStoredCodexTokens(runtime, { + codexTokens: "", + storedCodexTokensLocked: false, + storedCodexTokensValue: "" + }); + + assert.equal(result, ""); +});