Feat/LLM chat

[1amKhush]

Description

Resolves #26 - LLM chat integration

This PR implements Phase 1 of the ContextVM web chat interface. It provides a robust, zero-friction LLM experience for users out of the box, alongside full Bring-Your-Own-Token (BYOT) support and IndexedDB conversation persistence.

Key Features

• Zero-Friction Auto Mode: Ships with a default (fund-less) OpenRouter API key. It dynamically fetches models, filters for the :free suffix, and automatically rotates through free models if a 429 Rate Limit is encountered.
• Conversation Management: Uses IndexedDB to persist chats across sessions. The UI includes a collapsible Shadcn sidebar allowing users to create, rename, and delete conversations.
• Smart Comboboxes:
  • Providers: A preset list allowing one-click switching between OpenRouter, OpenAI, Ollama, LM Studio, and Custom base URLs.
  • Models: Dynamically queries the active provider's /v1/models endpoint. Features an "Auto (free models)" mode pinned to the top for OpenRouter.
• Phase 2 Readiness: The Sidebar component uses a panel-switcher pattern currently defaulting to "Conversations". This is stubbed and ready to accept a "Servers" panel in Phase 2 that will tap directly into the existing mcpClientService singleton.

Technical Notes

• All new chat types, logic, and components are namespaced under src/lib/types/chat-types.ts, src/lib/services/llm.ts, src/lib/services/auto-mode.ts, and src/lib/components/chat/.
• openai SDK is used under the hood configured with dangerouslyAllowBrowser: true as this is entirely a client-side execution.
• Implements robust stale-closure guards and AbortControllers to handle rapid context switching and stream cancellation.

Testing

• Verified zero-config Auto Mode successfully rotates models.
• Verified IndexedDB properly stores and recovers chat state on reload.
• Verified BYOT configuration accurately fetches and streams from selected models.
• Svelte-check passes with 0 errors/warnings.

[vercel]

@1amKhush is attempting to deploy a commit to the ContextVM's projects Team on Vercel.

A member of the Team first needs to authorize it.

[Copilot]

Pull request overview

Implements Phase 1 of an in-browser LLM chat workspace at /chat, including provider/model selection, OpenRouter "Auto free models" rotation, streaming responses via the OpenAI SDK in the browser, IndexedDB-backed conversation persistence, and a new Shadcn-style sidebar primitive set.

Changes:

• Adds chat UI (Chat, ChatInput, ChatBubble, sidebar with conversation list, settings sheet with provider/model/key comboboxes) and a top-nav entry.
• Adds client-side services: LLMService (OpenAI SDK with dangerouslyAllowBrowser), FreeModelRotator for OpenRouter :free rotation on 429s, and an IndexedDB conversation store with reactive $state.
• Adds new generic UI primitives (sidebar, popover, command, scroll-area) and the openai dependency; ships a hardcoded "default" OpenRouter API key for zero-config use.

Reviewed changes

Copilot reviewed 51 out of 52 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/routes/chat/+page.svelte, +page.ts	New chat route, wires sidebar/header/config/Chat.
src/lib/types/chat-types.ts	Shared chat types, provider presets, and the hardcoded default OpenRouter key.
src/lib/services/llm.ts	OpenAI SDK wrapper with streaming, base URL normalization, and free-model fallback.
src/lib/services/auto-mode.ts	Free-model fetching, caching, rate-limit detection, and round-robin rotation.
src/lib/services/conversation-store.svelte.ts	IndexedDB-backed conversation CRUD and reactive store.
src/lib/components/chat/*	Chat UI: Chat, ChatBubble, ChatInput, ChatSidebar, ConversationList, LLMConfig, ProviderCombobox, ModelCombobox, AutoModeBanner.
src/lib/components/ui/sidebar/*	New sidebar primitive set (provider/context/root/header/footer/menu/etc.).
src/lib/components/ui/popover/, command/, scroll-area/*	New UI primitives consumed by the chat settings and conversation list.
src/lib/components/header.svelte	Adds Chat link to desktop and mobile navs.
package.json, bun.lock	Adds openai@^4.0.0 and its transitive deps.
.changeset/chat-phase-one.md	Minor changeset entry.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
contextvm-site	Ready	Preview, Comment	May 26, 2026 7:42am

[ContextVM-org]

I'm noticing some issues. First, the sidebar collapses in a bad way and cannot be expanded again.

When I try the chat I got
'''
Uncaught (in promise) DataCloneError: Failed to execute 'put' on 'IDBObjectStore': [object Array] could not be cloned.
at zn (6.DvmS0QwY.js:1:14167)
at async Fr (6.DvmS0QwY.js:1:15140)
at async Object.m [as onSend] (6.DvmS0QwY.js:19:11371)
'''
In the browser console so it doesnt work.
Also the textarea for input is disabled after removing a conversation and you need to manually create a new one in order to write. This doesnt need to be like that and new conversations can be handled more gracefully.
In general this needs more care

[ContextVM-org]

Great, the error are gone, but I'm not able to receive a response when writing a message, tested with the auto mode, with another model, and with my own key. Nothing happens

[1amKhush]

but I'm not able to receive a response when writing a message, tested with the auto mode, with another model, and with my own key. Nothing happens

I'll have a look (did test before and it seemed to work)

[ContextVM-org]

I cannot get it to work. Tested in the vercel deployment and in my own dev enviroment. But nothing happens, I send a message to the chat an there is no loading or nothing, also nothing in the console

[1amKhush]

I cannot get it to work. Tested in the vercel deployment and in my own dev enviroment. But nothing happens, I send a message to the chat an there is no loading or nothing, also nothing in the console

Honestly was a very small issue, Svelte’s $state() automatically wraps objects to track changes. But some native browser objects and SDKs, like AbortController and the OpenAI client, break when wrapped this way.

The fix was simple, i stored those objects as normal let variables instead of reactive $state variables. That keeps them untouched and everything works normally again. I must have wrapped them during the compilation error bug fixing i think

[1amKhush]

@ContextVM-org Up for review

[ContextVM-org]

I reviewed the PR in detail, and overall I think it is in a good state. The feature works, the UX is strong, and the implementation shows good attention to streaming behavior, persistence, and cancellation. I do think there are a few places where I’d ask for cleanup before giving it a full green light, mostly around simplification and reducing future maintenance cost.

Summary

I’m generally positive on this PR. The main chat flow, auto-mode behavior, IndexedDB persistence, and UI composition all look solid. My concern is not that the code is broken, but that some parts are more complex than they need to be, and that complexity will make Phase 2 harder to extend.

What I like

• I think the overall separation of concerns is good: types in src/lib/types/chat-types.ts, service logic in src/lib/services/llm.ts and src/lib/services/auto-mode.ts, persistence in src/lib/services/conversation-store.svelte.ts, and UI under src/lib/components/chat/.
• I like the abort/cancellation handling in handleSend(). There is clear effort to prevent stale updates and handle rapid context switching safely.
• I think the config UX in src/lib/components/chat/LLMConfig.svelte is thoughtful, especially the local persistence without storing the API key.
• The model rotation logic in FreeModelRotator is easy to follow and appropriately scoped.
• The UI structure is clean. src/lib/components/chat/ChatSidebar.svelte, src/lib/components/chat/ConversationList.svelte, and src/lib/components/chat/AutoModeBanner.svelte are all nicely factored.

Main concerns

1. handleSend() is doing too much

This is the main area where I think the implementation could be simplified.

Right now handleSend() is carrying a lot of state orchestration:

• workingMessages
• syncMessages()
• schedulePersist()
• token guards
• assistant message replacement via repeated array mapping
• persistence timing concerns

It works, but it is hard to reason about, and I think it is more imperative than it needs to be for Svelte 5. Since messages is already a $state array, I think this could be simplified significantly by mutating the tracked objects/array directly instead of rebuilding arrays so often.

I would not block the PR on a full rewrite, but I would strongly recommend reducing the amount of state choreography here before Phase 2 expands this file further.

2. The skipLoadId pattern feels fragile

The logic around skipLoadId looks like a workaround for a race between conversation creation and the conversation-loading effect.

I understand why it exists, but I think it makes the flow harder to trust because it depends on effect timing and implicit knowledge of how state changes propagate. I would prefer a more explicit flow where a new conversation is created and initialized in a single path, rather than creating it and then teaching a separate effect to ignore that transition once.

This is one of the few places where I think the code is not just complex, but also somewhat brittle.

3. conversation-store.svelte.ts is internally inconsistent

I think the store itself is well-structured overall, but the implementation style is inconsistent.

For example:

• saveConversation() uses a clean async/await approach.
• updateConversation() and renameConversation() switch into manual new Promise(...), raw IndexedDB handlers, and nested callbacks.

That inconsistency makes the file harder to maintain than necessary. I would strongly prefer standardizing these functions around the existing helpers like requestToPromise() and waitForTransaction().

I don’t think the current code is wrong, but I do think it is more verbose and lower-signal than it needs to be.

4. Auto-mode/default-key logic is duplicated

I noticed the same core logic being rederived in multiple places:

• shouldUseAutoMode()
• isAutoMode
• usingDefaultKey
• usingDefaultKey
• keyLabel

This is the kind of duplication that stays harmless at first and then becomes a consistency bug later. I would rather see small shared helpers exported from src/lib/types/chat-types.ts or a dedicated config utility.

Medium-priority cleanup

5. ModelCombobox uses a full LLMService instance just to fetch models

In src/lib/components/chat/ModelCombobox.svelte, model listing is done by constructing new LLMService(...) with a placeholder model value.

That works, but I think it is an unnecessary abstraction leak. Fetching /v1/models is a simpler concern than full chat streaming, so I would prefer either:

• a static helper on LLMService, or
• a lightweight dedicated model-fetching utility.

The current version is acceptable, but it feels heavier than necessary.

6. Missing error handling in the page bootstrap

In onMount() inside src/routes/chat/+page.svelte, the initial conversation load/create path has no error handling.

If IndexedDB fails, the page can end up in a bad state with an unhandled rejection. I would want a try/catch there and some fallback behavior, even if it is just showing a user-facing error state.

7. SSR sanitization fallback in ChatBubble.svelte

In html, markdown is sanitized only when browser is true:

• browser: sanitized with DOMPurify
• server: returns raw rendered HTML

Given the current route setup this may not be an immediate issue, but I still think returning unsanitized HTML on the server is a bad default. I would prefer a safe fallback even if the current route is not prerendered.

8. isRetryableError() is a bit too loose

The fuzzy message check in isRetryableError() uses a generic includes('rate limit') fallback. I would tighten that. Retrying based on structured status/code is good; retrying on loose message matching is more error-prone.

Low-priority cleanup

9. Small simplifications in LLMService.fetchModels()

The signal ternary in fetchModels() is a little verbose. Not important, just easy cleanup.

10. formatRelativeTime() could be extracted or replaced

The helper is fine, but I think it is either utility-worthy or replaceable with Intl.RelativeTimeFormat. Not a blocker.

11. ChatInput.svelte may be more generic than needed

The use of HTMLAttributes<HTMLDivElement> in src/lib/components/chat/ChatInput.svelte feels slightly broader than necessary for the actual component surface.

Final recommendation

If I were reviewing this PR, I would say:

I think this PR is already in a good functional state and the feature looks strong overall. I’m comfortable with the direction and most of the architecture. Before giving it a full green light, I’d like to see some cleanup in the areas where the implementation is more complex than necessary, especially handleSend(), the skipLoadId flow, and the inconsistent IndexedDB patterns in src/lib/services/conversation-store.svelte.ts. I’d also suggest centralizing the repeated auto-mode/default-key logic and adding bootstrap error handling in src/routes/chat/+page.svelte. None of this changes my view that the PR is strong, but I do think these cleanups would materially improve maintainability before Phase 2 builds on top of it.

Bottom line

I would not call this a “needs major rework” PR. I would call it good and close, with a handful of targeted cleanup items that are worth addressing now while the surface area is still manageable.