Performance: updateFromUIMessageChunks assert eagerly JSON.stringify’s large messages

[KevinGrajeda]

I’ve been debugging some pretty bad frontend performance in an app using useThreadMessages with streaming enabled. In some cases the UI was dropping to 2 FPS 😭

I know we have a fair amount of re‑rendering going on, but even accounting for that, things felt way slower than expected. So I recorded a performance trace in Chrome and found this:

• updateFromUIMessageChunks in deltas.ts is taking a big chunk of the main thread time

• Inside that function, this assert line seems to be the hotspot :

agent/src/deltas.ts

Lines 80 to 83 in b48f51c

	assert(
	messagePart.id === message.id,
	`Expecting to only make one UIMessage in a stream, but have ${JSON.stringify(message)} and created ${JSON.stringify(messagePart)}`,
	);

it is surprising, because this is just an equality check. But my theory is that, regardless of whether the equality check passes, the JSON.stringify calls in the template string are always evaluated eagerly.

Since updateFromUIMessageChunks is called for every delta while streaming, this means we’re doing a full JSON.stringify of the whole message on every new delta.

useThreadMessages({ stream: true })
  ↓
useStreamingThreadMessages
  ↓
useStreamingUIMessages
  ↓
useDeltaStreams (fetches stream deltas)
  ↓
useEffect in useStreamingUIMessages (on new deltas)
  ↓
updateFromUIMessageChunks

this happens with both useThreadMessages and useUIMessages

Possible fix

If my understanding is correct, this should be a relatively easy performance win by making the error message lazy, so JSON.stringify only runs on failure.

Happy to provide more details if helpful.

[ianmacartney]

Fixed in #186 - try it out with npm i https://pkg.pr.new/get-convex/agent/@convex-dev/agent@186 and let me know if you still see issues! Will release it soon

[KevinGrajeda]

thank you for the quick response, i’ve been testing the pr and the updateFromUIMessageChunks issue seems to be gone 🙌

however, I’m now seeing another performance problem around the fixJson and secureJsonParse functions.
on a similar testing environment (not that similar tbh but should be good enough) these two functions take almost no time to run on 0.3.0:

but on #186, for some reason I don’t fully understand, these now take significantly more time and become the new bottleneck:

I’m guessing somewhere between 0.3.0 and #186 there was a change that makes a code path using fixJson / secureJsonParse much more common, but that’s just a guess

if I can provide any other info that would help debug this, let me know

[KevinGrajeda]

maybe this is the problem:

In #186, useDeltaStreams.ts was changed to always update state.deltaStreams even when there are no new deltas:

// Line 106: Always enters if streamMessages exist
if (streamMessages) {
  if (newDeltas?.length) {
    // process new deltas
  }
  // ⚠️ This ALWAYS runs, even with no new deltas:
  state.deltaStreams = streamMessages.map(...); // Creates new object reference
}

Previous behavior :

// Line 106: Only enters when there are new deltas
if (newDeltas?.length && streamMessages) {
  // process new deltas
  state.deltaStreams = streamMessages.map(...); // Only updates when needed
}

sooo

1. state.deltaStreams gets a new reference even with no new deltas
2. useStreamingUIMessages sees streams change (new reference)
3. useEffect runs unnecessarily
4. updateFromUIMessageChunks() is called with all existing deltas
5. readUIMessageStream() processes everything from scratch
6. fix-json.ts runs repeatedly, causing performance issues

[ianmacartney]

Can you try out @convex-dev/agent@0.3.1? It takes out the JSON stringify but doesn't include the change to the delta stream calculation

[KevinGrajeda]

I tested out 0.3.1 and seems to solve the problem 👍
^not true, im getting the same fixJson and secureJsonParse issue, not sure what is causing it

[KevinGrajeda]

also about #186 i tried doing a quick patch solving #187 (comment) and it seemed to solve the problem
not true ^

[KevinGrajeda]

Im currently using useThreadMessages instead of useUIMessages
is it possible that this is causing the issues?

[KevinGrajeda]

ok, I think I finally found the real issue that was causing all of these problems to bubble up. since this specific issue (the assert / updateFromUIMessageChunks one) is solved on 0.3.1, I’m going to close this issue and open a new one with what I believe is the actual core problem. will link it shortly.