Performance improvement discussion

[ewilderj]

Hey @xenodium following up on the shell-maker performance issue, I dug a bit deeper.

As I mentioned in agent-shell#403 there are some big table speedups to be had. I won't repeat that data here, but focus on the overlays side of the problem.

The other drag in performance relevant to xenodium/agent-shell#403 is the massive number of overlays that get generated by extended sessions. It is much faster to use text properties where we can.

In long-running agent-shell sessions, Emacs progressively slows down as overlays accumulate. e.g. in a ~530KB buffer, markdown-overlays creates ~9,000 overlays. In practice, users report that above a few thousand overlays, cursor movement, typing, and redisplay become noticeably sluggish. This is one of the root causes behind the high CPU reports in agent-shell#403.

Of the 8,982 markdown-overlays in my example buffer:

Property	Count	What
face	3,749	Bold, italic, code, source block highlighting
invisible	2,362	Hidden markdown syntax characters
display	602	Table border replacements
before-string	291	Table border prefixes

Source block syntax highlighting alone accounts for ~2,300 overlays — one per highlighted token per character run.

I have been investigating and done both table caching optimizations, and also changes that uses text properties for non-table Markdown fontification. For your perusal they are here:

• feature/perf-optimization — table caching + defconst extraction + span walk
• feature/overlay-to-properties — converts face/invisible overlays to text properties (90% overlay reduction)
• feature/perf-combined — rolls up both of them

I am currently testing the combined fix as daily driver. Let me know how you would prefer to sequence these as PRs, as they're pretty much orthogonal. In total there are 3 perf improvements I've readied:

• convert regexps to constants
• use text properties not overlays
• cache tables rather than re-rendering on markdown-overlays-put

[xenodium]

Thanks for this!

convert regexps to constants

While nice to have anyway, I suspect we may not get optimizations as the macro expansions get byte-compiled in?

cache tables rather than re-rendering on markdown-overlays-put

I'm thinking we can likely reduce caching into a single location. Take a look at https://github.com/xenodium/shell-maker/pull/23/changes. They also combine the proposed optimization from next-single-property-change. I'm hoping we can reduce the maintenance surface. Can you give these changes a try?

as they're pretty much orthogonal.

Nice. I'd say we start with caching and see how we get on. Sound ok?

[ewilderj]

Yeah, that's much cleaner. Love it!

I would like to suggest we still take a look at using text properties, it did make a material difference to interactive performance, particularly when scrolling around. But I understand it's a bigger change and more vulnerable to regressions so this sequencing of addressing table caching first is what I expected.