Skip to content

Compiler: auto-size dispatch chunks to stay under JIT HugeMethodLimit#1288

Merged
andreaTP merged 1 commit intodylibso:mainfrom
zklapow:aot-dispatch-dynamic
May 6, 2026
Merged

Compiler: auto-size dispatch chunks to stay under JIT HugeMethodLimit#1288
andreaTP merged 1 commit intodylibso:mainfrom
zklapow:aot-dispatch-dynamic

Conversation

@zklapow
Copy link
Copy Markdown
Contributor

@zklapow zklapow commented May 5, 2026

The JVM's C2 JIT compiler has a HugeMethodLimit (default 8KB bytecode).

At the previous chunk size of 1024 entries, the generated dispatch methods exceed 8KB and never get full C2 optimization. This only affects large wasm modules (>1024 functions), but many real-world modules like CPython, SQLite, etc. easily hit this.

Instead of a fixed chunk size, this PR derives it from the bytecode budget: highestOneBit(hugeMethodLimit / bytesPerEntry), which produces dispatch methods that stay under the limit and get full C2 compilation. The limit is configurable via -Dchicory.hugeMethodLimit=<bytes> for JVMs with non-default settings for the hugemethod limit.

Benchmark results (2000-function module, JMH, JDK 25):

targetFunc (1024/chunk) (128/chunk, auto-sized) speedup
0 0.033 us/op 0.004 us/op ~8x
999 0.034 us/op 0.004 us/op ~8.5x
1999 0.035 us/op 0.004 us/op ~8.75x

In practice in cpython I found this to give about a 15-20% speedup on hot loops.

@zklapow
Copy link
Copy Markdown
Contributor Author

zklapow commented May 5, 2026

I originally did this by simply having MAX_MACHINE_CALL_METHODS be configureable, but that seemed worse than simply deriving from the limit and I imagine most workloads will have left the default 8kb so this produces more optimal results in the base case.

@zklapow
Copy link
Copy Markdown
Contributor Author

zklapow commented May 5, 2026

Will take a look at CI failures

@zklapow zklapow force-pushed the aot-dispatch-dynamic branch 2 times, most recently from 70845fd to 1cff1dd Compare May 5, 2026 13:40
Copy link
Copy Markdown
Collaborator

@andreaTP andreaTP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea! Thanks, a couple of minor comments/questions.

Comment thread compiler/src/main/java/com/dylibso/chicory/compiler/internal/Compiler.java Outdated
@zklapow zklapow force-pushed the aot-dispatch-dynamic branch from 1cff1dd to 7958f14 Compare May 6, 2026 01:53
@zklapow zklapow force-pushed the aot-dispatch-dynamic branch from 7958f14 to b2ee7e9 Compare May 6, 2026 02:20
Copy link
Copy Markdown
Collaborator

@andreaTP andreaTP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@andreaTP andreaTP merged commit ea8351c into dylibso:main May 6, 2026
25 checks passed
@zklapow zklapow deleted the aot-dispatch-dynamic branch May 6, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants