Enable host JVM GC to collect unreachable WasmGC references

[andreaTP]

Currently GcRefStore keeps strong references and runs its own mark-sweep every 1024 allocations. The JVM GC can’t collect unreachable Wasm objects because the Wasm reference graph is encoded as integer IDs (struct fields and array elements), not Java object references.

To let the JVM handle collection, we need to mirror the Wasm graph with real Java references so JVM reachability matches Wasm reachability:

• WasmStruct and WasmArray should hold direct Java references to the WasmGcRef objects they point to (in addition to integer IDs) and they should go on the MStack
• Globals and Tables should keep strong Java references to GC refs (these act as roots).
• GcRefStore should use WeakReferences so entries are collectible once no root-reachable Java chain exists.

This would remove the periodic sweep and allow the JVM to collect unreachable Wasm GC subgraphs naturally.

[A248]

Hey, quick question on the GC algorithm, and maybe a potential bug.

I noticed that you're calling GC automatically when export functions are called, with the reasoning that the WASM stack is empty.

However, host functions can call export functions in a re-entrant fashion. Wouldn't this mean that the stack is not guaranteed empty?

chicory/runtime/src/main/java/com/dylibso/chicory/runtime/Instance.java

Line 263 in ad193f9

instance.gcSafePoint();

[andreaTP]

Hi @A248 nice to see you around!

I think you might be right in theory, that's why this is an high priority issue to be sorted out, I have delayed it since, very likely, needs breaking APIs.

In practice, I haven't found yet, a WasmGC payload that rely on the re-entrancy without storing any reference in local tables/globals etc.
Also this aspect seems to not be covered by the official testsuite and I couldn't find a specifc reference in the Wasm spec itself.

[A248]

Happy to help. I'm not an expert on this particular issue; I just spot what seems like a bug and figure it should be raised or discussed.

I don't use WasmGC, but I'm sure you want a runtime implementation that works for arbitrary, correct WASM guest programs, not just the payloads you happen to observe.