Fix plugin provider freeze caused by blocking fetch on the QuickJS runtime thread#185
Open
fmustafayaman wants to merge 1 commit into
Open
Conversation
… thread Plugin scrapers stop returning streams after a while and only recover after an app restart. The plugin runtime runs QuickJS on the shared Dispatchers.Default pool, and the fetch bridge bound `__native_fetch` as a synchronous function that wrapped the suspending `httpRequestRaw` in `runBlocking`. Every plugin network call therefore parked a Default-pool thread for the whole request duration and ignored coroutine cancellation. When several providers fetch at once (for example moving to the next episode a few times) or an endpoint stalls, all Default threads end up blocked inside runBlocking simultaneously. The dispatcher is then starved: no other coroutine can run, including the per-plugin withTimeout and the request cancellation in StreamsRepository, so the runtime appears frozen until the process restarts. Bind `__native_fetch` as an async (suspend) function and call httpRequestRaw directly instead of through runBlocking, and await it from the JS fetch polyfill. Network I/O now suspends instead of holding a thread, and the plugin timeout and job cancellation can actually interrupt in-flight requests. CancellationException is rethrown so cancellation is not swallowed by the existing error mapping. Co-authored-by: Cursor <cursoragent@cursor.com>
Author
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Fixes a freeze where JS plugin scrapers stop returning streams after a while and only recover after a full app restart. The streams screen gets stuck in the loading state and no plugin ever completes, even though the same providers worked moments earlier.
The root cause is in the plugin runtime's fetch bridge:
__native_fetchwas a synchronous binding that wrapped the suspendinghttpRequestRawinrunBlocking, so every plugin network request blocked aDispatchers.Defaultthread for the whole request and ignored cancellation. Under concurrency (or a stalled endpoint) the bounded Default pool gets fully blocked and the runtime can no longer make progress.This PR binds
__native_fetchas an async (suspend) binding and callshttpRequestRawdirectly (norunBlocking), andawaits it from the JSfetchpolyfill.PR type
Why
JsRuntimeruns QuickJS on the sharedDispatchers.Defaultdispatcher:FetchBridgethen registered the network entry point as a synchronous function that blocks on the suspending HTTP call:Consequences:
fetch()parks oneDispatchers.Defaultthread for the entire request duration.Dispatchers.Defaultis bounded by the CPU core count, so a handful of concurrent or slow plugin requests can block every thread in the pool at once.runBlockingdoes not observe coroutine cancellation, so neither the per-pluginwithTimeout(PLUGIN_TIMEOUT_MS)inPluginRuntimenoractiveJob.cancel()inStreamsRepositorycan interrupt an in-flight request.Once the pool is starved, no other coroutine scheduled on
Dispatchers.Defaultcan run and the plugin runtime appears frozen until the process is restarted. This is easy to hit when several plugins run together (e.g. moving to the next episode a few times) or when a provider talks to a slow/cold endpoint.Old vs new behavior
withTimeout/ cancellation can actually interrupt a stuck request.The fix
composeApp/src/fullCommonMain/kotlin/.../network/FetchBridge.kt__native_fetchwithasyncFunctioninstead offunction.performNativeFetchasuspendfunction and callhttpRequestRaw(...)directly instead of throughrunBlocking.CancellationExceptionso cancellation is not swallowed by the existing error-mappingcatch.composeApp/src/fullCommonMain/kotlin/.../js/JsBindings.ktawait __native_fetch(...)in thefetchpolyfill, since the binding now returns a promise. The polyfill function is alreadyasync, so this is a one-line change.No public API, dependency, or architecture change.
quickjs-ktalready supports async bindings (asyncFunction), so no new dependency is introduced.Desktop scope
The changed files are in the plugin runtime (
fullCommonMain) used by the desktop app. This is a desktop stability fix for installed JS scrapers.Issue or approval
Related to #78 (same user-visible symptom: provider requests stop working until the app is restarted). That report and #104 target the addon HTTP path (socket/port exhaustion from per-request
HttpClient); this PR fixes a separate root cause in the plugin runtime (Default-pool starvation from blocking fetch), so the two are complementary and do not overlap in code.UI / behavior impact
Testing
Policy check
CONTRIBUTING.md.Made with Cursor