Serve app as a web app and provide a max tokens option within sampling#8
Draft
ichim-david wants to merge 4 commits intostevibe:mainfrom
Draft
Serve app as a web app and provide a max tokens option within sampling#8ichim-david wants to merge 4 commits intostevibe:mainfrom
ichim-david wants to merge 4 commits intostevibe:mainfrom
Conversation
…es which broke some good results for models that are verbose
- Add Fastify server (app/src/server/) with REST API at /api/* - Add SSE event streaming at /api/events/sse for run events, mutation progress, verifier progress - Add in-process SSE bus and active run manager - Extract Electron-free helpers (themes, app-metadata, models) for server use - Add HTTP + SSE API client (app/src/renderer/src/api/client.ts) replacing IPC bridge - Adapt App.tsx: window.benchlocal.* -> bl.*, IPC listeners -> SSE, remove update/detached-logs UI - Add Vite web config (vite.config.web.ts) for renderer-only builds - Add npm scripts: web:dev, web:build, web:start - Add fastify, @fastify/static, tsx, esbuild, concurrently dependencies - Stub removed Electron features (updates, logs, onOpenAbout/onOpenSettings) for backward compatibility - Web server runs on port 4300 (configurable via BENCHLOCAL_PORT)
- Moved path resolution logic to a new module for better separation of concerns. - Updated app-metadata.ts to utilize new path resolution functions. - Enhanced error handling for license and package.json loading. - Improved server initialization in index.ts to check for renderer output directory. - Refactored SSE route handling for better response management. - Simplified theme loading logic in themes.ts by using path resolution functions. - Updated Vite configuration to use environment variables for ports and output directories.
Author
works surprisingly well given the mac os look :) Feel free to close this pr anytime and implement anything or nothing out of this work if it doesn't make sense for you. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pr isn't really meant to be merged as I'm sure it would fail your quality bar & I still need todo some cleanup before it would be merge ready but I wanted to open this pr just to see if any of these features make sense for your repo to have.
Currently it does two things:
a few benches that failed because the response got cut off (mac mini m4 pro 64gb and qwen is quite slow 6-11tps).
When I bumped to 2048 tokens or more for hermes benchmark some benches turned green with more tokens allowed for output