Skip to content

docs: add guide for running the service locally as an API endpoint#259

Merged
smirnovlad merged 2 commits into
mainfrom
docs/prm-service-guide
Jun 30, 2026
Merged

docs: add guide for running the service locally as an API endpoint#259
smirnovlad merged 2 commits into
mainfrom
docs/prm-service-guide

Conversation

@smirnovlad

@smirnovlad smirnovlad commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Adds docs/service/running_locally.md — a general guide for building the ThinkBooster service locally and calling it as an OpenAI-compatible endpoint, with the scaling strategy and scorer in the URL.

It leads with the no-GPU path (self_consistency against a remote model), then covers beam search / best-of-N with the uncertainty scorers, and folds the PRM scorer and local generation into one GPU section (env vars, Docker GPU override, the tts_api_key remote-generation routing). Cross-links api_guide.md for the full parameter and metadata reference.

This replaces the earlier PRM-specific draft on the branch — the guide is now general and not PRM-named. The launch article (#258) links here for the "run it as a service" path. All referenced files (docker-compose.yml, service_app/Dockerfile, .env.example, the [service] extra) are verified present on main.

Replace the PRM-specific running_with_prm.md with running_locally.md: a general guide to building the service and calling it as an OpenAI-compatible endpoint. Leads with the no-GPU self_consistency path; PRM and local generation are covered as the GPU section. Cross-links api_guide.md.
@smirnovlad smirnovlad changed the title docs: add guide for running the service with the PRM scorer docs: add guide for running the service locally as an API endpoint Jun 30, 2026
@smirnovlad-test smirnovlad-test self-requested a review June 30, 2026 10:38
@smirnovlad smirnovlad merged commit 561489b into main Jun 30, 2026
1 check passed
@smirnovlad smirnovlad deleted the docs/prm-service-guide branch June 30, 2026 12:22
smirnovlad added a commit that referenced this pull request Jun 30, 2026
Lead the CTA with the in-browser demo (http://demo-thinkbooster.nlpresearch.group, verified up). Note the http-only/ephemeral-pod caveat in the pre-publish checklist; mark PR #259 (service guide) merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants