GitHub - rawsh/mirrorllm: various experiments for scaling inference time compute with small reasoning models

various experiments for scaling inference time compute with small reasoning models

high throughput async mcts implementation for policy + prm hosted on serverless gpus on modal

Name		Name	Last commit message	Last commit date
Latest commit History 30 Commits
.github/workflows		.github/workflows
backup		backup
mcts		mcts
prm_rlhf_flow		prm_rlhf_flow
reinforcement_learning		reinforcement_learning
vespa		vespa
.dockerignore		.dockerignore
.gitignore		.gitignore
Dockerfile		Dockerfile
best_of_n.py		best_of_n.py
dspy_reflect.py		dspy_reflect.py
eval_reward.py		eval_reward.py
fly.toml		fly.toml
lib.py		lib.py
local_reward.py		local_reward.py
mcts_results.jsonl		mcts_results.jsonl
modal_orm_reward.py		modal_orm_reward.py
modal_prime.py		modal_prime.py
modal_prm_armorm.py		modal_prm_armorm.py
modal_prm_reward.py		modal_prm_reward.py
modal_train_policy_orpo.py		modal_train_policy_orpo.py
modal_train_policy_sft.py		modal_train_policy_sft.py
modal_train_policy_sft_metamath.py		modal_train_policy_sft_metamath.py
modal_train_policy_sft_prime.py		modal_train_policy_sft_prime.py
modal_train_policy_sft_qwq.py		modal_train_policy_sft_qwq.py
modal_train_policy_simpo.py		modal_train_policy_simpo.py
modal_train_prm_init.py		modal_train_prm_init.py
modal_train_prm_rlhf_flow.py		modal_train_prm_rlhf_flow.py
modal_train_prm_st.py		modal_train_prm_st.py
modal_tree_grpo.py		modal_tree_grpo.py
modal_vespa.py		modal_vespa.py
modal_vllm.py		modal_vllm.py
modal_vllm_chat.py		modal_vllm_chat.py
modal_vllm_prm.py		modal_vllm_prm.py
modal_vllm_qwq.py		modal_vllm_qwq.py
poetry.lock		poetry.lock
prime_training_local.py		prime_training_local.py
pyproject.toml		pyproject.toml
readme.md		readme.md
test_vllm_prm.py		test_vllm_prm.py
tweak_prime_rl_data.py		tweak_prime_rl_data.py

Provide feedback