the configs about the training settsing

[yfllllll]

Your article is excellent work! I'm currently studying it and would like to ask about the detailed configuration of the two-stage training, such as the learning rate, number of epochs, weights of various reward functions, the number of generations, etc. Could you please share these details?

[faedtodd]

Yes, thanks for the author's work, I would also appreciate some details about the configuration.
I have currently conducted training based on the descriptions in the article, initializing with the open-source 7B-CI model. Below are my configuration and reward change graphs. It can be observed that the response length begins to consistently increase in the later stages.

score_overall = 0.9 * acc_score + 0.1 * format_score + dIOU_score
And judge model is qwen2.5 72b.

python3 -m verl.trainer.main
config=examples/config.yaml
data.train_files=${TRAIN_FILE}
data.val_files=${TRAIN_FILE}
data.image_dir=${IMAGE_ROOT}
data.rollout_batch_size=1440
data.val_batch_size=1440
data.max_response_length=2048
data.filter_overlong_prompts=true
worker.actor.model.model_path=${MODEL_PATH}
worker.actor.global_batch_size=1440
worker.actor.micro_batch_size_per_device_for_update=4
worker.actor.micro_batch_size_per_device_for_experience=8
worker.rollout.n=16
trainer.experiment_name=${EXPERIMENT_NAME}
trainer.n_gpus_per_node=8
trainer.nnodes=6
trainer.save_checkpoint_path=${SAVE_PATH}
trainer.save_freq=25
trainer.total_epochs=5

[yfllllll]

@faedtodd can u share your implement code?

[2018hahazhufeng]

@faedtodd Could you please share more details? How many GPU hours were needed. Did your model reproduce the performance on TreeBench? Thanks.

[dearjohn12]

I also want to know the configs.