Questions about pretrained weights

[bbhbbxwy]

Hi, I want to train the DAVE model from scratch. Following the LOCA code for 200 epochs, the best counting results I obtained are: Val set MAE: 10.30 RMSE: 34.62, Test set MAE: 12.22 RMSE: 83.12. These results differ significantly from those of the pretrained weights you provided (Val MAE: 10.37, Val RMSE: 30.72, Test MAE: 10.35, Test RMSE: 51.80). Have you successfully replicated the LOCA model?

[jerpelhan]

Yes, this is quite common in the few-shot counting field and can be observed across many state-of-the-art methods. While results on the validation set tend to be relatively stable, the test set often shows large variability and noise, particularly in RMSE. As a result, it's typically necessary to run training multiple times to achieve optimal results.

I was able to replicate the LOCA results, but only after several runs. In my experience, the issue seems to be less about the method itself and more related to the dataset—especially the noisy annotations. We've recently found that a significant portion of the RMSE error comes from just a single image in the test set.

[bbhbbxwy]

Yes, this is quite common in the few-shot counting field and can be observed across many state-of-the-art methods. While results on the validation set tend to be relatively stable, the test set often shows large variability and noise, particularly in RMSE. As a result, it's typically necessary to run training multiple times to achieve optimal results.

I was able to replicate the LOCA results, but only after several runs. In my experience, the issue seems to be less about the method itself and more related to the dataset—especially the noisy annotations. We've recently found that a significant portion of the RMSE error comes from just a single image in the test set.

Thanks. Have your hyperparameters and training strategy matched the author's settings? Or did you make any changes?