Object detection performance for Short-term anticipation

[takfate]

We evaluate the official object detection results with the 'slowfast' model and get 'Box + Noun mAP: 17.55'.
When we train the object detector from scratch, the validated mAP result is about 2.3.

[antoninofurnari]

Hello, thank you for your interest in the short-term anticipation challenge!

Have you followed all the steps reported here? https://github.com/EGO4D/forecasting/blob/main/SHORT_TERM_ANTICIPATION.md

Are you evaluating the model using the provided evaluation code? https://github.com/EGO4D/forecasting/blob/main/ego4d/evaluation/sta_metrics.py

The number reported above (17.55) is Top-5 mAP (see paper for details). Maybe the number you obtained (2.3) is standard mAP?

[takfate]

Thanks for your response. I will check this issue.

[sanketsans]

Hi @antoninofurnari ,
The object detections downloaded from here : https://github.com/EGO4D/forecasting/blob/main/SHORT_TERM_ANTICIPATION.md have many IDs unavailable for v2 of EGO4D. Hence, I used the object detector (provided with pre-trained weights) to extract the object detections for all video IDs.
But the values for Top-5mAP Box + Noun is only 16.7 (v2) using those extracted object detections.

Can you help on how can I download the updated object_detections.json for v2 of EGO4D ?

Thanks.

Best,
Sanket