Question about collision metric

[InhwanBae]

Hi @YuejiangLIU

Thank you so much for this great work!

I have a question about the collision ratio results. ECCV2022 version of Trajectron++ has an issue that future two GT coordinates were additionally used while pre-processing the observed trajectories (issues #26 #40 #53). Therefore, when forecasting the trajectory, the first two frames can be predicted almost accurately, and I believe that two subsequent coordinates can also be estimated accurately using this additional information. Note that the solution to these issues has been updated on Trajectron++ GitHub.

As mentioned in the paper, the evaluation of collision rate is focused on the first four prediction steps. If so, my concern is that this issue of Trajectron++ affects the collision results in the paper.

Thank you!

[YuejiangLIU]

Hi @InhwanBae,

Thank you for your comments! This is a really great point!

In fact, I did notice in our trajectron++ evaluations last year that there were almost no collisions at all in the first two predicted steps. As a result, the collision rate of the trajectron++ was globally much lower than other base models. I previously thought it was simply because of the design the trajectron++ being better. But it's probably more related to the information leakage in their pre-processing, as you mentioned.

Unfortunately, I was not aware this underlying issue. Our code was actually developed and released before the commit that you shared.

The main goal of our experiment here is to compare forecasting models (e.g. trajectron++) trained with and without our method (social contrastive loss + negative data augmentation). In this regard, I guess the issue in the trajectron++ baseline does not affect our final conclusion much, but the collision numbers would have been larger globally if without the information leakage.

Thanks,
Yuejiang