Causal Motion Representations

Paper | Video | Adaptive Y-net

This is an official implementation for the paper

Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Yuejiang Liu, Riccardo Cadei, Jonas Schweizer, Sherwin Bahmani, Alexandre Alahi
École Polytechnique Fédérale de Lausanne (EPFL)

TL;DR: incorporate causal invariance and structure into the design and training of motion forecasting models to improve the robustness and reusability of the learned representations under common distribution shifts

• causal formalism of motion forecasting with three groups of latent variables
• causal (invariant) representations to suppress spurious features and promote robust generalization
• causal (modular) structure to approximate a sparse causal graph and facilitate efficient adaptation

Spurious Shifts

In the context of motion forecasting, the target trajectory is often correlated with some spurious features, such as observation noises and agent densities. Yet, such correlations are brittle and lead to poor robustness under spurious shifts. We simulate this effect in a controlled setting and demonstrate that models trained to perform equally well across training environments tend to suppress spurious features and generalize better.

Please check out the code in the spurious folder.

Style Shifts

One unique property of motion problems is that behavioral styles may naturally vary from one environment to another. To explicitly model this, we design a modular architecture that factorizes the representations of invariant features and style confounders. We train the style encoder with a contrastive loss, which allows for effective use of the structured knowledge during both training and deployment.

Please check out the code in the style folder.

Citation

@InProceedings{Liu2022CausalMotionRepresentations,
    title     = {Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective},
    author    = {Liu, Yuejiang and Cadei, Riccardo and Schweizer, Jonas and Bahmani, Sherwin and Alahi, Alexandre},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2022},
    pages     = {17081-17092}
}

Developers

Our code is mainly developed by Riccardo Cadei and Jonas Schweizer.

Acknowledgements

Our code is built upon the public code of the following papers:

• Human Trajectory Prediction via Counterfactual Analysis, ICCV'21
• It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction, ECCV'20
• STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction, ICCV'19