\[ \renewcommand{\d}{{\bf{d}}} \renewcommand{\b}{{\bf{b}}} \newcommand{\J}{{\bf{J}}} \newcommand{\A}{\bf{A}} \newcommand{\B}{\bf{B}} \newcommand{\RR}{\mathbf{R}} \newcommand{\h}{{\bf{h}}} \newcommand{\x}{{\bf{x}}} \newcommand{\bfa}{{\bf{a}}} \newcommand{\bfb}{{\bf{b}}} \newcommand{\bfc}{{\bf{c}}} \newcommand{\y}{{\bf{y}}} \newcommand{\z}{{\bf{z}}} \newcommand{\w}{{\bf{w}}} \newcommand{\f}{{\bf{f}}} \newcommand{\tf}{{\bf{\tilde f}}} \newcommand{\tx}{{\bf{\tilde x}}} \renewcommand{\d}{{\rm{d}}} \newcommand{\s}{{\bf{s}}} \newcommand{\g}{{\bf{g}}} \newcommand{\W}{{\bf{W}}} \newcommand{\vol}{{\operatorname{vol}}} \newcommand{\zz}{\mathbf{z}} \newcommand{\xx}{\mathbf{x}} \newcommand{\bdelta}{\bm{\delta}} \renewcommand{\H}{\mathbf{H}} \newcommand{\txx}{{\tilde{\mathbf{x}}}} \newcommand{\tzz}{{\tilde{\mathbf{z}}}} \newcommand{\tyy}{{\tilde{\mathbf{y}}}} \newcommand{\invf}{f^{-1}} \newcommand{\Sp}{\mathbb{S}} \]

Self-supervised contrastive learning performs non-linear system identification

Rodrigo Gonzalez Laiz*
Tobias Schmidt*
Steffen Schneider
Institute of Computational Biology ,
Computational Health Center, Helmholtz Munich

ArXiv

Paper

Code

tl;dr: Dynamics Contrastive Learning (DCL) is a framework to uncover linear, switching linear and non-linear dynamics under a non-linear observation model using contrastive learning.

News

May '25 The camera ready version of the paper is available on arXiv & OpenReview.
January '25 Our paper was accepted at ICLR 2025. See you in Singapore!
October '24 Our preprint is now available on arXiv!

Abstract

Self-supervised learning (SSL) approaches have brought tremendous success across many tasks and domains. It has been argued that these successes can be attributed to a link between SSL and identifiable representation learning: Temporal structure and auxiliary variables ensure that latent representations are related to the true underlying generative factors of the data. Here, we deepen this connection and show that SSL can perform system identification in latent space. We propose dynamics contrastive learning, a framework to uncover linear, switching linear and non-linear dynamics under a non-linear observation model, give theoretical guarantees and validate them empirically.

Overview of dynamics contrastive learning

Reference

@inproceedings{
    laiz2025selfsupervised,
    title={Self-supervised contrastive learning performs non-linear system identification},
    author={Rodrigo Gonz{\'a}lez Laiz and Tobias Schmidt and Steffen Schneider},
    booktitle={The Thirteenth International Conference on Learning Representations},
    year={2025},
    url={https://openreview.net/forum?id=ONfWFluZBI}
}
Webpage designed using Bootstrap 4.5.