Bi-Lipschitz Recurrent Equilibrium Network (BiLipREN)

📄 arXiv:2607.10026: Robustly Invertible Nonlinear Dynamics and the BiLipREN: From Inversion-Based Control to Generative Trajectory Modelling

TL;DR

BiLipREN is a neural dynamical system that defines a robustly invertible signal-to-signal mapping.

The REN architecture is a feedback interconnection between a learnable LTI system and a fixed nonlinear activation .

The following properties are guaranteed by construction (plug-and-play with AutoDiff and SGD):

1. The forward model is an invertible, stable and bi-Lipschitz REN.

2. Its analytical inverse is a causal, stable and bi-Lipschitz REN.

3. Both models enable robust signal reconstruction under disturbances and initial-state mismatch:

Applications

1. Optimization-Aware Dynamic Surrogate Loss

TD;LR: Learn an optimization-friendly surrogate loss for black-box trajectory optimization

Black-box Trajectory Optimization. Suppose that are unknown, and only a dataset is available:

Can we find a new input sequence that is likely to achieve a lower cost than any sample in the dataset?

• Surrogate optimization framework:

1. Fit a differentiable surrogate loss to the dataset:

where is a neural dynamical model that captures temporal structure and is a learnable parameter.

2. Optimize the surrogate loss:

• Our approach: parameterize as a BiLipREN, giving the surrogate two nice properties:

1. It satisfies the Polyak–Łojasiewicz (PL) condition. Consequently, despite being nonconvex, it has no spurious local minima, and gradient-based methods converge linearly under standard step-size conditions.
2. The minimizer can be computed efficiently through dynamic inversion:

• Results:

Model	Fitting loss $L$	Best cost $J$	Worst cost $J$
Dataset	-	1863	5055
LSTM	1718	1868	4758
C-REN	6014	1918	2996
BiLipREN	22805	1672	-
IPOPT	-	1618	5837

1. The LSTM fits the dataset well but is less suitable for the subsequent optimization step because its loss landscape may contain spurious local minima, flat regions, poorly conditioned gradients, and strong sensitivity to initialization.

2. The C-REN, which is stable but not necessarily invertible, encounters similar difficulties.

3. The BiLipREN has a higher fitting loss but yields a lower optimized cost and a more tractable optimization landscape.

4. Even when IPOPT is applied to the true optimization problem, poor initial guesses can produce poor solutions because the problem is highly nonconvex.

2. Signal-to-Signal Nomralizing Flow

TL;DR: Learn a signal-2-signal normalizing flow that generates trajectory distributions from Gaussian white noise

Generative trajectory modelling. We seek a robustly invertible dynamical model that generates samples matching the data distribution:

The model is trained by minimizing the negative log-likelihood (NLL) under the normalizing-flow change-of-variables formula:

• Results:

1. The generated trajectories capture the multimodal, obstacle-avoiding distribution of the training data.

2. Mapping the data through produces approximately white Gaussian latent variables: their autocorrelations remain within the 95% confidence band, and their Q–Q plot closely follows that of a standard Gaussian distribution.

3. Inversion-based Control Design

TL;DR: Design a tracking controller for a stable, nonminimum-phase plant.

Internal model control (IMC). We learn an inner–outer factorization of the plant and invert only its minimum-phase outer factor, thereby obtaining a stable controller.

1. Learn an inner–outer factorization from input–output data generated by the true system :

where the inner factor is an all-pass filter (stable but non-minimum-phase system) and the outer factor is a BiLipREN (stable minimum-phase system).

2. Construct the IMC controller in the Youla form with -parameter

where is a low-pass filter and is an approximate inverse of . For piecewise-constant inputs, converges exponentially to zero.

• Results. The controller achieves reference tracking for a four-tank system with delayed input flow.

Get started

git clone https://github.com/acfr/BiLipREN.git
cd BiLipREN
python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install -r requirements.txt

Repository layout

Folder	Description
BiLipRENs/	Core: BiLipREN models and orthogonal layers
surrogate_cost/	Application 1 — dynamic surrogate loss learning.
flow/	Application 2 — signal-to-signal normalizing flow.
imc/	Application 3 — inversion-based control design.
io_fact/	Example: nonlinear I/O factorization.
robust_inv/	Example: robust inversion

Contacts

Yurui Zhang (yurui.zhang@sydney.edu.au)

Ruigang Wang (ruigang.wang@sydney.edu.au)