Kernel Methods in Generative Modeling

We consider gradient flows with respect to the maximum mean discrepancy (MMD) of certain kernels. For the efficient computation, we propose slicing techniques that restrict the kernel evaluations to the one-dimensional setting, where they can be performed depending on the kernel, either by sorting or by fast non-equispaced Fourier transforms. This enables us to simulate MMD particle flows in high dimensions for a large number of particles. We approximate these particle flows by neural networks and apply them for generative modeling and posterior sampling in Bayesian inverse problems.

From a more theoretical viewpoint, we study Wasserstein gradient flows with respect to our MMD functionals. Interestingly, particles might "explode" for non-differentiable kernels as the negative diastance kernel, i.e., the flow turns atomic measures into absolutely continuous ones and vice versa. We analytically derive the Wasserstein flows for some special cases and propose a numerical approximation of suitable forward and backward time discretizations by generative neural networks.

References:
Wasserstein steepest descent flows of discrepancies with Riesz kernels. J Hertrich, M Gräf, R Beinert, G Steidl, Journal of Mathematical Analysis and Applications 531 (1), 127829.

Posterior sampling based on gradient flows of the MMD with negative distance kernel. P Hagemann, J Hertrich, F Altekrüger, R Beinert, J Chemseddine, G Steidl, ICLR 2024.

Conditional Wasserstein Distances with Applications in Bayesian OT Flow Matching. J Chemseddine, P Hagemann, C Wald, G Steidl, arXiv preprint arXiv:2403.18705.

Sliced optimal transport on the sphere. M Quellmalz, R Beinert, G Steidl, Inverse Problems 39 (10), 105005, 2023.