Conference Agenda

Statistical depth functions provide center-outward orderings in spaces of dimension larger than one, where a natural ordering does not exist. The computation of such depth functions can be computationally prohibitive, even for relative low dimensions. We present a novel sequential Monte Carlo methodology for the computation of depth functions and related quantities (seMCD), that outputs an interval, a so-called bucket, to which the quantity of interest belongs with a high probability prespecified by the user. For specific classes of depth functions, we adapt algorithms from sequential testing, providing finite-sample guarantees. For depth functions dependent on unknown distributions, we offer asymptotic guarantees using nonparametric statistical methods. In contrast to plain-vanilla Monte-Carlo-methodology the number of samples required in the algorithm is random but typically much smaller than standard choices suggested in the literature. The seMCD method can be applied to various depth functions, including the simplicial depth and the integrated rank-weighted depth, and covers multivariate as well as functional spaces. We demonstrate the efficiency and reliability of our approach through empirical studies, highlighting its applicability in outlier detection, classification, and depth region computation. In conclusion, the seMCD algorithm can achieve accurate depth approximations with fewer Monte Carlo samples while maintaining rigorous statistical guarantees.

Entropic optimal transport (EOT) presents an effective and computationally viable alternative to unregularized optimal transport (OT), offering diverse applications for large-scale data analysis. We derive novel statistical bounds for empirical plug-in estimators of the EOT cost and show that their statistical performance in the entropy regularization parameter $\epsilon$ and the sample size $n$ only depends on the simpler of the two probability measures. For instance, under sufficiently smooth costs this yields the parametric rate $n^{-1/2}$ with factor $\epsilon^{-d/2}$, where $d$ is the minimum dimension of the two population measures. This confirms that empirical EOT also adheres to the lower complexity adaptation principle, a hallmark feature only recently identified for unregularized OT. In particular, this suggests that the estimation of the EOT cost is only affected by the curse of dimensionality when both measures have a high intrinsic dimension. Our technique employs empirical process theory and relies on a dual formulation of EOT over a single function class. Central to our analysis is the observation that the entropic cost-transformation of a function class does not increase its uniform metric entropy by much.

In frequentist approaches to statistical learning, training involves finding optimal parameter values through techniques such as maximum likelihood estimation. In an end-to-end Bayesian approach, however, the training step is replaced by a sampling problem. One advantage of such a Bayesian framework is that parameter uncertainty is fully incorporated into the posterior predictive distribution (PPD). Another advantage is that marginalising parameters entirely yields the marginal likelihood, or model evidence, which is arguably the best measure of model likelihood, offering strong protection against overfitting. Unfortunately, computational feasibility rapidly deteriorates as model complexity increases, limiting the expressivity of applicable Bayesian models.

As a step forward, we present a novel algorithm that simultaneously estimates both posterior averages (e.g. the PPD) and the model evidence. Furthermore, we propose leveraging this simultaneous estimation in the form of evidence-weighted model averages. This approach allows candidate models to remain relatively simple, with the complexity managed through the PPDs and the model ensemble.

The presented algorithm is a non-equilibrium version of the traditional thermodynamic integration method for evidence estimation, adapting free-energy estimators from non-equilibrium thermodynamics of microscopic systems for use in the Bayesian context. The resulting non-equilibrium integration (NEQI) is formulated as the exponential average of an observable from a customised stochastic process. The non-equilibrium flavour of thermodynamic integration offers several advantages: it allows for combining arbitrarily many short trajectories to thoroughly explore the parameter space, and it eliminates the need for burn-in, thinning, or stationarity. We present the details of the estimator and its implementation in TensorFlow Probability, demonstrating through relevant examples how NEQI competes with state-of-the-art methods such as annealed importance sampling for evidence estimation and Hamiltonian Monte Carlo for posterior inference.