Conference Agenda

Slice sampling is a Markov chain Monte Carlo (MCMC) method for drawing (approximately) random samples from a posterior distribution that is typically only known up to a normalizing constant. The method is based on sampling a new state on a slice, i.e., a level set of the target density function. Slice sampling is especially interesting because it is tuning-free and guarantees a move to a new state, which can result in a lower autocorrelation compared to other MCMC methods.

However, finding such a new state can be computationally expensive due to frequent evaluations of the target density, especially in high-dimensional settings. To mitigate these costs, we introduce a delayed acceptance mechanism that incorporates an approximate target density for finding potential new states. We will demonstrate the effectiveness of our method through various numerical experiments and outline a possible extension of our two-level method into a multilevel framework.

In recent years, various interacting particle samplers have been developed to sample from complex target distributions, such as those found in Bayesian inference. These samplers are motivated by the mean-field limit perspective and implemented as ensembles of particles that move in the product state space according to coupled stochastic differential equations. The ensemble approximation and numerical time stepping used to simulate these systems can introduce bias and affect the invariance of the particle system with respect to the target distribution. To correct for this, we investigate the use of a Metropolization step, similar to the Metropolis-adjusted Langevin algorithm. We examine both ensemble- and particle-wise Metropolization and prove basic convergence of the resulting ensemble Markov chain to the target distribution. Our results demonstrate the benefits of this correction in numerical examples for popular interacting particle samplers such as affine invariant interacting Langevin dynamics, consensus-based sampling, and stochastic Stein variational gradient descent.

We consider the framework of penalized estimation where the penalty term is given by a polyhedral norm, or more generally, a polyhedral gauge, which encompasses methods such as LASSO and generalized LASSO, SLOPE, OSCAR, PACS and others. Each of these estimators can uncover a different structure or “pattern” of the unknown parameter vector. We define a novel and general notion of patterns based on subdifferentials and formalize an approach to measure pattern complexity. For pattern recovery, we provide a minimal condition for a particular pattern to be detected with positive probability, the so-called accessibility condition. We make the connection to estimation uniqueness by showing that uniqueness holds if and only if no pattern with complexity exceeding the rank of the $X$-matrix is accessible. Subsequently, we introduce the noiseless recovery condition which is a stronger requirement than accessibility and which can be shown to play exactly the same role as the well-known irrepresentability condition for the LASSO – in that the probability of pattern recovery is bounded by 1/2 if the condition is not satisfied. Through this, we unify and extend the irrepresentability condition to a broad class of penalized estimators using an interpretable criterion. We also look at the “gap” between accessibility and the noiseless recovery condition and discuss that our criteria show that it is more pronounced for simple patterns. Finally, we prove that the noiseless recovery condition can indeed be relaxed when turning to so-called thresholded penalized estimation: in this setting, the accessibility condition is already sufficient (and necessary) for sure pattern recovery provided that the signal of the pattern is large enough. We demonstrate how our findings can be interpreted through a geometrical lens throughout the talk and illustrate our results for the Lasso as well as other estimation procedures. [See also arXiv:2307.10158]

In Bayesian inference, approximating the posterior distribution accurately is essential for deriving meaningful probabilistic insights from data. The Laplace approximation is a widely used method to approximate posterior distributions by fitting a Gaussian distribution centered at the mode. However, as a Gaussian approximation, it often fails when the posterior distribution has multiple modes or exhibits non-Gaussian characteristics. To address these limitations, we introduce a method that utilizes a linear combination of multiple Laplace approximations. We demonstrate that this method remains robust as the number of observations increases or observation noise decreases. This improved approximation technique can be effectively integrated into sampling methods for integral calculations. We specifically implemented our approach in the context of importance sampling and analyzed the convergence of the effective sample size. The robustness and efficiency of our method are illustrated through two examples, showing its potential to enhance posterior approximation in practical scenarios.