We will review recent progress and open problems in our understanding of the statistical and computational performance of Bayesian inference algorithms in non-linear inverse regression problems arising with partial differential equation models.