Computed Tomography (CT) is a challenging inverse problem that involves reconstructing images from projection data. The CT pipeline typically comprises three stages: 1) acquisition of projection images, 2) transposition of projection images into sinogram images, and 3) computation of reconstruction images. In practice, the projection images are often corrupted, resulting in various imaging artifacts such as noise, zinger artifacts, and ring artifacts in the reconstructed images. Although recent deep learning-based methods have shown promise in reducing noise through post-processing of CT images, they struggle to effectively address globally distributed artifacts along with noise.

Classical artifact reduction methods, on the other hand, have demonstrated success in reducing globally distributed artifacts by targeting individual types of artifacts before the reconstruction stage. These methods operate in the natural domain where the artifacts are most prominent. Inspired by that, we propose to reduce artifacts in all projection, sinogram, and reconstruction stages with deep learning. This approach enables accurate reduction of globally distributed artifacts along with noise, leading to improved CT image quality. Experiments on both simulated and real-world datasets validate the effectiveness of our proposed approach.

We present our recent developments in phase retrieval from propagation-based X-ray phase contrast images using deep learning-based approaches. Previously, deep convolutional neural networks had been used as post-processing step to a linear phase retrieval algorithm [1]. In a first approach, we investigated the use of deep convolutional neural networks to directly retrieve phase and amplitude from a propagation distance series of phase contrast images [2]. Due to a structure that seems well adapted to the properties of the phase contrast images, taking into account features at several scales and connecting the corresponding feature maps, we chose the mixed-scale dense network (MS-DN) [3] architecture as network structure. We developed a transfer learning approach where the network is trained on simulated phase contrast images generated from projections of random objects with simple geometric shape.

We showed that the use of a simple pre-processing to transform the input to the image domain improved results, providing some support to the hypothesis that including knowledge of the image formation process in the network improves reconstruction quality. Some work has been done in this direction using generative adversarial networks by introducing a model of the image formation in a CycleGAN network [4].

Going one step further, information on how to solve the phase retrieval problem can be introduced into the neural network, algorithm unrolling being one such approach [5]. In algorithm unrolling, parts of an iterative algorithm, usually the regularization part, are replaced by neural networks. The networks learn the steps of the chosen iterative algorithm. These networks can then be applied in a sequential fashion, making the run-time application very efficient, moving the calculation load from the iterative reconstruction to the off-line training of the networks.

Based on this idea, we proposed the Deep Gauss-Newton network (DGN) [6]. Gauss-Newton type algorithms have been successfully used for phase retrieval from Fresnel diffraction patterns [7]. Inspired by this, we developed an unrolling-type algorithm based on a Gauss-Newton iteration. Both the regularization and the inverse Hessian are replaced by neural networks. The same network is used for each iteration, making the method very economical in terms of network weights. An initial reconstruction is not required; the algorithm can be initialized at zero. It can retrieve simultaneously the phase and attenuation sorption from one single diffraction pattern. We applied the DGN to both simulated and experimental data, for which it substantially improved the reconstruction error and the resolution compared both to the standard iterative algorithm and the MSDN-based method.

Future work includes extension of the algorithms to tomographic and time-resolved imaging, as well as to other imaging problems. Code for both algorithms will be made available through the PyPhase package [8] in a future release.

[1] C. Bai, M. Zhou, J. Min, S. Dand, X. Yu, P. Zhang, T. Peng, B. Yao. Robust contrast-transfer-function phase retrieval via flexible deep learning networks, Opt. Lett. 44 (21), 5141–5144, 2019.

[2] K. Mom, B. Sixou, M. Langer. Mixed scale dense convolutional networks for x-ray phase contrast imaging, Appl. Opt. 61, 2497–2505, 2022.

[3] D. M. Pelt, J. A. Sethian, A mixed-scale dense convolutional neural network for image analysis, Proc. Natl.402 Acad. Sci. USA 115, 254–259, 2018.

[4] Y. Zhang, M. A. Noack, P. Vagovic, K. Fezzaa, F. Garcia-Moreno, T. Ritschel, P. Villanueva-Perez. PhaseGAN: a deep-learning phase-retrieval approach for unpaired datasets, Opt. Express 29, 19593–19604, 2021.

[5] V. Monga, Y. Li, Y. C. Eldar. Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing, IEEE Signal Proc. Mag. 38, 18–44, 2021.

[6] K. Mom, M. Langer, B. Sixou. Deep Gauss-Newton for phase retrieval, Opt. Lett. 48, 1136–1139, 2023.

[7] S. Maretzke, M. Bartels, M. Krenkel, T. Salditt, T. Hohage. Regularized newton methods for X-ray phase contrast and general imaging problems, Opt. Express 24, 6490–6506, 2016.

[8] M. Langer, Y. Zhang, D. Figueirinhas, J.-B. Forien, K. Mom, C. Mouton, R. Mokso, P. Villanueva-Perez. PyPhase – a Python package for X-ray phase imaging, J. Synch. Radiat. 28, 1261–1266, 2021.

A collimated X-ray beam is the trademark of synchrotron X-ray sources and comes with certain benefits for tomography, namely the simplicity of parallel beam tomographic reconstruction. Building on this a number of new approaches emerges to reconstruct a 3D volume from truncated X-ray projections. I will mainly consider here truncation in the time domain. One specificity of imaging at synchrotron instruments is that individual angular projections are acquired on a sub-ms time-frame and the entire tomographic dataset in a fraction of a second [1,2]. This enables time resolved studies of dynamic processes at the micrometer spatial and sub-second temporal resolution. Despite this fast acquisition the sample is often evolving at a faster rate, giving rise to motion artefact in the reconstructed volume. One possible approach to reconstruct an artifact-free 3D volume from (in the traditional sense) inconsistent projections is to use the concept of compressed sensing in the way that data in the temporal direction is represented by a linear combination of appropriate basis functions [3]. In our approach we perform L1 norm minimization for the gradient in both spatial and temporal variables. The optimal choice of basis functions is case specific and is the matter of further investigation.

Multiresolution acquisition is an attractive tomographic approach, but comes with it’s own challenges. I will discuss an approach to merge high and low resolution datasets of the same sample [4] for the extension of the reconstructed volume.

[1] R. Mokso, D.A. Schwyn, S.M. Walker et al. Four-dimensional in vivo X-ray microscopy with projection guided gating, Scientific Reports 5 (1), 8727, 2015.

[2] F. Garcia-Moreno et al. Using X-ray tomoscopy to explore the dynamics of foaming metal, Nature Communications. 10(1), 3762, 2019.

[3] V. Nikitin, M. Carlsson, F. Andersson, R. Mokso. Four-dimensional tomographic reconstruction by time domain decomposition, IEEE Transaction on Computational Imaging 5(3), 409, 2019.

[4] L. Varga, R. Mokso. Iterative High Resolution Tomography from Combined High-Low Resolution Sinogram Pairs, Proceedings of International Workshop on Combinatorial Image Analysis, 150–163, 2018.

X-rays can provide information about the structure of matter, on multiple length scales from bulk materials to nanoscale devices, from organs to organelle, from the organism to macromolecule. Due to the widespread lack of suitable lenses, the majority of investigations are rather indirect – apart from classical shadow radiography perhaps. While diffraction problems have been solved since long, the modern era has brought about lensless coherent imaging with X-rays, down to the nanoscale. How can we address and implement optimized tomography solutions for phase contrast inhouse and synchrotron data, taking into account partial coherence, propagation and cone beam geometry? We show how solutions and algorithms of mathematics of inverse problems [1-3] help us to meet the challenges of phase retrieval, tomographic reconstruction, and more generally image processing of bulky data. We also include illustrative bioimaging projects such as mapping the human brain [4,6] of fighting infectious diseases [6]. References:

[1] T. Salditt, A. Egner, R. D. Luke (Eds.) Nanoscale Photonic Imaging Springer Nature, TAP, 134, Open Access Book, 2020.

[2] L. M. Lohse, A.-L. Robisch, M. Töpperwien, S. Maretzke, M. Krenkel, J. Hagemann, T. Salditt A phase-retrieval toolbox for X-ray holography and tomography Journal of Synchrotron Radiation, 27, 3, 2020.

[3] S. Huhn, L.M. Lohse, J. Lucht, T. Salditt. Fast algorithms for nonlinear and constrained phase retrieval in near-field X-ray holography based on Tikhonov regularization - arXiv preprint arXiv:2205.01099, 2022.

[4] M. Eckermann, B. Schmitzer, F. van der Meer, J. Franz, O. Hansen, C. Stadelmann, T. Salditt. Three-dimensional virtual histology of the human hippocampus based on phase-contrast computed tomography Proc. Natl. Acad. Sci., 118, 48, e2113835118, 2021.

[5] M. Eckermann, J. Frohn, M. Reichardt, M. Osterhoff, M. Sprung, F. Westermeier, A.Tzankov, C. Werlein, M. Kuehnel, D. Jonigk, T. Salditt. 3d Virtual Patho-Histology of Lung Tissue from Covid-19 Patients based on Phase Contrast X-ray Tomography, eLife, 9:e60408, 2020.

[6] M. Reichardt, P.M. Jensen, V.A. Dahl, A.B. Dahl, M. Ackermann, H. Shah, F. Länger, C. Werlein, M.P. Kuehnel, D. Jonigk, T. Salditt. 3D virtual histopathology of cardiac tissue from Covid-19 patients based on phase-contrast X-ray tomography, eLife, 10:e71359, 2021.