close
close

The next step in light microscopy image enhancement

The next step in light microscopy image enhancement

It is the computational processing of images that reveals the finest details of a sample placed under all kinds of different light microscopes. Although this processing has come a long way, there is still room to increase image contrast and resolution, for example. A new computational model based on a unique deep learning architecture, developed by researchers from the Center for Advanced Systems Understanding (CASUS) at the Helmholtz-Zentrum Dresden-Rossendorf (HZDR) and the Max Delbrück Center for Molecular Medicine, is faster than traditional models when matching. it even exceeds the quality of its visuals. The model, called Multiphase Residue-BCR Net (m-rBCR), was developed specifically for microscopy images. The corresponding peer-reviewed conference paper, first presented at the biennial European Conference on Computer Vision (ECCV), the leading event in computer vision and machine learning, is now available.

The new model brings a new breath to the image processing technique called deconvolution. This computationally intensive method increases the contrast and resolution of digital images captured on optical microscopes such as wide-field, confocal, or transmission microscopes. Deconvolution aims to reduce blur, a specific type of image distortion caused by the microscopic system used. The two main strategies are explicit deconvolution and deep learning-based deconvolution.

Explicit deconvolution approaches are based on the concept of point spread function (PSF). A PSF basically describes how the infinitesimal point light source emerging from the sample is expanded by the optical system and spread into a three-dimensional diffraction pattern. This means that in a recorded (two-dimensional) image there is always some light from out-of-focus structures that causes blurring. By knowing the PSF of a microscopic system, you can calculate the blur to obtain a much better lifelike image than the raw recorded image.

SPY Young Researcher Group Leader and corresponding author of the ECCV article, Dr. “The biggest problem with PSF-based deconvolution techniques is that the PSF of a given microscopic system is often not available or imprecise,” says Artur Yakimovich. “For decades people have been working on what is called blind deconvolution, where the PSF is estimated from the image or set of images. But blind deconvolution is still a very challenging problem and the progress achieved has been modest.”

As Yakimovich’s team has shown in the past, using the “inverse problem-solving” toolbox works well in microscopy. Inverse problems concern recovering causal factors that lead to particular observations. Usually you need a lot of data and deep learning algorithms to successfully solve such problems. As with explicit deconvolution methods, the results are higher resolution or better quality images. For the approach presented in ECCV, scientists used a physics-informed neural network called the Multistage Residual-BCR Network (m-rBCR).

Deep learning implemented differently

In general, there are two basic types of image processing. It can start with a classical spatial representation of an image or with a frequency representation (requiring a conversion step from the spatial representation). In the latter, each image is represented as a collection of waves. Both representations are valuable. Some machining operations are easier to perform in one form and some in the other. The vast majority of deep learning architectures operate in the spatial domain. It is very suitable for photos. However, microscopy images are different. They are mostly monochrome. Techniques such as fluorescence microscopy deal with specific light sources on a black background. So m-rBCR uses the frequency representation as the starting point.

First author Rui Li explains: “Using the frequency domain in such cases can help create optically meaningful data representations – a concept that allows m-rBCR to solve the deconvolution task with surprisingly few parameters compared to other modern deep learning architectures. ” and hosts ECCV. Li proposed developing the neural network architecture of a model called BCR-Net, inspired by the frequency representation-based signal compression scheme introduced by Gregory Beylkin, Ronald Coifman, and Vladimir Rokhlin in the 1990s (explaining the name of BCR-Net). transformation).

The team validated the m-rBCR model on four different datasets, two simulated microscopy image datasets and two real microscopy datasets. It exhibits high performance with significantly fewer training parameters and shorter running time compared to state-of-the-art deep learning-based models, and of course outperforms explicit deconvolution methods.

A model adapted to microscopy images

Co-author Prof. “This new architecture exploits a neglected pathway for learning representations beyond classical convolutional neural network approaches,” summarizes Misha Kudryashev.In place “Structural Biology” group of the Max-Delbrück-Centrum für Moleculare Medizin in Berlin. “Our model significantly reduces potentially unnecessary parameters. As the results show, this is not accompanied by performance loss. “The model is clearly suitable for microscopy images and, as it has a lightweight architecture, it defies the trend of larger models requiring more computing power.”

The Yakimovich group recently published an image quality enhancement model based on generative artificial intelligence. This Conditional Variational Diffusion Model produces state-of-the-art results that exceed the m-rBCR model presented here. “But you need training data and computational resources, including adequate graphical processing units, which are highly sought after these days,” Yakimovich recalls. “The lightweight m-rBCR model does not have these limitations and still produces very good results. Therefore, I am confident that we will see good interest in the imaging community. To facilitate this, we have begun to improve user-friendliness.”

Yakimovich’s “Machine Learning for Infection and Disease” group aims to understand the complex network of molecular interactions that are active after the body is infected with a pathogen. Using the new possibilities of machine learning is very important here. His research interests include improving image resolution, 3D image reconstruction, automatic disease diagnosis, and evaluation of image reconstruction quality.