Minimally invasive surgical techniques, also known as "keyhole surgery", are increasingly replacing conventional open operations - also in cancer care. For example, certain tumors of the internal organs can be removed in such laparoscopic procedures. This involves the use of an instrument known as a laparoscope, which is inserted into the abdomen through a small incision in the skin and then allows the physician to view the affected organs.
The video camera connected to the laparoscope delivers an image signal consisting of the three color channels red, green and blue (RGB). This color space is also used, for example, in televisions or smartphones. However, it has been shown that human tissue has optical properties that cannot be represented with this image information alone. In the scope of the European Research Council (ERC) project NEURAL spicing and in collaboration with the Städtisches Klinikum Karlsruhe, Lena Maier-Hein (DKFZ) and her team therefore investigate the use of multispectral imaging in this context. These multispectral camera systems not only capture the three wavelength ranges of the optical spectrum mentioned above, but provide a multispectral measurement covering over 16 wavelength ranges. This makes it possible to visualize certain functional properties of the tissue that are invisible to conventional camera systems, such as the blood perfusion status of an organ.
This is relevant, for example, in patients with kidney cancer when the tumor has to be removed surgically. During this intervention, the blood flow to the affected areas must be interrupted by clamping the arteries. Only relying on the conventional laparoscopic RGB video makes it virtually impossible to assess the state of tissue perfusion. Only by using a fluorescent contrast agent, which is injected into the patient and subsequently accumulates in the perfused tissues, the physician is able to see whether the correct segments of the organ have been cut off from the blood supply. If this was not successful, the contrast medium has to be washed out for about 30 minutes so that the procedure can then be repeated.
"We were able to show that blood perfusion in the kidney can be visualized without contrast agents and in real time using a combination of multispectral imaging and artificial intelligence," says Lena Meier-Hein. The new technology has now been successfully used in a study in ten patients who had to undergo partial nephrectomy. "Because our approach allows the blood perfusion status of the tissue to be monitored live, the clamping of the blood vessels can be corrected immediately if necessary. This reduces the duration of the procedure, making it safer for patients," says Leonardo Ayala, lead author of the study. The new technology also eliminates the risk of an allergic reaction to the contrast agent.
Two aspects played a decisive role in the acquisition and processing of the image data. The use of multispectral imaging systems in minimally invasive surgery has failed so far, among other things, due to the time required by the systems to acquire and process the data. If several seconds are needed to calculate a new image, the information is practically unusable during the procedure. By using novel multispectral camera technology, the researchers have now been able to solve this problem. Their newly developed system delivers 25 frames per second and thus a fluid video image. On the other hand, the perfusion estimation for the first time could be accomplished using AI methods that do not rely on large amounts of training data. The special feature here is that the AI learns to distinguish perfused from non-perfused tissue for each patient anew. "This is necessary because the tissue is not the same in every patient, so the optical properties can also be very different," explains Leonardo Ayala. Therefore, at the beginning of a surgery, a short video sequence of the kidney is recorded for each patient with the help of a laparoscope. "The algorithm then calculates, based on the patient's individual image data, how the state of blood flow affects the optical properties of the tissue, so that the degree of blood flow can be displayed during surgery," Tim Adler, second lead author complements.
Comparable methods could also be used for other surgical issues in the future. Dogu Teber, clinical lead of the project, sees great potential for this new technological approach: "Spectral imaging in combination with new AI-based analysis tools could become an important tool for fast, reliable and safe functional imaging in minimally invasive surgery."
