The first is to develop federated learning approaches to exploit surgical data from different clinical centers and build new AI-methods to support surgeries in real time. The second is to deploy cross-site AI supported, robot assisted telesurgery, even across national borders.
A research team from Fraunhofer IPA is working with the Bosch Digital Innovation Hub at the Bosch Health Campus and the Institute of Image Guided Surgery (IHU) of Strasbourg to use data to help medical staff during a patient’s treatment or surgery. By analyzing data, optimizing workflows and using empirical values from previous surgeries, AI algorithms will be trained to support clinical staff in real time and independent of location. This ultimately aims to improve patient care.
Local knowledge as cross location training data for AI model
The use of medical data is usually limited to one site. Treatments are site dependent because there are hardly any possibilities for data exchange in the healthcare system. Another challenge is the variety of formats of the data such as images, texts, and videos. These make it difficult to recognize correlations and use them for the treatment of patients. The DAIOR project aims to change this. AI models, trained with the locally available data and data from other centers using distributed learning approaches, will be the enabling technology.
»The knowledge can thus be used elsewhere without sensitive medical data leaving the respective location,« says Johannes Horsch, one of the DAIOR scientists. In this way, distributed knowledge is bundled and made available regardless of location. »Through the methods of federated learning, training data from different locations can be used in DAIOR, even across national borders, while the data remains at the site, thus ensuring protection for patient data at the same time.«
Location independent operations with robot assisted telesurgery
The core of the collaboration between Fraunhofer IPA and IHU Strasbourg is based on the joint project »5G-OR« and the communication infrastructure that has already been successfully established and installed. The 5G infrastructure implemented will be used to realize remote surgery. An AI model for cross location robot assisted telesurgery, will compensate for delays in data communication on both sides by predicting the subsequent steps. »It works in a similar way to our brain,« explains Horsch. »Our brain is constantly calculating possible immediate future scenarios. The AI acts in exactly the same way.« This AI model will be able to predict the next steps and assist surgeons.
With this robot assisted telesurgery, in accordance with the scientists’ goal, it will soon be possible to perform surgeries via the internet, regardless of location. This is a milestone in the care of patients, especially in emergency medicine, where seconds often matter, for example in the case of a heart attack or stroke. Surgeries can be performed faster and patients can receive better medical care, for example, if surgeons do not have to change locations for a surgery. This, in turn, should be noticeable for the patients, as clinical staff will have more time available for treatment elsewhere.
Long term Franco-German cooperation
DAIOR is already the second long term cooperation between the Fraunhofer IPA and the IHU Strasbourg and falls into a plethora of projects between the three partners. Fraunhofer IPA and IHU were, e.g. already able to realize the »5G-OR« project together, as mentioned. The DAIOR project strengthens the Franco-German cooperation once again and enables an international technology and knowledge transfer.
IHU Strasbourg is a multidisciplinary institute dedicated to innovative image guided therapies for patient care. Its research and development activities focus on minimally invasive precision interventions enhanced by virtual reality technologies, robotics and artificial intelligence, and novel »patient journeys« combining accelerated diagnosis, outpatient surgery and improved post operative rehabilitation.
With its focus on medical robotics, the Fraunhofer IPA sees AI as an opportunity to provide surgeons in particular with increasing support through intelligent assistance systems and as a way to automate substeps of surgeries. Telesurgery was identified as an important milestone here, as it already includes the infrastructure necessary for automation and offers the possibility of collecting large amounts of data from the interventions.
The Bosch Digital Innovation Hub formerly known as KTBW is an agile innovation and implementation unit at the Bosch Health Campus (Stuttgart) with close, long term and successful cooperations with the two aforementioned institutions and long-standing networks in the Mannheim and Strasbourg ecosystems. It aims to increase the implementation rate of research projects, especially in the field of digital innovations and AI applications in healthcare, and acts as a catalyst for AI and digital healthcare innovations and innovative care concepts.
