During surgery, doctors must identify vital anatomical structures (nerves and lymph nodes) to prevent unnecessary healthy tissue damage. Correct intraoperative identification by normal eyesight remains enormously challenging due to natural variation. Therefore, surgeons require high-tech intra-operative imaging for reliable high-resolution visual discrimination of critical anatomical structures.
The H3D-VISIOnAiR project, under the ATTRACT umbrella, is a breakthrough innovation based on infrared imaging and the recognition of tissues in the human body. The idea is to make them visible in real-time and on 3D images as an Augmented Reality (AR) overlay for the surgeon. It consists of two multi-spectral cameras, combining visual range with NIR (Near Infrared) visualization, a belt computer with real-time data processing, and a high-end stereoscopic head-mounted display with wireless connection to the operating room infrastructure.
Get to know more about it through this interview with Jaap Heukelom, coordinator of the H3D-VISIOnAiR project, and Gabrielle Tuijthof, professor at the University of Twente, who explain what this project is about.
Can you describe which are your roles at the H3D-VISIOnAiR project?
My name is Jaap Heukelom, I’m the coordinator of the H3D-VISIOnAiR project and I’m also the co-founder of i-Med Technology based in the Netherlands.
My name is Gabrielle Tuijthof, a professor at the University of Twente and I have two roles: I facilitate the bridge between the hardcore engineers and the clinicians, and I also coach the students who are involved in this project and take care of the dissemination.
What is the project about? And which partners are involved?
The project is a breakthrough very innovative system, a head-worn 3D microscope with two hyperspectral cameras that can make invisible tissues of the human body visible in real-time for the surgeon. The partners involved in the project are i-Med Technology; Quest Medical Imaging, which has skills in hyperspectral imaging. Also, the Amsterdam University Medical Centre with a lot of knowledge on high perspective imaging and algorithms; the Maastricht University Medical Centre is involved, they also have a lot of knowledge on recognizing tissues. The Twente University is involved, with a lot of knowledge about AI and machine learning.
What challenges have you faced so far?
The challenges we faced so far are finding the right camera that can do the job: the hyperspectral camera, which we solved. Secondly, is very important to have the right lighting conditions, especially in the infrared area. We have overcome that, and now it’s important to get sufficient data sets of live human tissues.
How would you explain the potential implications of this breakthrough to a non-scientific audience?
Imagine that you would need surgery and I think many of you have maybe seen one on television and you see that it’s pretty bloody. The surgeon needs to identify and see which structures are critical, for instance, the nerve. We developed a new 3D camera that allows the surgeon to get superhero eyes, which means the surgeon can see more than the normal eyes can see. This way we improve the safety of all the operations.
For more information
Visit the H3D-VISIOnAiR project site.
