Virtual Reality: HoloCare’s AI-generated holograms for surgical decision making

Editorial Content Producer Oli Johnson speaks to Alison Sundset, Chief Executive Officer at HoloCare, about the company’s interactive 3D holograms of organs for surgical decision making.

HoloCare, a Norwegian medtech company that provides AI generated holograms for pre-operative planning, is working with European leaders in technology and healthcare research as part of the project ‘HoloSurge’, a four-year collaboration with a shared goal of reducing the risk of complications during planned surgery. The project received a grant worth 8.9 million EUR in early 2024, allocated by transnational research and innovation funding body Horizon Europe, to further the development of organ hologram technology to power informed surgical decision making.

HoloCare says the technology is set to improve both the surgical process and the decision-making beforehand, potentially reducing complications. This includes issues such as infection, bleeding, organ damage and death, which currently affect up to 48% of liver surgeries, and 60% of pancreatic surgeries according to the company.

“We created the whole of HoloCare Studio during the pandemic, it is made by clinicians for clinicians.”

Speaking to MTI, Alison Sundset, CEO at HoloCare explained how the company’s technology works: “What we’re doing is we are taking a normal CT image, so in this case a liver, which is our first application, then we put it through our HoloCare studio and we make a 3D model of it using AI. With current care, it takes about a week to send to another company or if you do it yourself a whole day. We’ve taken it from that to two minutes using AI. We create the 3D model, then it becomes a 3D image which is pushed up into a Microsoft HoloLens, but you can use an Apple Vision Pro, or it can be a 2D image on a 3D screen.”

The technology allows surgeons to explore a patient-specific scan of an organ and plan out precisely what needs to happen in the operation on that patient. Another benefit according to the company is that the scans can be shared in real time with people across the world. A surgeon in one area of a country could jump on a teams call with a surgeon in a different city, and they could both explore the scan on their computers or in virtual reality using headsets and discuss what needs to be done for the procedure.

A method of pre-surgical planning that has made headlines in recent times is the 3D printing of anatomical models, for surgeons to handle the model practically and get a physical feel for what they need to do in the operating theatre. This process can take time, and surgeons in different cities or countries cannot view the same model at the same time.

“You can just pull someone into a meeting and show you a model and we can interact with it. We created the whole of HoloCare Studio during the pandemic, it’s made by clinicians for clinicians,” said Sundset.

Speaking about what the 8.9 million EUR funding from Horizon Europe means for the HoloSurge project, Sundset said: “It means that we can move a little bit quicker with our mission, our vision moving forward is to have it in preoperative planning, but our next goal is to transform it into the operating theatre. So imagine that suddenly you’re able to overlay the organ actually on the body as you are operating, onto a patient. Then you can see it in real time. We are also adding MRI, CT and ultrasound, this will allow us to move on from the liver to another organ, which is in this case the pancreas. Pancreatic cancer is quite difficult to firstly detect, diagnose and operate. So we’re adding the pancreas to the application, the pancreas and the liver are obviously in the same area, the abdomen, so we are calling it the ‘HoloGut’ program. It’s really exciting and we have some huge centres of excellence, one of them being Leeds. It’s going to be a big four years.”

Another potential use for the technology outside of pre-operative planning is its use as an educational tool, something that HoloCare has its eyes on already.

Sundset said: “That comes outside of the CE marks, you don’t have the regulatory hurdle, that goes away because it becomes an educational tool. We have an education piece rolling out in Leeds, on the heart, so a total other arena. Obviously these people will also be the early adopters as well, so the ones who are trained on it, it’s obviously natural that they would continue surgery. So its really essential that we capture the young surgeons in schools now.”

In the HoloSurge project, HoloCare is working with companies from various different specialisms, which all come together to make the technology what it is, such as specialists in ultrasound and MRI, as well as a patient organisation on board, radiologists and more.

Sundset says that the HoloSurge team was at first nervous that the older, more traditional surgeons would be more apprehensive to the adoption of the technology, but they have discovered that hasn’t been the case. The CEO added: “We thought it would be difficult when you haven’t been a gamer, or you know, my generation, we didn’t grow up with this. So when talking to surgeons and asking, how would you see this, how would you do that, and its taking like half an hour and they’re up and running.”

Speaking on the adoption aspect, Sundset said that the technology is not replacing anything, it is not taking away a CT or MRI, it is adding to something, and is giving an added benefit rather than taking something away and making people learn how to adopt that. Surgeons will still see what they need to see with traditional surgery, but they have this as an addition.

“Surgeons will still see what they need to see with traditional surgery, but they have this as an addition.”

Sundset added: “I think generally, technology adoption is a challenge and it takes time, but you have to just make sure that you’ve addressed all the issues. You might imagine how frustrating it would be if you pick up a HoloLens and its not charged. Teething issues that make you think ‘Oh, I hadn’t thought of that’, that somebody has to be the administrator and make sure everything is charged. So small things. And then my worst nightmare is that it becomes like a surgical toy, so its just put on the shelf somewhere and its like, oh it was fun. So as a company, we have to drive that adoption process.”

Sundest told MTI about how the price of the VR technology will come down as adoption increases: “They will come down as more people adopt it for different reasons, even for planning a hospital, for the smart hospitals of the future, everything will be planned in 3D moving forward.”