New 3D models of the colon can help detect disease more rapidly

“Our work shows that it's possible to recreate a fairly accurate three-dimensional model of the colon of some patients based on a single image taken by a capsule endoscopy camera – even if it’s a low-quality image,” says Pål Anders Floor.

He is a researcher at NTNU's Department of Computer Science.

Goal: Better images, faster diagnoses

Floor has spent several years using images from capsule endoscopy cameras to reconstruct an almost identical 3D model of the intestines. 

These cameras were first introduced over 20 years ago. However, low quality and visual noise in the images prevented this smart technology from becoming widely adopted.

Floor and Professor Ivar Farup supervised Bilal Ahmad's doctoral research. Their main aim is to improve the images specialists use to detect abnormalities in patients' gastrointestinal tracts. This improvement will help in easier disease detection and quicker diagnoses. 

These are important steps towards the ultimate goal: Fighting colon cancer, the second most common type of cancer in Norway.

A rubber colon, an endoscope, and mathematics

Using an artificial colon, single images taken by endoscopy, and a special algorithm, NTNU researchers have managed to reconstruct a 3D colon model. 

The endoscope does the same job as a capsule camera, with some important differences: The images have a higher resolution, and the camera brightness can be adjusted manually.

The researchers have used a mathematical model that can reconstruct a 3D shape using a single, two-dimensional image. This is called a Shape from Shading (SFS) algorithm. 

True colon copy recreated on a PC

If algorithms are fed with images that are noisy, or where the colour is distorted, the errors will spread to the 3D model.

“We show that with careful calibration and pre-processing the images, we can obtain a good 3D model based on just one image – even if the image is full of noise and visual distortion,” says Floor.

Significant benefits for doctors

Floor and his colleagues have also been working on developing 3D models based on sequences of multiple images taken by capsule cameras. This allows them to recreate longer sections of the colon. The models should make it possible to find out exactly where in the intestines there are abnormalities and signs of disease.

“We have previously shown that doctors find this type of model useful when assessing patients, along with having a video from a capsule camera. Now we have shown that we can calculate relatively accurate 3D models using real images and technology that already exists,” says Floor.

Better, faster, and more accurate

In their new work, the researchers are reconstructing three-dimensional colon images with relatively high quality from a rather poor original. Their combined efforts open the possiblity of reconstructing the geometry of an entire colon. 

“The 3D model can be used to adjust the lighting in an original image so that darker areas become better illuminated, and more detail can be seen,” says Floor.

“We all have a colon, but the anatomy and geometry of your bowel is probably quite different from your neighbour’s. New 3D models can be adapted to each patient. They allow specialists to examine the digestive tract from different angles on a PC, and enable them to plan better and determine exactly where an intervention should take place," he says. 

Doctors can also practise ahead of particularly difficult procedures. This can reduce the risk of something going wrong, according to Floor. 

The big question is whether the new methods could lead to a breakthrough in the fight against colon cancer.

“It’s too early to draw any definite conclusions. First, we need to refine what we have developed and do more research. It could take quite a bit of time to achieve results that are statistically valid,” says Floor. 

Top of the cancer statistics

Data from the Cancer Registry of Norway shows that Norway tops global statistics in respect to the incidence and mortality of colorectal cancer. It is the second most widespread form of cancer in the country, with 4,912 diagnoses of colorectal cancer in 2023.

Expectations for the first capsule cameras, known as Wireless Capsule Endoscopy (WCE), were high. It was thought that the small capsules could replace traditional colonoscopy and gastroscopy. Some people find these traditional methods so painful that they refuse to undergo them.

Camera in a capsule

Capsule cameras, by contrast, can investigate all the nooks and crannies in the intestines. They do this almost imperceptibly and painlessly while the patient goes about their daily life. 

As soon as it is swallowed, the capsule commences its laborious journey through the digestive tract. This journey can take many hours, and along the way, it transmits tens of thousands of images. 

As good as it is, this innovation does not work equally well for the whole digestive tract. The capsule camera is the best option for screening the small intestine. As far as the colon is concerned, there is still a way to go before it can replace a colonoscopy.

Part of the explanation is that it takes a lot of time to review and analyse such large volumes of data. There is also a distinct possibility of missing signs of a disease. The capsule follows the movements of the intestine, progressing in fits and starts. Sometimes, the camera suddenly zooms past a polyp.

Missing important details

It is therefore possible that only one image might be taken of a critical site with signs of disease. 

Visibility inside the intestine is also very varied, often with little light and all kinds of debris. The images have poor resolution, and there are no guarantees that the most important image will be sharp and clear. 

Herein lies the importance of being able to reconstruct a 3D model based on just one image, as Floor and his colleagues have now done. The new 3D models could contribute to more accurate and less demanding gastrointestinal examinations, leading to better treatment. 

From cultural heritage to robotics

Floor and his colleagues have made several recommendations about how the design of the technology could be improved to obtain even higher quality images. 

They also believe that the technique could easily be transferred to other areas where it would be helpful to generate 3D shapes and models from images.

The researchers wrote that this could include cultural heritage, robotics, and medical diagnostics, to name but a few.

Reference:

Ahmad et al. Single-Image-Based 3D Reconstruction of Endoscopic Images, Journal of Imaging, vol. 10, 2024. DOI: 10.3390/jimaging10040082

CAPSULE

The research effort is part of work package 2 in the Research Council of Norway’s project entitled Improved Pathology Detection in Wireless Capsule Endoscopy Images through Artificial Intelligence and 3D Reconstruction (CAPSULE).

Floor praises the collaboration with Innlandet Hospital Trust and says it has been crucial for the progress of the project. Their specialists have played a crucial role, in both the choice of methodology and evaluating the models along the way.

“Without the feedback of these doctors, we would largely be fumbling in the dark,” he says.

This content is paid for and presented by NTNU

This content is created by NTNU's communication staff, who use this platform to communicate science and share results from research with the public. NTNU is one of more than 80 owners of ScienceNorway.no. Read more here.

More content from NTNU:

• 

  Could a digital driving instructor offer the same quality as a human, but at a lower cost?

• 

  Language affects how quickly we perceive shades of colour

• 

  Cholesterol type can affect your heart attack risk

• 

  5 tips for if you get lost

• 

  Artificial intelligence: Can we recreate the smell of ancient Rome from 2,000 years ago?

• 

  Being young and beautiful can lead to risky behaviours