An article from Norwegian SciTech News at NTNU
Can you spot the difference between these cats?
Using algorithms in a computer program, researchers can simulate the processes that happen in our brains when we sift through important and unimportant image information?
Have you ever tried a Spot-the-differences test?
It’s a game where you have two pictures that look absolutely identical, but where small details are different. Often you have to study the pictures for several minutes to discover the differences. We even know the differences are there, so why can’t we spot them right away?
“This is called ‘change blindness’ and is a very interesting phenomenon,” says Steven Le Moan, who until recently worked at NTNU’s Norwegian Colour and Visual Computing Laboratory in Gjøvik. He wants to learn more about why humans fail to notice even major changes in images.
“We compress the image in our mind as we look at it,” says Le Moan. “We take in the entirety of a picture within microseconds, and leave out the image details.”
Le Moan asked himself if it could be possible to program a computer to compress an image in the same way. Using algorithms in a computer program, could he simulate the processes that happens in our brains when we sift through important and unimportant image information?
Break from traditional methods
Le Moan and his colleagues at the Colour and Visual Computing Lab have developed an image compression method that works a bit like change blindness. Large areas of the image are simply removed, reducing the file size considerably.
This contrasts with JPEG and other traditional methods for compressing images.
The traditional methods throw out information from the image with little regard to the subject. Although all compression algorithms remove information—to a greater or lesser degree—based on how humans see the image, Le Moan’s method stands out because it goes a step further and involves understanding the image content. Entire contiguous areas of the image can be removed this way, without interfering with what you see.
“The more time you need to detect a change, the less important we can assume the information is. We measure what people see and don’t see, and can leverage the change blindness phenomenon this way. We can remove parts of the image without you noticing that they’re missing. This offers new possibilities compared to traditional compression methods,” says Le Moan.
Removing unimportant parts of the picture
Let’s say you look at a picture of a cat walking on grass. First you’ll focus on the cat over the grass. You won’t remember or notice small changes in the background.
The computer algorithm can detect which areas of an image are best suited to take advantage of the change blindness phenomenon, and finds areas that can be removed or altered without our discovering it. These pixels can later be recreated from what the adjacent area looks like.
An “unimportant” area on the blue summer sky can be recreated from how the rest of the sky looks. The forest in the background can be reconstructed to blend in with the surroundings, whether the differences from the original are large or small.
How far can image compression go?
So computer algorithms can remove parts of your image that are considered to have low importance. But, you may ask, what are the limits of what can be altered? What is defined as natural?
These are challenging questions in image processing research. Le Moan believes it is okay to change an image, as long as it does not look unnatural or disturb the viewer. Experiments have shown that up to fifteen per cent of an image can be altered without being detected or looking unnatural.
“There are also a lot of unanswered questions in this field, especially in terms of how we look at a picture and why we look at what we do,” says Le Moan. He has high expectations for where further research in this area may lead.
He hopes that in time research can expand the field into applications beyond image compression. “It would be exciting to use this in situations where something has to look identical but isn’t, like copies of documents or photographs, banknotes or watermarks,” Le Moan says.
