Making eagle-eyed robots

If we want to grab hold of an object, we have to know what it is we’re getting to grips with. And we have to know how far away it is. Viewing the world as a flat image simply isn’t good enough, and the same applies to robots. They have to see their surroundings as they are – in three dimensions. SINTEF researchers have developed a new 3D camera which enables robots to do just that – to make out and differentiate between distances, shapes and colours.

“In general, older generations of robots could only be used to perform simple and repetitive movements according to a fixed and pre-programmed pattern”, says Øystein Skotheim, a Senior Researcher at SINTEF ICT. “Our aim is to make a contribution to the development of robots with a little more intelligence and a much greater capacity to adapt to their surroundings”, he says.

Skotheim has been working on the development of machine vision for many years, including everything from the digital imaging of prehistoric rock carvings to the quality assurance of car components. The new 3D camera has been developed as part of a major strategic in-house project now being carried out at SINTEF. With better vision, robots will find it easier to learn how to adapt to changes going on around them. If Norwegian manufacturing is to become more competitive, it is vital to be able to combine tailored solutions with mass production.

“Robots must be able to recognise an object even among a randomly sorted pile of components, and then pick it up and piece it together with others”, says Skotheim. “And they will have to be able to quality assure products passing along a production line. Better machine vision can also open the door for robots that can work closely together with people”, he says.

Robots provide an ideal application for the 3D camera. But Skotheim also envisages a number of other uses.

“We’re talking about a generic technology with multiple potential applications”, he says. “Such as the manufacture of specially adapted prosthetics. Or for product designers making 3D models. Maybe we can combine these models with a 3D printer to produce a 3D copying machine?

Games technology

3D sensors for industrial applications have been with us for some time, but most of this technology is very expensive, and the sensors slow to react. For this reason they’re not well suited to mobile robot or production line applications. Three or four years ago, Microsoft developed a 3D motion sensing camera designed for the games market called Kinect. Naturally, this was marketed at an entirely different price level than the systems developed for industrial use.

“And, of course, the product wasn’t adapted for industrial applications”, says Skotheim. “What’s even more important, with an error margin of several millimetres it was nowhere near accurate enough. Kinect simply isn’t good enough for use on a robot designed to carry out quality assurance or to grab hold of components”, he says.

But he does admit that it has been a source of inspiration. He goes and fetches the first prototype of SINTEF’s 3D camera, developed and assembled at their own workshop. It’s called the ‘SINTEF ShapeCrafter 3D’, but for all that, it doesn’t look very impressive or futuristic – a tin can with two holes, not unlike the rat traps you find in old backyards in Trondheim. But the new prototype is now ready. This one have a polished black finish.

Structured light

Inside the box it bears little or no resemblance to a rat trap. Hidden behind the two holes are a projector and a super-fast camera which can take 150 images per second with two megapixel resolution. The projector sends out so-called structured light, or light beam patterns. To increase accuracy, it also sends out a sequence of twelve patterns per second. This results in twelve 3D images a second with a depth accuracy of 0.1 millimetres. This compares with the Kinect camera’s 5-10 millimetre depth accuracy using its fixed infra-red light pattern.

But the researchers have even more to thank the wealthy computer game industry for. Increasingly advanced games require the development of state-of-the-art graphics cards, and Skotheim and his colleagues make use of the massive computing capacity available on these cards to produce 3D images in real time.

Showcase on the world

Naturally enough, Skotheim is proud of his ‘baby’. “We’ve produced a 3D camera which is fast, accurate and inexpensive. I haven’t made a detailed price estimate, but based on the components we use, competing technologies are probably five times more expensive”, he says.

The SINTEF camera will soon be unveiled at the VISION 2014 exhibition. This is an international event held in Stuttgart in Germany aimed at showcasing machine vision technologies.

“Of course we hope it will attract a lot of interest”, says Skotheim. “We’re considering commercialising the camera by creating a SINTEF spin-off company, although this will depend on the response we get in Stuttgart”, he says.