Accessible computer games developed to train artificial intelligence

“Many computer games are expensive and require a lot of data and power. We need games that require little computing power to train algorithms in industrial environments,” says Per-Arne Andersen, assistant professor at the University of Agder's (UiA’s) Department of ICT.

He recently earned his PhD with a thesis on how artificial intelligence in computer games can function well even if there is not much computing power. Andersen has developed artificial intelligence algorithms that can be used in systems where frequent decisions have to be made. Here, computer games are widely used to train artificial intelligence in game environments that are modelled on complicated industrial environments.

Artificial intelligence algorithms, also called recipes, are self-learning. When the researcher enters data, the algorithm learns to make smart choices.

Andersen’s area of research is called deep reinforcement learning, a combination of deep learning and reinforcement learning.

“The goal has been to enable computer systems to implement decisions without making mistakes. That will increase and maintain the security of the system,” Andersen says.

However, training algorithms are data-intensive. Different computer games also require a lot of computing power and electricity. According to Andersen, a lot of groundbreaking research is being done using the computer game StarCraft II, which requires expensive computer systems.

“Such computer systems are generally not available to all research institutions,” he says.

Six games

That is why Andersen developed six new game environments that can be used to train algorithms. All are easy to use and require little computing power compared to StarCraft. They are also inspired by different types of industrial environments that, for example, use robots for different tasks.

“In some of the games you can train algorithms in planning and learning with little information. Another game is a labyrinth game where algorithms learn to navigate labyrinths using memory. In short, the algorithms learn to memorise the shortest or smartest path out of a maze,” Andersen says.

The game Deep Warehouse was specially designed to evaluate the security of algorithms in automated warehouses. In these warehouses, the work is automated and carried out by robots. Andersen has been in touch with the company AutoStore AS, which offers a system for automated warehouses. In these warehouses, robots perform tasks such as registering orders, picking up goods and packing them for shipment.

“All the games have a low difficulty level and require little computing power. That makes them more accessible for others in the research community,” Andersen says.

In the game, researchers can test algorithms for industrial applications without doing any harm or endangering humans in the physical world.

“It is cheaper to destroy robots in a game than in an actual factory. Such methods can be very useful for industrial companies that want to test machines and computer systems,” Andersen says.

Further development of dream algorithm

Andersen’s PhD thesis is based on the further development of the dream algorithm that he developed while working on his master’s thesis. The algorithm needed little data.

“The algorithm has become even better at creating new game scenarios. It creates its own data along the way. I can also see that the new algorithm picks up on more details,” Andersen says.

Algorithm teaches itself

Andersen has tested the dream algorithm, and several variants of it, in strategy games that can be played in real time, such as StarCraft II. He and his research colleagues have also tested how the algorithm works in industrial settings where early detection of faulty production is crucial.

Since reinforcement learning requires a lot of data, learning becomes slow and difficult to implement if there is little computing power.

“What is unique about the redeveloped dream algorithm is that it learns parts of the game, and based on that, it envisions new ways to play in order to win. In an industrial setting, it is a matter of finding new and more efficient ways of carrying out tasks. The algorithm manages to see a little ahead,” Andersen says.

Andersen believes the algorithm can be a useful tool for industrial environments, for example in companies with automated processes where many decisions need to be made within a certain time frame.

“One example can be in an environment where the consequence of making mistakes is fatal. You can use such an algorithm to train the other algorithms in a dream environment that tries out different solutions without risking any damage should you make a mistake,” he says.

“The dream algorithm is a small step in the right direction towards more efficient and less data-intensive algorithms, but it is not the end of this type of research,” he says.

Available to all

Andersen’s games and algorithms are available in the research toolkit CaiRL. The toolkit was developed by the Centre for Artificial Intelligence Research (CAIR) at UiA.

“Andersen has developed game environments that researchers all over the world can use. Everything is open for everyone to use,” says Professor Morten Goodwin, deputy director of CAIR and the primary supervisor of Andersen’s doctoral work.

This article/press release is paid for and presented by the University of Agder

This content is created by the University of Agder's communication staff, who use this platform to communicate science and share results from research with the public. The University of Agder is one of more than 80 owners of ScienceNorway.no. Read more here.

See more content from the University of Agder:

• 

  Study reveals gaps in how children experience being heard in Norwegian child welfare services

• 

  AI-generated images of teenagers: "They're made so beautiful that it becomes intrusive"

• 

  20 years ago, it took around 17 minutes for teenagers to fall asleep once they got into bed. Today, it takes over an hour

• 

  Refugees in kindergartens: "Mutual trust is key"

• 

  New report: One in four middle managers in schools work in stressful environments

• 

  How AI can provide too broad an overview of society: "Open data threatens Norway's security"