What is AI Reinforcement Learning, and how is it applied in our games, especially mobile games? Mobile games and Artificial Intelligence or AI may seem like they walk hand in hand together, but it hasn’t always been the case. We want our games to be smart enough only to the point where we can have fun and not where our games are becoming smarter than us and become unbeatable.
AI Reinforcement Learning is a subset of Machine Learning that allows the Software Agents (computer programs, AI algorithms) to act in Mobile Devices Environments (task or simulation) according to what will give them the most rewards. In that sense, they are used in our Mobile Video Games precisely because of how they can learn.
This ability allows our games to be as fun and as challenging as they are supposed to be in an environment. Let us talk more about how reinforcement learning is used in mobile games today as our smartphone applications are also getting more advanced.
What Is Machine Learning in Gaming
One of the many areas of AI that is still being explored today is Machine Learning in Gaming. Our computers, smartphones, and machines can learn more about the patterns and habits of humans they commonly interact with and use a wide base of knowledge and data that allows them to become more interactive.
In a sense, machine learning aims to make our machines have the cognitive capacity to learn and interact with humans as other humans do. The human-to-machine interaction feels more like a human-to-human interaction.
But, while there are many practical uses for this kind of AI, especially when it comes to different industries worldwide, do we really need a smart enough machine to outsmart humans in certain controlled environments? This is an ongoing debate and where reinforcement learning comes in.
Machine Learning Flowchart
What Is Reinforcement Learning (RL)?
Reinforcement Learning (RL) is an area of Machine Learning (ML). With Reinforcement Learning algorithms, we imply Artificial Intelligence and the ability to train machine learning models to make decisions. The AI Reinforcement Learning software “Agent,” say in a ChatBot, is programmed and trained to perform various decisions and movements that allow the machine to maximize its rewards in a controlled environment.
Suppose you think about the word “reinforcement.” In that case, it usually has something to do with how we raise our children or train our pets because we tend to reinforce good habits by rewarding them while minimizing bad habits by not rewarding them. Reinforcement learning in AI can be similar to that.
Reinforcement learning algorithms that incorporate Deep Neural Networks can beat the best in the world human expert game players, as demonstrated by Googles AlphaGo and Deepmind MuZero. MuZero learned the games Go, Chess, Shogi, and Atari without rules.
Please see our other AI articles in our series like “Smartphone AI: Can Your Mobile Phone Really Become Self-Aware?” and Smartphone AI: What is Mobile Deep Learning?
How Is Deep Reinforcement Learning (DRL) Used In Mobile Games?
The world we live in today is now more obsessed with video games than ever, especially with the rise of different gaming platforms outside of the traditional computer. We now have computers in our pockets known as Smartphones, and such mobile devices are now able to play powerful and exciting games designed and created by different programmers all over the world.
When we play our games, we want them to be fun and exciting enough to challenge us where the games themselves are already learning to be like humans instead of simply learning how to be better bots. That’s why Deep Reinforcement Learning (DRL) AI is so prevalent in mobile gaming or gaming in general because it is learning how that human element is there when playing against a computer program or a bot.
Deep Reinforcement Learning can be in video games and different platforms such as StarCraft, Dota2, and Atari. This kind of AI has been making progress when used in such games and can make the games more challenging for humans to play. This new threshold is because of how the machine on the other end can learn through experience and the data available to it.
Rise of Mobile Gaming
With the rise of mobile gaming, reinforcement learning has made its way even more into our smartphones, especially in the most popular games being played today in the ever-changing list of Gamers ranking the most popular mobile games each year.
These gaming lists include both paid and free video games of the most played on mobile platforms. The Mobile Video Game lists include defense games, fighting games, RPGs, sports games, multiplayer battle royales, strategy games, tower, and puzzle games.
According to Ranker, the top video games on mobile are Call of Duty: Mobile, Candy Crush, Fortnite, Pokémon Go, and Clash of Clans, which are still some of the best-selling games on mobile. Other good games on mobile include Among Us, PUBG, Minecraft, Dr. Mario World, and Final Fantasy XV: A New Empire.
Reinforcement Learning in Games
Reinforcement Learning is integral to the improvement and programing of many games. One good example is King of Glory, an eSports Multiplayer Online Battle Arena or MOBA that makes waves in China as the most popular MOBA in that country. The International eSports Tournament in 2019, which took place in Shanghai, China, in August 2019 and featured DOTA 2, ranked first, with a total prize pool of 34.33 million U.S. dollars.
OpenAI has been proposing AI to be used in MOBA such as King of Glory and Dota2 as the AI uses reinforcement learning when applied to such games. Simultaneously, some bots and programs have been in place for MOBA and other similar mobile games.
There is a new AI team challenge in these games with the human element of team fighting and collaboration. Multiple players learn from each other so that they can collaborate against a common enemy more efficiently. However, OpenAI has been able to display a mastery of such a style of play where the AI itself can learn and adapt to the playing style of their fellow AI and even the human players.
OpenAI Reinforcement Learning Team Beat Human Champions Team OG at Dota2 in 2019
And while OpenAI has yet to see an extended application in mobile games, it has seen its fair share of success in traditional computer games. The AI used reinforcement learning to beat human champions team OG in Dota 2.
In 2019 OpenAI’s five Dota 2 bots had trained for the equivalent of 45,000 human years in its ten months of existence before playing and beating the human team of the five top Dota 2 pros from team OG for a best of three matches.
If Artificial Intelligence Beating Humans is possible in traditional computing, it is for mobile games, and our smartphones, software, and apps are only getting more advanced every year. But who wants to play a game you cant win against?
The video below shows the OpenAI Bots Dota2 world Champion accomplishments:
What is a Bot?
A “Bot” is short for “Robot” or “Internet Bot” compared to a Mechanical Robot. A Bot is a computer program that operates as an agent who works on behalf of another person or group for a “User” or another program or simulates a human activity like collecting things or data.
We can use Bots to automate certain tasks in software, machines, and mobile games, meaning they can run independently without specific instructions from humans. So, in a sense, reinforcement learning will allow the AI in video game bots for this example to think, learn, and act based on what will enable them to win in a certain type of environment.
We don’t want the AI in our games to be pushing past the limits of our intelligence in the same way as we do when it comes to the practical applications of AI. This ability is why reinforcement learning is one of the best AI applications to increase the challenges in gaming constantly. But it also eventually leaves us the leeway to win in our video games against AI.
Reinforcement Learning AI Agents Learning Hide and Seek
In the attached video below, you can watch OpenAI’s Multi-Agent Hide and Seek Teams play millions of games against themselves and past games and learn to overcome many hilarious obstacles with their child-like smiling faces.
This video shows the ability of Reinforcement learning algorithms and the potential future power of these applications.
Artificial intelligence or AI has always been thought of as part of science fiction wherein computers, robots, and machines are getting smarter to the point that they can outsmart humans and even rule over them. As you can see in the video, that is slowly becoming a reality. This reality is not something that has to take place in the far-off future and come to fruition in the more advanced Artificial General Intelligence (AGI) and Artificial Super Intelligence stages (ASI), as it is very apparent now.
Reinforcement Learning Components
| Agent | Agents take actions; agents make decisions and are trained based on rewards and punishments. An algorithm is one form of Agent. |
| Action (A) | Action is the set of all possible moves an agent can make. |
| Environment | The environment is a world through which an agent moves and responds to the task or simulation. |
| Markov Decision Processes (MDPs) | Markov Decision Processes (MDPs) are mathematical frameworks to describe an environment in reinforcement learning. |
| Policy (π) | A policy is a method or strategy to map an agent’s state to actions. |
| Q-Value or Action-Value (Q) | Q-Value is similar to Value, except that it takes an extra parameter which is the current action. |
| Q-Learning | Q-learning is a commonly used model-free approach that can be used for building a self-playing agent. |
| Reward (R) | A reward is s central idea of information by which we measure the failure or success of an agent’s actions within a state. |
| Reinforcement Learning (RL) | Reinforcement Learning (RL) is a form of a machine learning technique that enables an agent to learn in an interactive environment by trial and error. |
| State (S) | A state is a specific place and moment in an immediate situation in which the agent finds itself. |
| Trajectory(T) | A trajectory is a sequence of states and actions that influence those states. |
| Value (V) | Future reward that an agent would receive by taking action in a particular state. |
What Are The Applications Of Reinforcement Learning?
Believe it or not, we have been applying reinforcement learning in many different things in today’s world and even in the past when our AI and our computers were not as sophisticated as they are now. That’s because there are many practical applications that reinforcement learning can be good for, especially in an environment that is not controlled and limited.
There are two main components of applying reinforcement learning. The first one is the game or the environment where reinforcement learning will occur by placing certain problems that need to be solved for the rewards. Meanwhile, the second component is the agent, which is the one that is placed in that environment where it needs to learn and adapt for it to reap the greatest rewards available to it.
What happens here is that the machine or the computer, in a game-like situation where it has to learn through trial and error certain moves and actions that will allow it to solve a problem and reap the rewards at the end of it. The goal here is to set the reward, but the programmer doesn’t give the machine or the computer any hints or tips that will allow it to solve the problem. In a sense, it is up to the programmed model to think of a solution to the problem it faces before it to maximize the reward.
Reinforcement Learning Agent and Environment
The most common use of reinforcement learning has always been in games because it places the agent or the machine in a situation that is ideal for this kind of artificial intelligence and training.
One example of reinforcement learning where this is applied is as a game in the world of trading and finance. The game has been using reinforcement learning as certain models can look at the available data and market benchmarks to determine or predict whether it is time to buy, sell, or hold on to certain stocks.
The controlled environment or the game here is the stock market while the agent here is the AI or the model. The agent uses a deep neural network such as the available market data to learn and determine the next action it should take to reap the greatest rewards. Of course, the rewards are the financial gains of playing in the stock market.
Reinforcement Learning in Self Driving Vehicles
From Teslas to self-driving Fourteen Wheeled Semi Trucks on the nation’s interstates, another good example of the applications of reinforcement learning in a more practical environment is self-driving autonomous vehicles. When you look at self-driving cars, they are autonomous in their decisions when driving and, if not already, far better, eventually than their human counterparts.
The autonomous vehicle’s ability to drive as well as or better than how a human drives is because of how reinforcement learning AI is applied to the vehicle’s programming. One way to look at it and explain how reinforcement learning works in self-driving cars is that humans can’t program these self-driving cars to react to every possibility that could happen on the road.
Humans are not aware of all of the possibilities that can happen while driving. Self-driving cars can learn and adapt to situations by using reinforcement learning and a deep neural network. This Artificial Intelligence software allows the vehicle’s computer to learn and adjust the speed limits, the drivable routes, and how to avoid possible collisions while it is on the road.
Conclusion
There are plenty of AI applications that tend to be on the more primitive side when we look at technological applications. But for gaming and future mobile gaming, the possibilities are endless.
Augmented Reality (AR) works on smartphones with AR apps and games, which use your phone’s camera to track your surroundings and overlay additional information on top of its screen.
As we say in our articles on Artificial Intelligence, with the exciting acceleration of technology and AI, the possibilities are endless.
References:
- Reinforcement Learning algorithms — an intuitive overview | by SmartLab AI | Medium
- How to teach an AI to play Games: Deep Reinforcement Learning
- https://arxiv.org/pdf/1912.10944.pdf
- OpenAI’s Dota 2 AI steamrolls world champion e-sports team with back-to-back victories