These days, one of the biggest scientific problems is to understand intelligence and build intelligent machines. A fundamental component of intelligence in both machines and biological things is the capacity to learn from experience.
Alan Turing, the pioneer of modern computer science, proposed the creation of machines capable of intelligent behavior in a remarkably foresighted 1948 report. He also envisaged “education” of such devices “by means of rewards and punishments.”
Turing’s theories eventually resulted in the creation of the artificial intelligence field of reinforcement learning. In order to create intelligent agents, reinforcement learning trains them to maximize rewards while interacting with their surroundings.
What is reinforcement learning?
It is well known among animal trainers that rewarding desired behaviors can affect the behavior of animals. When a dog successfully completes a trick, a dog trainer rewards it with a treat. This increases the likelihood that the dog will perform the trick correctly the second time by reinforcing the habit. This realization came from animal psychology and was applied to reinforcement learning.
However, the goal of reinforcement learning is to train computational beings rather than animals. A software agent, such as a program that plays chess, could be the agent. However, the agent may also be an embodied creature, such as a robot that is being trained to do domestic tasks. Similar to a chessboard or the constructed world in a video game, an agent’s environment can also be virtual. However, it might also be a home where a robot is employed.
Similar to animals, agents are able to sense their surroundings and respond accordingly. An agent capable of playing chess can access the setup of the chessboard and execute moves. Robots can use cameras and microphones to perceive their environment. In the real world, it can move around thanks to its motors.
Additionally, the human creators of agents program them with goals. Winning the game is the aim of an agent that plays chess. For example, a robot may be designed to help its human owner with home tasks.
The challenge of AI reinforcement learning is to create agents that see and act in their surroundings to accomplish their objectives. Reinforcement learning boldly asserts that every objective can be accomplished by creating a numerical signal, known as the reward, and instructing the agent to maximize the sum of all rewards it receives.
The vast range of potential objectives makes it impossible for researchers to determine whether this claim is valid. It is frequently called the reward hypothesis as a result.
In certain cases, choosing a reward signal that corresponds to a goal is simple. For an agent that plays chess, the reward may be +1 for winning, 0 for a draw, and -1 for losing. How to provide a rewarding signal for a useful home robot assistant is less obvious. However, there is an expanding list of applications in which academics studying reinforcement learning have successfully created effective reward signals.
In the board game Go, reinforcement learning was a huge success. Researchers believed that machines could not learn Go as easily as they could chess. AlphaGo was developed via reinforcement learning by DeepMind, which is now Google DeepMind. In a five-match match in 2016, AlphaGo defeated Lee Sedol, the world’s best Go player.
Reinforcement learning has been used more recently to improve the usefulness of chatbots like ChatGPT. Another application of reinforcement learning is to enhance chatbots’ capacity for reasoning.
Origins of Reinforcement Learning
But none of these achievements were anticipated in the 1980s. Reinforcement learning was then put up by Barto and Sutton, his doctoral student at the time, as a general paradigm for problem-solving. Animal psychology was not the only source of inspiration for them; control theory, which uses feedback to change a system’s behavior, and optimization, a mathematical area that examines how to choose the optimal option from a variety of alternatives, were also mentioned. They contributed mathematical underpinnings to the scientific community that have endured over time. Additionally, the algorithms they developed are now considered standard tools in the field.
Pioneers who take the time to produce a textbook are a rare asset to a discipline. As a result of their rarity, shining examples such as Donald E. Knuth’s “The Art of Computer Programming” and Linus Pauling’s “The Nature of the Chemical Bond” are noteworthy. The first edition of “Reinforcement Learning: An Introduction” by Sutton and Barto was released in 1998. In 2018, a revised edition was published. More than 75,000 citations have been made to their book, which has impacted a generation of researchers.
Unexpectedly, reinforcement learning has also affected neuroscience. Dopamine is a neurotransmitter that is essential to both human and animal reward-driven behavior. Researchers have employed certain reinforcement learning algorithms to interpret experimental results in the dopamine systems of humans and animals.
Barto and Sutton’s pioneering efforts, foresight, and support have contributed to the development of reinforcement learning. Their work has influenced practical applications, sparked a lot of study, and drawn significant funding from tech firms. There is no doubt that researchers studying reinforcement learning will keep looking ahead by standing on their shoulders.
