With open communication lines, individual agents like robots, and drones can cooperate and work jointly to complete a task. But what happens when these communication lines are closed or hindered due to incorrect hardware or clogged signals? This is one of the most arduous challenges and the researchers of the University of Illinois Urbana-Champaign went ahead with this.
The researchers used multi-agent reinforcement learning, which is a type of artificial intelligence to establish a method for training multiple agents to work together. Huy Tran – an aerospace engineer at Illinois stated that when agents communicate with each other, it is simpler. However, their intent was not to make them communicate with each other – i.e. they wanted it in a decentralized manner.
Their concentration was also on scenarios where no clarity existed regarding what the various roles or jobs for the agents should be.
This scenario, according to Tran, is very much complicated to solve since it is unclear what one agent should do versus another. But, the intriguing question arising here is how to learn to work together to complete a task over time, Tran said.
This issue was solved by Tran and his collaborators with the help of machine learning. They developed a utility function that alerts the agent when it is performing a beneficial task for the team.
According to Tran, when working as a team towards the same goal, it’s difficult to know who contributed to the victory. They developed a machine learning process enabling them to recognize when an individual agent contributes to the overall team goal. For instance, in soccer, one player may be responsible for scoring, yet there might be other teammates too whose actions would have played a supporting role in making the scoring happen. And these supporting actions must also come to light, yet these delayed effects are difficult to comprehend.
The researchers’ algorithms can also detect when an agent or robot is doing something counterproductive to the goal. It is not that the robot deliberately chose to do something wrong, it is just that it is not helpful to the end goal.
Their algorithms were put to test with the help of simulated games such as Capture the Flag and StarCraft, a famous computer game.
The method performed well in an environment such as StarCraft, which can be a little more unreliable.
Tran stated that these algorithms may be applied in several real-life scenarios such as:
- Military surveillance
- Robots collaborating in a warehouse
- Controlling traffic signals
- Delivery coordination by self-driving vehicles
- Controlling an electric power grid
Tran stated that Seung Hyun Kim did the majority of the theory behind the idea as an undergraduate student studying mechanical engineering, with Neale Van Stralen, an aerospace student, assisting with implementation. Tran and Girish Chowdhary provided advice to both students. The work was recently presented to the AI community at the peer-reviewed conference Autonomous Agents and Multi-Agent Systems.
The paper, titled “Disentangling Successor Features for Coordination in Multi-agent Reinforcement Learning,” will be published in the Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, which will be held in May 2022.
