Nobody enjoys waiting at a red light. However, signalized intersections are more than just a minor annoyance for drivers; while waiting for the light to change, vehicles consume fuel and emit greenhouse gases.
What if drivers could plan their routes so that they arrive at the intersection when the light is green? While that may be a lucky break for a human driver, an autonomous vehicle that uses artificial intelligence to control its speed could achieve it more consistently.
MIT researchers present a machine-learning approach that can learn to control a fleet of autonomous vehicles as they approach and travel through a signalized intersection while keeping traffic flowing smoothly.
They discovered, through simulations, that their approach reduces fuel consumption and emissions while increasing average vehicle speed. The technique produces the best results when all vehicles on the road are autonomous, but even if only 25% of vehicles use their control algorithm, it still results in significant fuel and emissions savings.
This is a very interesting place to be. Nobody’s life has improved as a result of being stuck at an intersection. With many other climate change interventions, there is an expected difference in the quality of life, so there is a barrier to entry. The barrier is much lower here, says senior author Cathy Wu, the Gilbert W. Winslow Career Development Assistant Professor of Civil and Environmental Engineering and a member of the Institute for Data, Systems, and Society (IDSS) and the Laboratory for Information and Decision Systems (LIDS).
Vindula Jayawardana, a graduate student in LIDS and the Department of Electrical Engineering and Computer Science, is the study’s lead author. The findings of the study will be presented at the European Control Conference.
Intricacies of intersections
While humans may drive through a green light without thinking twice, intersections can present billions of different scenarios depending on the number of lanes, how the signals work, the number of vehicles and their speeds, the presence of pedestrians and cyclists, and so on.
Typical approaches to dealing with intersection control issues solve one simple, ideal intersection using mathematical models That sound good on paper, but it’s unlikely to work in practice, where traffic patterns are frequently as chaotic as they come.
Wu and Jayawardana shifted gears and approached the problem using deep reinforcement learning, a model-free technique. Reinforcement learning is a method of trial and error in which the control algorithm learns to make a series of decisions. When it discovers a good sequence, it is rewarded. Deep reinforcement learning algorithms use neural network assumptions to find shortcuts to good sequences, even when there are billions of possibilities.
This is useful for solving a long-horizon problem like this because the control algorithm must issue up to 500 acceleration instructions to a vehicle over a long period, according to Wu.
And we have to get the sequence right before we know we’ve done a good job of reducing emissions and getting to the intersection quickly, she adds.
But there’s one more complication. The researchers want the system to learn a strategy that uses less fuel and has a smaller impact on travel time. These objectives may be incompatible.
We want the car to go fast to reduce travel time, but we want the car to slow down or not move at all to reduce emissions. Those competing rewards can be extremely perplexing to the learning agent, Wu explains.
While solving this problem in its entirety is difficult, the researchers devised a workaround using a technique known as reward shaping. They use reward shaping to teach the system domain knowledge that it would not have learned on its own. In this case, they punished the system whenever the vehicle came to a complete stop so that it would learn to avoid doing so in the future.
Traffic tests
They tested an effective control algorithm using a traffic simulation platform with a single intersection after developing it. The control algorithm is applied to a fleet of connected autonomous vehicles that can communicate with approaching traffic lights to receive signal phase and timing information as well as observe their surroundings. Each vehicle’s control algorithm tells it how to accelerate and decelerate.
As vehicles approached the intersection, their system did not generate any stop-and-go traffic. (Stop-and-go traffic occurs when vehicles are forced to stop due to stopped traffic ahead.) More cars passed through in a single green phase in simulations, outperforming a model that simulates human drivers.
When compared to other optimization methods aimed at avoiding stop-and-go traffic, their technique resulted in higher fuel consumption and lower emissions. If every vehicle on the road is self-driving, the control system can reduce fuel consumption by 18%, carbon dioxide emissions by 25%, and travel speeds by 20%.
A single intervention that reduces fuel or emissions by 20 to 25% is truly incredible. But what interests me, and what I was hoping to see, is this non-linear scaling. If we only control 25% of the vehicles, we get 50% of the benefits in terms of fuel and emissions reduction. That means we don’t have to wait until we have fully autonomous vehicles to reap the benefits of this approach, she says.
The researchers hope to study the interaction effects of multiple intersections in the future. They also intend to investigate how different intersection configurations (number of lanes, signals, timings, etc.) affect travel time, emissions, and fuel consumption. Furthermore, they intend to investigate how their control system may affect safety when autonomous vehicles and human drivers share the road.
For example, while autonomous vehicles may drive differently than human drivers, slower roadways and more consistent speeds may improve safety, according to Wu.
While this research is still in its early stages, Wu believes that this approach could be more easily implemented shortly.
The goal of this work is to move the needle in terms of sustainable mobility. We want to dream, too, but these systems are massive inertia monsters. What gets me up in the morning is identifying points of intervention that are small changes to the system but have a significant impact, she says.
Recent research by Professor Cathy Wu demonstrates how eco-driving provides a unified framework for reducing fuel consumption, thus minimizing carbon dioxide emissions, while also providing good results on average travel time.
More specifically, the reinforcement learning approach pursued in Wu’s work, by leveraging the use of connected autonomous vehicle technology, provides a feasible and appealing framework for other researchers in the same space, says Ozan Tonguz, professor of electrical and computer engineering at Carnegie Mellon University who was not involved in this research. Overall, this is an extremely timely contribution to this burgeoning and vital research area.
The MIT-IBM Watson AI Lab helped fund some of this research.
