Perimeter metering control for two-region urban networks: A deep reinforcement learning approach

Various perimeter control strategies have been proposed for urban traffic networks that rely on the existence of well-defined relationships between network productivity and accumulation, known more commonly as network Macroscopic Fundamental Diagrams (MFD). Most existing perimeter control strategies require accurate modeling of traffic dynamics with full knowledge of the network’s MFD. However, such information is generally difficult to obtain and subject to error. This talk describes recent efforts to alleviate this using deep reinforcement learning for networks made up of two unique regions. The proposed methods are completely model free in that they do not require knowledge of the network’s MFD. The algorithm learns the consequences of different control actions over time and uses this information to obtain optimal control policies under different situations. Results from numerical experiments show that the proposed method: (a) can stably learn perimeter control strategies under various types of environment configurations; (b) can consistently outperform the state-of-the-art, model predictive control (MPC); (c) demonstrates sufficient transferability to a wide range of traffic conditions and dynamics in the environment; and, (d) exhibits great potential for practical implementation.

Dr. Vikash V. Gayah received his B.S. and M.S. degrees from the University of Central Florida (2005 and 2006, respectively) and his Ph.D. degree from the University of California, Berkeley (2012). Dr. Gayah’s research focuses on urban mobility, traffic operations, traffic flow theory, traffic safety and public transportation. He has authored over 50 peer-reviewed journal articles, over 50 refereed conference proceedings, and numerous research reports to sponsors. He has worked on research contracts valued at more than $5 million, sponsored by the Pennsylvania, Washington State, Montana and South Dakota Departments of Transportation, US Department of Transportation (via the Mineta National Transit Research Consortium and the Mid-Atlantic Universities Transportation Center), Federal Highway Administration, National Cooperative Highway Research Program and National Science Foundation.