Q Learning Fixed Time and Actuated Signal Control under Simulated Demand Shifts

Abstract

Demand shifts can change the performance of a learned traffic-signal controller. This study reports a point-queue corridor simulation with six intersections, two-hour episodes, and Poisson arrivals. A single trained decentralized tabular Q-learning policy is compared with fixed-time and gap-out actuated controllers over nine demand regimes, three excluded from training, with twenty evaluation arrival realizations per regime. On training regimes the reported Q-learning delay reductions relative to fixed time range from 24.8% to 34.1%, versus 49.4% to 55.1% for actuated control. On held-out high arterial demand, Q-learning delay is 180.9 s per admitted vehicle versus 30.5 for fixed time, with throughput 94.9%. Actuated control has the lowest displayed delay in every regime. These results describe the reported simulator and one learned policy. Unbounded accumulation of unused discharge credit, service bypass for arrivals on empty green, unequal sensing and green constraints, and an unresolved green-share denominator prevent treating the comparison as validated physical capacity or field evidence. Independent simulator and policy replications are required before attributing the failure to a particular mechanism.