Treffer: Reinforcement learning for train dispatching : A study on the possibility to use reinforcement learning to optimize train ordering and minimize train delays in disrupted situations, inside the r ail simulator OSRD

Train dispatching is a complex process, especially when the train traffic is disrupted, as the decisions taken by the dispatchers can have substantial consequences on the delays of the trains. The most frequent dispatching decisions consists in changing the order of trains at convergence points, where two tracks unite to become a single track. Choosing the right train order is crucial, as the trains cannot bypass each other again while they are on the single track after the convergence point. The OSRD team of SNCF R´eseau has designed the rail simulator OSRD (Open Source Railway Designer), which can simulate any traffic situation. The goal of this degree project was to study if reinforcement learning could be implemented in that simulator to find optimal ordering policies under traffic disruptions. A thorough literature review was carried out to identify what reinforcement learning models have already been used in the literature to handle similar problems. None of the models seen in the literature could directly be adapted to the OSRD simulator but key features which seemed to be necessary to build an efficient reinforcement learning model in OSRD were determined. Based on those features and on the specificities of OSRD, a custom reinforcement learning model (states, actions, rewards) was created. This model was then implemented into a Python reinforcement learning environment after designing an interactive simulation module which enabled communication between the Python reinforcement learning environment and OSRD. After ensuring that the model was running and enabled interacting with an OSRD simulation to retrieve decisions from it and take decisions which modified the train order, the study focused on what reinforcement learning algorithms could be used to implement a reinforcement learning algorithm which learns based on the implemented reinforcement learning model. Another in-depth literature review was performed on the existing reinforcement learning algorithms, and it was concluded that the most suitable .