Code review: Reinforcement learning floor plan exercise

There is so little code that I have been able to find on reinforcement learning that I decided to write a code review on a simple exercise I obtained whilst watching a data science by Edureka!.

The updating equation for Q learning based on the Bellman Equation can be seen below:-

The reinforcement learning exercise discussed in this post is to move the marker to room 5, which is outside. A diagram of the floorplan can be seen below:-