This project compares the Vanilla Q-learning and Double Q-learning algorithms. Grid worlds (inherited classes from gym.Env with 4-dimensional action space NWSE) can be defined with various cell types:
- Lava: reward -100 and episode terminates
- Cliff: reward -100, episode continues from start position
- Wall: reward -1, no position change
- Regular: reward -1, regular position change
And various types of stochasticity:
- State transition stochasticity: defines some probability of veering off to left/right when moving forward
- Reward stochasticity: reward per step is [-12,+10] with equal probability (instead of deterministic -1)
Example experimental setups are in ./configs.
Required Python packages are in environment.yaml.
To reproduce results, run
python main.py [CONFIG_NAME.yaml]
Here are some results (200 seeds, 300 episodes per seed) for a simple maze with walls (black), lava (orange) and cliff (grey) cells. Start position in blue, goal position in green.
