An Introduction to Deep Reinforcement Learning

Summary

This introductory chapter presents deep reinforcement learning as a machine-learning approach in which an agent learns by interacting with an environment, taking actions, and receiving positive or negative rewards. It establishes the reinforcement-learning framework through the loop of state, action, reward, and next state, and explains the reward hypothesis, expected return, Markov property, observations, states, and action spaces. The chapter then introduces policies as the agent’s decision-making mechanism and contrasts policy-based methods with value-based methods, including the use of neural networks to estimate policies or state values in deep RL. It concludes by preparing learners to train a lander agent, share it on the Hugging Face Hub, and continue with Q-Learning and value-based methods.

Takeaways

1. 1

   An RL agent learns through trial-and-error interaction with an environment, using positive or negative rewards as feedback for its actions.

2. 2

   The RL process produces a sequence of state, action, reward, and next state, while the agent seeks to maximize expected cumulative reward, also called expected return.

3. 3

   Policy-based methods train a policy directly, whereas value-based methods train a value function and use it to define the policy; deep RL applies neural networks to these estimations.