Saltar al contenido ForoHackers

Reinforcement learning: start here

Aprendizaje por refuerzo
#343 · Ophuscado · · Denunciar

Reinforcement learning is learning to act by trial and error: an agent takes actions in an environment, receives rewards, and tries to collect as much total reward as it can over time. It suits games, robot control, scheduling and resource allocation, where no one can label the right move but a score is available. Its difficulty is that rewards are rare, delayed and easy to exploit.

Key concepts

  • Markov decision process: states, actions, transition probabilities and rewards, the formal frame of the field.
  • Policy and value: a policy chooses actions; a value function estimates the future reward from a state or action.
  • Exploration versus exploitation: trying new actions to learn, against using what already works.
  • Credit assignment: working out which of many past actions earned a late reward.
  • Method families: value-based, policy-gradient, actor-critic and model-based approaches, each with its own trade-offs.

Vocabulary

  • Episode: one run from a start state to an end.
  • Discount factor: the weight that makes later rewards count less than immediate ones.
  • Bellman equation: the recursive relation that links the value of a state to the values of its successors.
  • On-policy and off-policy: learning from data produced by the current policy, or by a different one.
  • Replay buffer: stored past experience that is sampled again for training.

What to study, in order

  1. Probability and a little dynamic programming; solve a small grid world exactly with value iteration.
  2. Multi-armed bandits: implement epsilon-greedy and measure regret.
  3. Tabular Q-learning on a simple environment, plotting returns over many seeds.
  4. A policy-gradient method on a standard control task; compare runs and watch the variance.
  5. Write a reward you think is right, then find how an agent exploits it, and fix it.

Related tags

#machine-learning #optimization #robotics #control-systems #probability

Inicia sesión o únete para responder.