Saltar al contenido ForoHackers

Deep learning: start here

Aprendizaje profundo
#333 · Ophuscado · · Denunciar

Deep learning is machine learning with neural networks of many layers, trained on large amounts of data with gradient descent. It learns its own features from raw pixels, sound or text instead of relying on hand-built ones, which is why it drives most modern vision, speech and language systems. It is also data-hungry, costly to train and hard to inspect.

Key concepts

  • Backpropagation: the chain rule applied layer by layer to get the gradient of the loss for every weight.
  • Architectures: fully connected, convolutional, recurrent and attention-based layers, each with a different assumption about the data.
  • Optimisation: stochastic gradient descent with momentum or adaptive step sizes, learning-rate schedules, batches.
  • Regularisation: weight decay, dropout, data augmentation and early stopping fight overfitting.
  • Representation learning: the hidden layers become reusable features; pretraining and fine-tuning reuse them.

Vocabulary

  • Tensor: an n-dimensional array, the basic data type of the field.
  • Epoch: one pass over the training set.
  • Activation function: the non-linearity between layers, such as ReLU.
  • Vanishing gradient: gradients shrinking through many layers until early layers stop learning.

What to study, in order

  1. Machine learning basics first: loss, overfitting, validation.
  2. Write a tiny network and backpropagation in NumPy, and check your gradients numerically.
  3. Move to an autodiff framework; train a convolutional network on a small image set and watch the loss curves.
  4. Break it on purpose: bad learning rate, no normalisation, leaked labels, and diagnose each.
  5. Read about attention and sequence models, then fine-tune a pretrained network on your own small dataset.

Related tags

#machine-learning #linear-algebra #gpu #computer-vision #optimization

Inicia sesión o únete para responder.