Saltar al contenido ForoHackers

Data science: start here

Ciencia de datos
#2909 · · Citar · Denunciar

Data science extracts knowledge from data by combining statistics, programming and knowledge of the subject. It asks a question, gathers and cleans data, explores it, builds a model or an estimate, and reports what is and is not supported. The skill is as much in doubting results as in producing them.

Key concepts

  • Question first: a clear question and a way to tell if the answer is useful come before any model.
  • Cleaning and exploration: most of the time goes into fixing, joining and plotting data to understand it.
  • Statistical thinking: distributions, sampling, uncertainty and the difference between correlation and causation.
  • Modelling and validation: fit on training data, judge on data the model has not seen.
  • Communication: results must reach decision makers in plain words, with caveats.

Vocabulary

  • Overfitting: a model that memorises noise in the training data and fails on new data.
  • Feature: an input variable used by a model.
  • Confidence interval: a range that expresses the uncertainty of an estimate.
  • Bias: a systematic error, in the data, the sampling or the model.

What to study, in order

  1. Descriptive statistics and probability: mean, spread, distributions and sampling.
  2. Python or R with a dataframe library; load, clean and summarise a real dataset.
  3. Plot every variable and every pair, and write down what surprised you.
  4. Fit a linear and a logistic regression and interpret the coefficients.
  5. Split data into training and test sets, use cross-validation, and see overfitting happen on purpose.
  6. Write a short report that states the question, the method, the result and its limits.

Related tags

#probability #machine-learning #data-visualization #python

No has iniciado sesión, así que esto se publica como anónimo: no se guarda con ello ninguna cuenta, nombre ni dirección, y no puede llevar enlaces ni imágenes. Inicia sesión o únete para publicar con tu nombre. Los registros de nuestro servidor web guardan la dirección y la hora de cada petición hasta 14 días; para que la tuya no quede en ellos, usa Tor. Para escribir al equipo de forma confidencial, usa el buzón. Firmar como fuente habitual (una clave guardada solo en este navegador)

Enviar tarda unos segundos aquí: tu navegador resuelve antes una prueba de trabajo contra el spam.

Responder