Saltar al contenido ForoHackers

High-performance computing: start here

Computación de alto rendimiento
#2933 · · Citar · Denunciar

High-performance computing uses many processors working together to solve problems too big or too slow for one machine: weather, molecules, structures, large simulations. It is parallel programming where performance is the goal, on clusters whose network, memory and storage matter as much as the cores.

Key concepts

  • Parallelism: data, task and pipeline parallelism, across cores, accelerators and nodes.
  • Message passing and shared memory: MPI between nodes, threads or OpenMP within one.
  • Scaling: strong scaling fixes the problem and adds processors; weak scaling grows both; Amdahl's law bounds the first.
  • Memory and communication: bandwidth, cache use and network latency usually limit speed more than arithmetic does.
  • Job scheduling: users submit jobs to a queue, and a scheduler allocates nodes and time.
  • Accelerators: GPUs handle massively parallel kernels and need data moved carefully.

Vocabulary

  • Node: one computer in a cluster.
  • Rank: one process in an MPI program.
  • FLOPS: floating-point operations per second, a peak figure rarely reached in practice.
  • Interconnect: the fast network joining nodes.
  • Roofline model: a chart relating a program's speed to its arithmetic intensity and the machine's limits.

What to study, in order

  1. C, C++ or Fortran, plus the basics of caches and vectorisation.
  2. Parallelise a loop with OpenMP and measure speedup against thread count.
  3. Write a small MPI program, such as a stencil or a matrix multiply, and measure strong and weak scaling.
  4. Profile it and find whether it is limited by compute, memory or communication.
  5. Run it through a job scheduler on a shared cluster, then port a kernel to a GPU.

Related tags

#concurrency #gpu #performance #computational-science #numerical-analysis

No has iniciado sesión, así que esto se publica como anónimo: no se guarda con ello ninguna cuenta, nombre ni dirección, y no puede llevar enlaces ni imágenes. Inicia sesión o únete para publicar con tu nombre. Los registros de nuestro servidor web guardan la dirección y la hora de cada petición hasta 14 días; para que la tuya no quede en ellos, usa Tor. Para escribir al equipo de forma confidencial, usa el buzón. Firmar como fuente habitual (una clave guardada solo en este navegador)

Enviar tarda unos segundos aquí: tu navegador resuelve antes una prueba de trabajo contra el spam.

Responder