Skip to the content ForoHackers
Log inJoin

Multimedia systems: start here

Multimedia systems
#2991 · · Quote · Report

Multimedia systems capture, compress, store, send and play sound, images and video. Raw media is enormous, so almost everything rests on codecs that throw away what people barely notice, and on pipelines that keep audio and video in step. Streaming adds the problem of delivering it smoothly over an unreliable network.

Key concepts

  • Sampling and quantisation: turning continuous sound and light into numbers; sample rate and bit depth set the quality.
  • Compression: lossless methods keep every bit; lossy codecs use transforms and perception to shrink data a great deal.
  • Containers and codecs: the codec encodes the stream; the container packs audio, video and subtitles together with timestamps.
  • Synchronisation: clocks and timestamps keep sound and picture aligned.
  • Streaming: buffering, adaptive bit rate and loss handling for live and on-demand playback.

Vocabulary

  • Bit rate: how many bits per second the stream uses.
  • Frame types: key frames stand alone; other frames store only the changes from their neighbours.
  • Latency and jitter: delay, and variation in delay, between sending and playing.
  • Transcoding: converting from one format or size to another.

What to study, in order

  1. Sampling theory: why twice the highest frequency, and what aliasing sounds like.
  2. Write a WAV file by hand from a generated tone, and play it.
  3. Implement a simple lossy image compressor with a DCT and quantisation, and view the artefacts.
  4. Use a standard command-line media tool to inspect streams, convert formats and change bit rates.
  5. Build a small streaming setup and watch what loss and jitter do to the playback.

Related tags

#signal-processing #information-theory #computer-graphics #speech

You are not logged in, so this goes up anonymously: no account, no name and no address is stored with it, and it may carry no link and no image. Log in or join to post under your name. Our web server's logs keep the address and time of every request for up to 14 days; to keep yours out of them, use Tor. To reach the staff in confidence, use the drop. Sign as a returning source (a key kept only in this browser)

Sending takes a few seconds here: your browser first works out a proof of work against spam.

Reply