Saltar al contenido ForoHackers

Characterization tests: how to touch code nobody understands

Mantenimiento y código heredadoAseguramiento de la calidadPruebas
#1054 · legacy_cobol_whisperer · · Denunciar

You inherit a program that works, nobody knows exactly how, and you have to change it. Writing "correct" tests is impossible because you do not know what correct is. Write characterization tests instead: tests that record what the code does today.

The procedure:

  1. Pick a unit (a function, a batch step, a report).
  2. Feed it a set of inputs, including ugly real-world ones (anonymised copies, of course).
  3. Save the actual outputs as the expected results ("golden files").
  4. From now on, any change in output fails the test and you must decide whether the change was intended.
def test_report_matches_golden():
    out = run_report_batch("fixtures/records_small.dat")
    assert out == Path("golden/report_small.txt").read_text()

Why it works: you are not asserting that the behaviour is right, only that it did not change by accident. Bugs that have been there for twenty years are now documented as bugs, and fixing one becomes a deliberate act with a diff you can read.

Cautions:

  • Non-deterministic parts (dates, run ids, ordering) must be normalised or injected, or the test is flaky.
  • Keep the fixtures small enough to review. A 2 GB golden file is a hostage, not a test.
  • When you find a real bug, add a test for the correct behaviour and leave the characterization in a clearly named place until you decide to change it.

Calm refactoring is boring refactoring. Boring refactoring is how the nightly batch keeps running.

#1586 · tabs_over_spaces_42 · · Denunciar

Review-wise, this is the best thing you can attach to a risky change: "here is the golden output before and after, and here is the diff, all intended". I will approve that kind of PR every time.

#1587 · race_condition_rita · · Denunciar

One more caution: a golden file that is regenerated with a single command gets regenerated when it fails, without reading the diff. Make the update step explicit and review it like code, or the test turns into a rubber stamp.

#1588 · legacy_cobol_whisperer · · Denunciar

Yes, and that is a social problem more than a technical one. The rule on my team was: the golden update goes in its own commit, with a message explaining what changed and why.

#1589 · two_phase_commitment · · Denunciar

In database land the same thing: capture the result set of the important queries before a schema change, and compare after. I once caught a collation difference that sorted accented names differently. Nobody had asked for that.

#1590 · rubber_duck_supervisor · · Denunciar

What would you do with a part of the code whose output depends on the current time? Think about where the clock should come from before you answer.

#1591 · legacy_cobol_whisperer · · Denunciar

Inject it, duck. Pass the date in as a parameter, default to the real clock only at the outermost layer. Then the test runs "on" the same day forever.

Inicia sesión o únete para responder.