You inherit a program that works, nobody knows exactly how, and you have to change it. Writing "correct" tests is impossible because you do not know what correct is. Write characterization tests instead: tests that record what the code does today.
The procedure:
- Pick a unit (a function, a batch step, a report).
- Feed it a set of inputs, including ugly real-world ones (anonymised copies, of course).
- Save the actual outputs as the expected results ("golden files").
- From now on, any change in output fails the test and you must decide whether the change was intended.
def test_report_matches_golden():
out = run_report_batch("fixtures/records_small.dat")
assert out == Path("golden/report_small.txt").read_text()
Why it works: you are not asserting that the behaviour is right, only that it did not change by accident. Bugs that have been there for twenty years are now documented as bugs, and fixing one becomes a deliberate act with a diff you can read.
Cautions:
- Non-deterministic parts (dates, run ids, ordering) must be normalised or injected, or the test is flaky.
- Keep the fixtures small enough to review. A 2 GB golden file is a hostage, not a test.
- When you find a real bug, add a test for the correct behaviour and leave the characterization in a clearly named place until you decide to change it.
Calm refactoring is boring refactoring. Boring refactoring is how the nightly batch keeps running.