Kinds of test
Many fast tests at the bottom, few slow ones at the top
Unit tests are many and fast. End-to-end tests are few and slow.
Tests come in three sizes, and the picture is drawn to scale by how many of each a healthy project has. At the bottom, unit tests. Each checks one function on its own, in a hundredth of a second. Five hundred of them finish in five seconds. In the middle, integration tests check several parts together, such as code with a real database. There are fewer, and each takes seconds. At the top, a handful of end-to-end tests drive the whole system. Five of them take five minutes.
Test doubles · Going deeper
A unit test replaces what is slow or outside your control with a double
A test double keeps a unit test fast, and proves nothing about the real service.
Unit tests
A test is an executable statement of what the code must do
Call the code with a known input and assert on the result.
Here is the smallest useful test. The function under test decides whether a processor load deserves a warning. Below it, the test. It calls the function exactly at the limit and expects ok, then one above the limit and expects a warning. It passes. Months later, someone changes greater-than to greater-or-equal. It looks harmless. The test fails at once, in the pipeline, before the change can be merged.
Parametrised tests · Going deeper
One test function, many cases: each passes or fails under its own name
parametrize runs one test once per case and reports each separately.
Quality gates
Machines check what they can, so reviewers can think
Format, lint, types, tests, scan. Then a person.
Before any person looks at a change, a series of automatic gates has already examined it. A formatter fixes the layout. A linter spots unused names and risky patterns. A type checker notices text being passed where a number belongs. None of these even runs the code. Then the tests run it, and check its behaviour. A scan looks for vulnerable dependencies and leaked keys. Only then does a reviewer spend attention, on the questions a machine cannot answer.
Quality gates · Going deeper · 1 of 2
What test coverage does and does not tell you
Coverage finds untested lines. It does not prove the tested ones are right.
Code review · Going deeper · 2 of 2
What a reviewer looks for, once the machines are done
Review design and behaviour first, style last: the linter has already had its say.
Review design and behaviour first, style last: the linter has already had its say. The gates have checked layout, types and tests. A person is there for what a machine cannot judge. Google's engineering practices list what to look at, roughly in this order. Look for, in this order. Design. Does the change belong here, and does it fit the rest of the system? Functionality. Does it do what the author intended, including at the edges and under concurrency? Complexity. Could a reader understand it quickly? Is it solving a problem that does not exist yet? Tests. Are they there, and would they fail if the code were wrong? Naming and comments. Names say what a thing is. Comments say why, not what. Style and documentation. Left to the formatter and the style guide. Docs updated where behaviour changed. As the author. Keep it small. One purpose per pull request. A small change gets read; a large one gets skimmed. Say what and why. The description should let a reviewer start without asking. Answer every comment. With a change or a reason. If the code needed explaining, improve the code. Tone: Review the code, not the person. "This loop reads the file twice" is a comment; "you are careless" is not.
The gates have checked layout, types and tests. A person is there for what a machine cannot judge. Google's engineering practices list what to look at, roughly in this order.
Look for, in this order
- Design. Does the change belong here, and does it fit the rest of the system?
- Functionality. Does it do what the author intended, including at the edges and under concurrency?
- Complexity. Could a reader understand it quickly? Is it solving a problem that does not exist yet?
- Tests. Are they there, and would they fail if the code were wrong?
- Naming and comments. Names say what a thing is. Comments say why, not what.
- Style and documentation. Left to the formatter and the style guide. Docs updated where behaviour changed.
As the author
- Keep it small. One purpose per pull request. A small change gets read; a large one gets skimmed.
- Say what and why. The description should let a reviewer start without asking.
- Answer every comment. With a change or a reason. If the code needed explaining, improve the code.
Lesson complete
You finished “Testing and code quality”
Nicely done.