It's a classic misunderstanding. I don't feel the need to write a test for every public function or method, every constructor or getter or setter, and some folks will conclude that I therefore don't have 100% test coverage. But I don't write a line of solution code unless it's required to pass a test. And if you were to break any part of my code, the chances are very high that at least one of my tests will fail. In fact, I make sure each test fails when the code is wrong. So, I don't have a dedicated test for every part of the code, but all of my code is meaningfully tested.
Great perspective. Here is the math behind that line of thinking. Traditional code coverage metrics are looking at code as a series of lines. They instrument code before your tests, run the code, and then tell you whether that line was hit. In reality, code is not a bunch of lines. It’s a graph. Code compiled into an abstract syntax tree – and a tree is a directed, cyclical graph. Each node can be visited multiple times, from different contexts. Code coverage can’t measure that, because it doesn’t know context. That’s why it’s at best a hint. Some folks index too highly on code coverage because it’s _a_ measure, even if it’s not a particularly good one, and ascribe way more importance to it than needed.
This resonates. One way I keep the focus on behavior while avoiding test sprawl is mutation testing: if flipping a conditional or removing a call doesn’t fail a test, I’m missing a meaningful check. I also add contract tests at module boundaries and a few property-based tests for invariants. They catch regressions without locking tests to structure and keep change velocity high. Thanks for articulating the difference between coverage and confidence. The demo of breaking code to prove tests is gold.
What a lot of folks seem to miss is that when you write tests for your use cases like this, and you're sure you covered all the use cases, and then your SonarQube tells you that 35% of your code is not covered, that means over a third of the code should be deleted because it's not used. It's not being sloppy with code coverage to do things this way. Code coverage doesn't mean what some people think it means.
I've always been a fan of writing tests as you describe. Often my initial few tests will provide 90%+ line coverage, however there is also an expectation to write tests for almost every public function. Unit testing the very key building blocks / core calculation functions then integration test (with test containers and mock API servers) the rest. I find this to be a good balance that makes refactoring simpler.
Exactly. Chasing 100% coverage can turn into a checkbox exercise instead of real validation. I care way more about tests that would actually catch a bug than about having one for every getter/setter.
Instead of thinking of tests as something to add to make sure it doesn't break, what if you thought of establishing test cases with a domain expert and other appropriate stakeholders first. Using tests are permission to modify production code, treating them as the requirements itself? Seems under this model, the idea of balancing what needs to be covered and what doesn't stops mattering. Even if you break apart the problem on your own to smaller methods, if you plan what those methods are supposed to do, then you already have test cases. From this, there is no coverage debate to be had. Tests become about standard communication and requirements.
The real problem lies in situations when someone breaks any part of your code and yet all your tests still pass. It's easy to get some tests to fail when part of the code breaks; but it is hard to catch situations where a test should fail, but it remains unperturbed. Blissfully ignorant tests are the worst tests.
It's great when you have feature tests (testing a whole endpoints or whatever) that still trigger line coverage. Currently I'm working in a project where that is not the case, and the team only writes api tests. They may cover a lot, but it doesn't show up in any reports so you can't tell how high/low the coverage actually is. Quite annoying.
It's quite common, when devs see me demonstrating TDD for the first time, that if I extract some of the implementation code into its own method and/or its own class, they'll ask "Shouldn't we write a test for this new thing?" And when I reply "It's already being tested", they can be a little incredulous. I usually demonstrate what I mean by breaking that code and running the tests.