Testing is Science

04 May 2014. comments

Surely by now you’ve heard about the TDD fiasco that was kicked off at Railconf 2014. In his keynote at the conference David Heinemeier Hansson proclaimed that TDD is dead

“Maybe it was necessary to use test-first as the counterintuitive ram for breaking down the industry’s sorry lack of automated, regression testing.”

– David Heinemeier Hansson

I think something got lost in translation for David. TDD is absolutely not about automated regression testing. A nice side effect of TDD is indeed automated regression testing but that is not the primary purpose. TDD’s primary purpose is to get you to ask the question “How do I want to consume the code that I’m about to write?”. It’s about forcing future-you to follow through on promises that past-you made before you dove into the implementation, thus driving designs that are pleasant to use and easy to understand.

TDD is also about reminding future-you that you don’t need to write that other bit of code just “because I might need it”. TDD doesn’t permit you to write code unless it was the minimal amount required to make the tests pass, and therefor you by definition can’t add extraneous gold plating.

TDD pushes for your design to be pleasant and easy for consumers because you wrote the consumption of it before you wrote the implementation. Opponents of TDD often trot out the argument of “sometimes I just know what I need and don’t need a test”. This way of thinking ignores 2 critical pieces of information:

  1. Outside feedback
  2. Provable methodology.

Let’s address each one individually.

When you develop code under the “sometimes I just know what it should look like” mentality you are actively proclaiming that your knowledge and yours alone is the One True Way and that it will be completely clear and magical to everyone else who encounters it. It also implies that you know ahead of time that it will solve the problem at hand. These are falsehoods. You by definition will not be able to see your blindspots and for you to ignore that and dive into an implementation without driving the design with consumer code (a test) is irresponsible and arrogant.

I like to relate TDD to the scientific method. The scientific method is one of the greatest inventions the human race has ever created:

  1. Formulate a question.
  2. Make a hypothesis.
  3. Determine the consequences of the hypothesis outcomes.
  4. Create a test for the hypotheses.
  5. Execute the test and analyze the results.

What a sorry state we would be in if we didn’t leverage the impartiality of having a test to prove or disprove a hypothesis. Perhaps we’d build buildings and bridges out of inadequete materials based on what ‘we think might be good’ or ‘I just know what will work’. It is completely irresponsible to operate this way. Unless you have a test that goes from red-to-green you have no idea if your solution will address the needs of the system.

This exchange on twitter highlights it well:

Aaron Patterson said it best in his closing keynote at Railsconf:

“Science is important. I can’t believe I actually had to say this.”

TDD also leaves behind a trail of documentation for the various ways in which the system is consumed. This is what I as a developer care about when I want to know how a system works. English documentation is nice but it’s also full of flaws, misinterpretations, and other inadequecies. Code is the only truth when it comes to how the system actually works; it cannot lie or be misinterpreted because it’s executable. When I don’t understand documentation I always look for the tests to show me the examples of how to consume the system.

David went on to say:

“Test-first units leads to an overly complex web of intermediary objects and indirection in order to avoid doing anything that’s ‘slow’. Like hitting the database.”

Not hitting I/O in a unit test will improve the speed of your tests, for sure. But once again he’s completely missed the point. The purpose of avoiding I/O in unit testing is about isolation. The speed gains are important and a nice side effect that allow your feedback cycle to be fast, but speed alone is not the primary motivator. Being able to isolate the code under test allows you to quickly respond to regressions because the only thing that can cause a test failure is the unit being exercised.

As much as I hate making analogies to construction (they are always flawed in some way): Ignoring unit test isolation and allowing your tests to hit external systems is a lot like blindly building an entire building without blueprints, having it collapse, and then asking the question “gee, why did it fall?”. There are too many reasons for the failure and you’ll have to iterate each one before finding the flaw.

If you want to know more about this topic I’d recommend the following 2 books in series:

And if you want to listen to a fair and balanced debate on TDD I highly recommend the following video exchange between Robert “Uncle Bob” Martin and James “Cope” Coplien:


Tagged: software craftsmanship tdd professionalism

2018 Ben Lakey

The words here do not reflect those of my employer.