The Testing Dogma

When I was in college, we had a very talented physics professor coming directly from Scotland. And as I was in a French engineering school, I was more used to the French way of teaching science, with a lot of theories, demonstrations and proofs. So when I first saw this guy with all his experiments and observations, I thought the man was crazy but in the end, a fact was a fact, and I didn’t really care about why a specific equation applied. I just needed to be convinced it was true… until someone proves us wrong. That’s what science is all about.

Yet, some French scientists tend to be distrustful with the whole empirical stuff, probably because they can’t help thinking that it’s not because you observe something a 999 times that it’s going to repeat itself  a thousandth. So the best alternative we have is to prove it by theory, using mathematics and a lot of tools like theorems, matrices, integrals, etc., even though it’s far more difficult. And once it’s proven with theory, we can more or less take it for granted (provided our reasoning is valid).

Now what if I want to assert something about a piece of software? I can either create an experiment to show that it repeats itself: that’s a unit test! Or we can prove it theoretically using mathematics… Yes, there are formal languages that allow you to prove your programs. They’re very complex to use but hey, reliability has a price!

Fortunately for us, the metaphor stops where the specificity of our case begins, because with software, we have a third option. I won’t say the dirty word right now because I know you’ll run away. Let’s just consider that there are basically three types of testable code:

  • code that is very simple to write but you’re testing it because you want to make sure that no one can break it with a wrong copy-paste or something like that
  • business code which is a bit more complex, for which you have to write equally complex test code, with the risk of having correct code tested by wrong test
  • all the rest, which in my experience is not as much as we think usually.

Based on my experience, I would tend to do the following:

  • I write regular unit tests for ALL the rest.
  • I don’t test complex business code, I do code review on it.
  • For simple code, I shouldn’t even have to write it in the first place. If it’s so simple, then it can certainly be generated automatically by a generator I trust (that’s it! I said it!)

What’s your position really? Set aside the “testing-is-good vs. not-testing-is-bad” dichotomy, what would be your testing strategy? Do you use code generators to avoid testing boilerplate code?