It’s always amazing to see how people find it hard to compromise, to nuance their opinions about things. And I’m probably the first one finding it hard. But when it comes to software practices, boy it’s annoying.
I was talking to a colleague of mine the other day, we were talking about unit testing and other best practices, and I boldly and shamelessly said to him: “I haven’t written any unit test on this project.†He stared at me with a bit of surprise, then with a bit of disappointment and finally with a bit of something like… disgust!
“What!? No tests, but why? How can you pretend to be a software architect if you don’t practice systematic unit testing?! Oh my god!…â€
My first reaction was a furious urge to hit him in the face. But then I remembered myself a few years back, and I did nothing but leave him in his panic state and disappointment.
When I started working as a consultant, I had already written a few applications in engineering school, but strangely enough they didn’t really talked to us about software quality, agile methodologies and other modern best practices. We did formal programming, Prolog, we reprogrammed a memory management unit and wrote a compiler for a subset of the Java language, but nothing useful ;o).So when I discovered that some people were actually doing things in a rational way, to write better software, I was at the same time impressed and overwhelmed by all of that. Design patterns, continuous integration, test-driven development… I had never seen those in the real world, but I read a lot of books and websites and it seemed so nice that I came to think that it was the only way. And of course I was wrong.Fortunately, I started working on real-world projects, dealing with all sorts of external constraints. And no matter how hard I tried to impose “my†great software best practices, everybody looked at me like an alien because they just didn’t understand why I was talking about all this stuff. So when I was fed up with complaining about no one understanding what should be done, I realized that they were at least right to wonder. Why are things like TDD, CI or Design Patterns so important?
When you read so many articles explaining to you how to write unit tests, they all seem to assume that unit-testing is good in itself, as if it was some kind of axiom. But the truth is that no matter how good they are, writing them and maintaining them is a real pain, right? Lesson learned: when something is painful but good for you, there’s gotta be a better way. But a better way to what?
Because for me that’s the real problem: losing sight of the objective. Unit-testing and TDD are not good in themselves, they are just one best practice which allows you to improve the overall quality and robustness of your code. Why so? Because when you write your tests before the corresponding functional code, you considerably reduce the probability of a bug in the functional code. And robustness comes from the fact that whenever the implementation of a given functionality is modified over time, unit tests are there to ensure that the overall result will remain the same. Fine! But how about a better way?
First, let’s talk about reducing the probability of bugs. The main problem with code is that it’s written by us, human beings. And like it or not, we are faillible, we make mistakes, and the more we think we don’t, the more we actually do. Now rather than testing everything that we write, isn’t there a way to reduce the amount of code that we have to write? I can tell you that: there is a way!
I already see my colleague smiling if he reads this, because I’m so obsessive about this “way†that it becomes ridiculous. Take care, here comes the dirty words: code generation. Of course I’m not talking about the kind of code generation that just displaces the problem elsewhere, that is in diagrams that are the ont-to-one mapping of your code. I’m talking of a higher level of code generation, I’m talking of Model-Driven Architecture.
I know that many programmers don’t like it, have read many articles about and introductions to MDA without ever going further, mostly because programmers like to… program, not draw. And I can understand that. And if you’re in that situation, then you’re stuck with test-driven development… or any other “way†you can find. But if you like to have a better view of the big picture, if you want to accelerate your development and increase the robustness and quality of your code, then have a look at this (my favourite), this and this. You might be surprised.
Will it completely replace unit tests? I don’t think so, but it can create a situation in which you have so dramatically reduced the amount of code to test, that it can be interesting to wonder whether it’s still worth the effort.
My main point is, software best practices are nothing without the goal they allow you to reach, and forgetting that is the best way to stop looking for better ways, to stop innovation and progress.
6 Comments
Iwein Fuld · March 12, 2008 at 4:56 pm
You’re forgetting something: it is not relevant how you’re software is defined/created, the tests are about *verifying that it actually works*.
So you don’t write a test for every method. Fine. If you drag an arrow to the wrong place in your fancy tool and screw up the system not test will show that mistake. Wrong!
I don’t have a problem so much with the fact that you use fancy tools to generate your software, I have a problem with the fact that you don’t provide proof that it works alongside it.
Especially in an environment where some people are working on the code directly your approach is completely naive. Even if you have generated the perfect code, somebody will come along during refactoring and get no warning whatsoever if he breaks it.
The ultimate goal is working software, which is not the same as correct software. Your tools might generate code without bugs, but they might also generate code that doesn’t do what the client expects of it.
In a way similar to what code coverage junkies do, you value the process over the result in your post. That is the kind of behavior that gives software developers a bad name.
Sébastien · March 12, 2008 at 5:44 pm
I see at least two delusions, two things you tend to forget as well:
1- Unit tests are code, and if you write unit tests because there might be some mistakes in your code, then what prevents you from having mistakes in your unit tests as well? My point is: unit tests are by no way a proof of anything. If you want to prove that your code is doing what you expect it to, then you’re stuck with formal programming, something like B for example.
2- Unit tests do NOT test that your code does what the client expects of it, they test that your code does what YOU expect of it, and in my experience, most of the time, developers are the ones who misinterpret what the client wants, so you end up testing correctly something that does not correspond to what the customer needs. And if THAT’s really your objective, then “my” fancy graphical and high-level representation seems more suitable to spot misinterpretation mistakes than a very local and low-level unit test.
My ultimate goal as a software developer is dual:
– the application should do what the customer expects it to do
– the application should NOT do what the customer does not want it to do
Now those are goals, something I tend do, but I also keep in mind that the closer I want to be to these, the more difficult it is to focus on both goals at the same time. IMHO, unit tests can be useful for the second goal, but not very much for the first one. MDA does not pretend to remove all bugs, but at least it reduces the probability of bugs, while allowing you to work with a much higher-level view of your application, which is great for the first goal. Of course, I could combine MDA and Unit tests, and I have seen it done on a few MDA projects. But then the question is, is it worth the time and effort… and pain? For me, the jury is still out.
My only point was to open up the debate, because I’m kind of fed up with hearing that unit tests are the only way to salvation, that if you’re doing unit tests, you’re the good guy, if not you’re bad. That’s dangerous, because software quality is not only about bugs, it’s about business value and usability too. And there are various techniques that you can use to improve the quality of your software. And because all those techniques have costs, your software will never be perfect. And all I’m saying is that the cost of unit tests is sometimes too big, big enough for me to look for other techniques to reach similar goals.
Konstantin Solomatov · July 6, 2008 at 4:25 pm
IMO, moving towards higher abstraction layer with MDA, DSLs or other similar technology doesn’t remove need to write tests. It just makes a user write less but higher level tests.
On the other hand, not all projects require unit tests. If you are creating prototype or a proof of concept application, it’s almost always a bad idea to write unit tests since they slow down experimentation. Why writing unit tests for code if you are 90% sure that you will throw it away. As project and its architecture mature it test stop slowing it down and improve its quality.
Sébastien · July 6, 2008 at 4:34 pm
I still find it hard to figure out what “higher level tests” would look like. But I’m writing unit tests for a small application I’m developing right now, and using JMock is really painful. I feel like I’m just writing the algorithm of my service methods using some kind of awful unreadable DSL (Expectations… ierk). From that point of view, it might be interesting. Are you working on such a test DSL with MPS?
Konstantin Solomatov · July 6, 2008 at 4:42 pm
Tests might be written in a special test DSL for a main DSL.It might provide assertions and other common stuff in a easier to read/write notation.
We have such a DSL for testing language implementation (it will probably be available in the next MPS build). For example in it, you can annotate you program with special notation which requires that a particular expression has a type you set. Another use case is testing of refactorings (like extract/inline method). They have a lot of special cases to make them look intelligent which makes them very error prone. In these tests you specify how code looks before and after refactoring and test checks this with respect to refactoring implementation.
Sébastien · July 6, 2008 at 4:49 pm
Very interesting indeed. I can’t wait to be able to try this. For now, I’m trying to write an enterprise application DSL (entities, services, value objects, etc.) with concepts I’m familiar with in AndroMDA. And making it possible to write tests for those constructs would really be amazing. That’s one of the cartridges I missed the most in AndroMDA: a way to model unit tests using instance diagrams.