Tuesday, 30 September 2014

Agile's Fifth Element

You have probably heard of the "Agile Manifesto". This was designed to enunciate Agile's fundamental approach by those who coined the term "Agile" software development (see Agile Manifesto).
  • Favor individuals and interactions over processes and tools
  • Favor working software over comprehensive documentation
  • Favor customer collaboration over contract negotiation
  • Favor responding to change over following a plan
Elements, Principles, ???

I'm never sure what to call the Agile Manifesto's 4 main statements. They can't be principles as there are also 12 Agile Principles (see Agile Principles).


I have seen various names, such a guidelines, values or statements. I think Elements is the logical choice as it emphasizes their fundamental importance.

For a long time now I have felt there is one thing missing from the above. It is at least as important as a couple of them. I call this Agile's Fifth Element.

  • Favor simple design over reusability and generality

What is an Agile Element?

To see why we it should be added to the other four we need to decide how a statement qualifies to be an Agile Element. In my opinion an Agile Element needs to:
  • challenge, or at least de-emphasize, (previously accepted) best practice
  • be fundamental to the Agile approach
  • be important enough not to simply be considered an Agile Principle
There are many books on software project management typical of the Waterfall era (mainly written from the 1960's to the 1980's) that emphasize the importance of following strict procedures, having everything well-documented, and having a water-tight contract with the customer. Most of all they talk about the importance of planning a project down to the smallest detail.

I won't go into why these ideas don't work (again), though the authors make them look very good on paper. Luckily Agile came along and turned all of this on its head. The four elements of the Agile Manifesto clearly highlight what was (and often still is) wrong with what was considered the correct way to develop software.

How Does it Qualify as an Agile Element?

Let's look at how the proposed fifth element satisfies the above criteria by looking at each criterion in turn.

Does it challenge accepted best practice?

The big paradigm to come out of the 1980's was the emphasis on reusability. Many in the Agile world now think that this emphasis was a mistake as I previously explained in Reusability Futility.

In brief, experience has shown that too much software is designed to be flexible and reusable for no purpose. This leads to increased complexity to the detriment of other attributes such as understandability, verifiability, efficiency, etc. The worst effect is that software becomes harder to maintain and cannot adapt to change.

OK, but why is it fundamental to the Agile approach?

Keeping things simple is part of the agile approach. In fact the 10th Agile Principle says:

Simplicity -- the art of maximizing the amount of work not done -- is essential.    

Simplicity is necessary in order to create small, frequent software releases - a major part of the Agile approach. Further, keeping things as simple as possible allows the software to more easily evolve, since unnecessary constraints interfere with continual refactoring -another fundamental feature of Agile development.

designing for flexibility
adds complexity
Of course, you may ask: can't you design software to be both simple and flexible? Sometimes you can find a design that is simpler and more flexible than any alternative but usually there is a trade-off, particularly as modules become larger. The overwhelming general experience is that designing for flexibility adds unnecessary complexity.
  
Is it really important enough to be the Fifth Element rather than just a principle?
  
Yes, it is! It counters one of the biggest problems currently facing software development - the tendency of developers to over-design and add unnecessary code. There are lots of reasons this occurs such as:

  • temptation to add something now which is "sure to be needed later"
  • desire to use some new design pattern that one has recently been read about
  • hope that a general purpose module can have use in other projects
  • use of particular new technology to have on your CV
  • desire to avoid future code modifications

These reasons, and their consequences, are further elaborated in Gas Factory Anti-Pattern.

If you're still not convinced I recommend reading Chapter 18 of the excellent book 97 Things Every Software Architect Should Know. It is written by Kevlin Henney and entitled Simplicity Before Generality, Use Before Reuse. It says exactly what I have been trying to get people to understand for many years.

Why Is It Missing?

You would have to ask the authors of the Agile Manifesto why they left it out. I suspect that its importance was simply not appreciated at the time.

I guess the first four elements highlight problems with best practice that have been promulgated since the 1960's. The proposed fifth element highlights the problem of something that only became considered best practice later (1980's) - that is designing software for reusability.

Another theory is that many of the creators of the Agile Manifesto were heavily involved in OO (Object-Oriented) technologies. To de-emphasize reusability would be anathema to them, since reusability is a fundamental part of OO design.

A Different Fifth Element Proposal

As I write this post I have discovered that our dear old friend Uncle Bob (Martin) has also proposed an additional element for the Agile Manifesto:
  • Favor craftsmanship over execution
In essence he is saying that many developers are only concerned with getting the code working without regard to the quality of the code they produce. I agree with Uncle Bob that this is a serious problem that affects much, if not most, software development.

But Uncle Bob's proposal, though eminently worthwhile, does not qualify as an Agile Element for the reasons mentioned above (see What is an Agile Element?).

First, it does not question accepted best practice since software quality experts have been emphasizing the importance of various quality attributes of software (such as maintainability) for decades. I also do not believe it is fundamental to the Agile approach, though many Agile practices, which could be considered as promoting craftmanship (such as pair programming), will improve code quality.

Summary

My proposal is for an additional element to be added to the existing four elements of the Agile Manifesto. I consider this new element to be a least as important as a couple of the big four - personally I would rank it third. I believe it to be fundamental to the Agile approach as it is practiced today. For example, many practitioners of XP (Extreme Programming) talk about the importance of YAGNI and simple design (see Simplicity is the Key). Similarly in Scrum, developers focus on solving current customer needs by concentrating solely on the most important backlog items.

... software is designed
for general purpose
and, ironically,
ends up being fit
for nothing at all!
The problem of unnecessary generality and reusability is largely unrecognized, but significant and growing. I believe it is mainly due to the training and mentality of developers, particularly software architects. Perhaps with the best intent, software is designed for general purpose and, ironically, ends up being fit for nothing at all!

Far better is to start with a simple design that can be easily modified (with the assistance of Unit Tests) into whatever the future requires. Without the burden of unnecessary complexity the software may be able to evolve forever or at least not become a "legacy" application almost immediately as most software tends to do.

So do you think this is a worthwhile proposal? I would be interested to hear your comments.



Sunday, 25 May 2014

Agile Design

There is one aspect of Agile which people have trouble coming to grips with. I talked a little about this recently when I mentioned JIT design (see JIT). You may have heard it called other names like evolutionary design, incremental design, emergent design, continuous design, etc. I will use the term Agile Design or JIT Design.

Some people are adamant that designing like this is fraught with danger or just doesn't work. However, there are many, many projects where it has worked better than anyone expected and certainly better than the Waterfall approach of Big Design Up Front (BDUF). For example, see Continuous Design by Jim Shore.

The opposition is simply due to not really understanding how design is done in Agile development. In particular, using JIT design without using other aspects of Agile methodologies such as regular feedback, collaboration, continuous integration, collective code ownership, etc, will doom it to failure. And as usual I will mention that Unit Tests are crucial (see Unit Tests).

How it works as part of an Agile methodology is hard to understand so I will use an analogy. Hopefully, this allows you to understand my point.

An Analogy

I will use the analogy of designing a house. I chose this analogy as you probably already have an idea on how that's done.

Houses are built from a plan. Often an architect has spent many weeks or months creating a design on paper. Even the worst houses are built from some sort of floor plan. (In fact, in my country, government regulations preclude building without a detailed plan that has been approved by the local council.) The point is that for any house, a lot of time is spent in design and planning before construction can even begin.

Over many years (even millenia) this has been refined to be the best way to build a house. In fact this is essentially the same way that all complex physical devices and structures are built - some sort of designer, architect, or engineer (or team thereof) creates a blueprint which is carefully perused, verified, and approved before the final product is actually built.

If you just went and bought some materials and started building a house do you think it would be anything other than a disaster? Of course not!

So why do I think that designing and building software should be any different? The simple fact is that you can't build a house or software in the conventional manner without a fairly detailed design. However, if you change the way you build the software things are different. This is why Agile software development is considered to be a methodology - it only works when you understand and embrace it, in toto.

Let's see how we might build a house in the Agile manner. Note that, in the discussion below, D stands for developer (the house creator) and C stands for client (the house user).
From this ...

D: What is your most basic requirement for the house?
C: I want something that stops me getting wet when it rains.
D: Try this. [A piece of corrugated iron on four posts.]
C: I keep bumping my head on it.
D: How's this? [The corner posts have been extended.]
C: Better. What about something to stop the wind.
D: Try this. [Four walls attached to the posts.]
C: It's less breezy but I can't get out.
D: I could add a door. ...
to this!

This continues until C has their dream house (or they can't afford to pay for any more changes).

Software is different

I hope you realize that I am not advocating this way of building houses! This would be a particularly slow and expensive way to do so. However, when building software this can be a better way (and I would argue almost always is). Building software is different for these reasons:

1. Modifying physical objects (like extending the above-mentioned corner posts) can be time-consuming and expensive. Software is far more malleable. To make something larger may be a simple matter of changing a constant and recompiling.

2. Continually changing the design of anything of any complexity quickly becomes a nightmare of managing and controlling the interactions. With software this is a little easier because you do not have to worry about physical interactions (such as the location of pipes and wiring). However, it is still important to decouple different parts of the software whenever there are no major drawbacks to doing so (see DIRE). Crucially, Unit Tests can be used to ensure nothing is broken as the software is continuously redesigned and refactored.

3. House design is well-understood and there are many standards and conventions that are well-known, proven and followed. Software design is far more variable. Moreover, the client does not have a very good idea of what they want. Even if they did the developers do not have a very good idea of how to give it to them until they try to build it.

4. For practical reasons design constraints for houses do not change much if at all. For example, the government is not going to one day say that all electrical wiring in all houses is to be changed immediately. (Even if there were an urgent safety issue this would be logistically impossible.) However, these sort of major changes are regularly thrown at software developers. BDUF cannot easily cope with change the way continuous design does.

The Argument For BDUF

I have often read the blog of Joel Spolsky, whom I respect and admire greatly. However, many years ago I read a post which saddened and disappointed me. Part of Joel's blog is even quoted on the BDUF page on Wikipedia as an argument in favour of BDUF.

However, if you carefully read the full paragraph from which the quote is taken, Joel's example is actually an excellent argument for Agile Design. Here is the full paragraph:

"Many times, thinking things out in advance saved us serious development headaches later on. When I wrote the first draft of this spec, I had a more complicated flowchart that assumed that either party (helper or victim) could initiate the process. This became extremely complicated and convoluted for various reasons. As I worked through the screens that would be needed to allow either party to initiate the process, I realized that Aardvark would be just as useful, and radically simpler, if the helper was required to start the whole process. Making this change in the spec took an hour or two. If we had made this change in code, it would have added weeks to the schedule. I can’t tell you how strongly I believe in Big Design Up Front, which the proponents of Extreme Programming consider anathema. I have consistently saved time and made better products by using BDUF and I’m proud to use it, no matter what the XP fanatics claim. They’re just wrong on this point and I can’t be any clearer than that."
- Joel Spolsky, 2006

The first thing that hit me is the mention of the first draft of the spec with the "complicated flowchart". It is obvious that the first draft is already suffering from the problems commonly associated with BDUF. In particular, it is clearly making unnecessary assumptions.


Flowcharts      

Does anyone use flowcharts anymore? Back in 1979 I was told by my lecturer to avoid flow charts (probably due to their association with the evil goto). Personally, I have found UML activity diagrams (and sequence diagrams) to be more useful.


But using Agile Design obviates the need for lots of diagrams.
Luckily (for Joel) he had the experience and insight to be able to identify a problem in the spec and fix it before it made it into the code. Apparently in his development environment code was hard to modify since a change to the spec that takes an hour or two would require weeks of code changes. (Note that this is another symptom of not using Agile techniques such as Unit Tests.)

Now imagine if the designer(s) of the software had less time, less experience or less insight than Joel. They may have gone with the first draft of the spec with the consequent later weeks of recoding that this entailed. This is plausible as I doubt many designers are as experienced and insightful as Joel. So rather than demonstrating why BDUF is better than Agile, Joel's example clearly shows its weakness.

Now consider the Agile approach, which is to first create a minimal working version of the software. In this initial version it might not have even been obvious who (helper or victim) could initiate the process. Users would try the software and provide feedback which is used to evolve it into what they need. In this way the conundrum of who should initiate the process would probably not even have arisen.

Another thing to note is how Joel simulated the use of the software in his own mind by "working through the screens". Not many people can do this effectively. It's always far more effective to have working software than try to imagine how the software would work. (Note that this is another advantage of Agile element 2: Favour working software over comprehensive documentation.)

Finally there is one thing I need to be absolutely clear on: thinking things out in advance is not the only way to avoid serious headaches later on. And most of the time it's not even the best way.


Problems of BDUF

The plain fact is that for more than 50 years BDUF has been tried and it consistently fails. Even for simple, well-understood projects, no matter how hard you try, it's impossible to produce anything approaching a good design without attempting to implement that design. I have seen this time and again. Trust me on this.

"What?" I hear you say. "You can't say that without an explanation." Well, I can if I want to, so there! Actually, I don't need to, since it has been discussed elsewhere. For example, see the Wikipedia page.

I have also touched on the subject many times in previous posts in this blog. For example, it is the principle cause of the Gas Factory Anti-pattern. More recently, I have talked about it in the series of posts on Unit Tests and specifically in the Cost of Rework section in JIT.

But just because you asked so nicely, here is a summary of recognized problems with creating a detailed specification up-front:
  • the customer often does not really know what they want, at least till they try it
  • it does not adapt to changing requirements
  • it tries to anticipate future changes, adding unnecessary complexity
  • it tries to document every known "fact" (of which many are usually incorrect)
  • it makes unnecessary assumptions about how the design will be implemented
  • it is very hard to foresee problems
  • many things (often major things) are overlooked
  • it takes too long to create - stakeholders become worried when there is little sign of progress
  • it separates design from coding, even though they are very closely related (see DIRE)
  • it results in a contorted implementation - it's often easier to change the code than fix the spec
  • the document is not updated when the inevitable changes are made
There are probably more, but those are the ones that occur to me now.

However, now that I think about it there is one major drawback that I have not seen discussed. This is that developers don't really come to a full understanding of what the software is supposed to do until they actually try to implement it. It's only then that logical inconsistencies, omissions and other mistakes in the design become obvious. Further, limitations (unknown to the designer and often even to the developers) only become apparent when you attempt to write the code.

Possibly most significantly, only when the developers implement the design do better ways to do things occur to them. I have lost count of the times with BDUF when I have finally completed the code, only to then think of a much better solution (simpler, more efficient, more usable, more maintainable, etc).

Agile Design

The Agile approach is only to design what is needed for the next feature or iteration. This is usually the smallest possible addition to the system. The designer should only concentrate on the current iteration and never consider any possible future changes. This is a lot harder than it sounds. (I do concede that occasionally there are exceptions to this rule which I will discuss later.)

Since past iterations did not look to the future, a particular new feature may completely invalidate a large part of the existing design. Don't panic! This is normal. Traditionally such frequent refactoring would create chaos, since every iteration would have the distinct possibility of breaking existing functionality. Without complete regression testing after each iteration bugs would creep in and not be found till some time later

This is why Agile Design requires use of Unit TestsUnit Tests allow continual redesign and refactoring to be performed without fear of breaking existing functionality.

The advantage of the Agile approach is that the closely related activities of design and coding are done consecutively and, to some extent, simultaneously. As I discussed previously in DIRE separating the design and coding leads to the major problems of Waterfall development.

The one huge benefit that I have found is that, after developing earlier features, the developers are very familiar with how the software works. This greatly improves the chances of a better design being developed for the next iteration. Mistakes and omissions are far less likely and a good (simple, DRY, etc) solution is much more likely.

B2DB Syndrome

Every programmer experiences the B2DB (Back To Drawing Board) syndrome when they are first learning to program. This is where one tries to write some code but in doing so obtains a better understanding of how to do it. The consequence is one has to go back to the drawing board, often many times. (I discussed this in more detail in JIT Testing).

For many this is a very painful experience and they believe there must be a better way, especially (as is often the case) they have already been trained in another field (such as a branch of engineering) where it is essential to do up-front design.

I believe this is the real reason for much opposition to Agile Design - it feels like the bad old B2DB syndrome.

It's true that there is a lot of refactoring going on in Agile Design, but this is actually an advantage. Of course, more experienced developers are going to have a better initial design, but even the best will see better ways to do things once they fully understand what they are trying to do. Also (as all the Agile experts emphasize) it makes the process much more amenable to changing requirements.

Finally, at the risk of sounding like a broken record, I will mention that Unit Tests make the refactoring safer and easier. Rather than having the feel of dread of B2DB, you can actually learn to enjoy refactoring and the process of creating the best design you can find.

Summary
  • only make the simplest possible enhancement that will work
  • forget the large design document with lots of UML diagrams
  • use the code (especially Unit Tests) as the "documentation"
  • don't design for the future, concentrate on what is needed now
  • don't make the code flexible, just write for the specific task at hand
  • try to forget all the extra (irrelevant) information that has been given to you
  • don't be afraid to refactor existing code
  • use Unit Tests to allow changes to be made without fear of breaking anything

Sunday, 13 April 2014

DIRE

The most useful ideas in software design can be applied in different ways and in different settings. They can save a lot of time and trouble if you can learn to recognize when they are appropriate. Otherwise, you can try to keep them in mind when designing and building software.

One such idea is the DRY principle (see the DRY section in Software Design). It simply says that keeping an eye out for duplication (in its many subtle forms), and trying to eliminate it, will save you a lot of time and worry in the long run. The main advantage is that it is easy to remember and, with practice, easy to identify and apply. The DRY acronym was invented by Thomas and Hunt (see The Pragmatic Programmer). Of course, the idea behind it has been around a long time but Thomas and Hunt elucidated and promoted it well.

Unfortunately DIRE is a 4 letter acronym. I know the fashion nowadays is for TLA's but this was the best I could do.
DIRE is an acronym that I invented for another idea. It stands for Don't Isolate Related Entities. It basically means that you should make every effort to keep things together that belong together. There is a close relationship between DRY and DIRE -- DRY is really a special case of DIRE where the "related entities" are so closely related that they can be partially or totally combined.

DIRE may seem obvious but many things conspire to defeat it. It is not always apparent how or why "entities" are closely related. Often related entities are separated for technical (and even non-technical) reasons, or simply by accident. I give examples below to show the broad range of areas and how you might apply it.


Origins


I guess everyone has had the experience of being interrupted in a sequence of tasks. When you come back later you've forgotten most of the context. Continuing where you left off is tedious and error-prone. In particular, interrupting developers a lot is a great way to generate bugs.

Like everyone I quickly discovered this when I started programming professionally (about 3 decades ago). However, it was not until about 2 decades ago that I began to think of this as an example of a more general principle. This was precipitated by my first encounter with JIT.

I discussed JIT in detail last week but in brief JIT is a QA technique where things are done "Just In Time". JIT is particularly useful in software development to:
  1. Increase visibility making it easier to find problems in the development process
  2. Reduce costs of delay (such as interruptions as discussed above)
  3. Reduce costs of rework (such as redoing design and testing)
SPACE vs TIME

JIT is an example of not isolating related activities in time. I guess in that case DIRE actually stands for Don't Isolate Related Events.

There are also many examples of not isolating related things physically (ie in space) - from the order of lines in source code to the seating arrangements of developers.
The cost of rework has by far been the biggest problems in traditional software development. For example, a large up-front design can mean a lot of rework as the design needs to change. But typically, to avoid the cost, we somehow find a way to live with a bad design, with many undesirable consequences which I explained a few months ago (see Change).
JIT
is an example
of DIRE

Gradually developers are realizing the advantages of JIT, and hence DIRE, since JIT is an example of DIRE.  For example, many Agile practices (such as JIT-design and TDD) avoids the costs of rework by applying the idea of JIT, and performing related activities more closely in time.


Examples


Coding

A good example of DIRE as applied to code is the declaration of variables. Separating the declaration of a variable from its use is a common source of bugs (as I mentioned in point 8 of this Code Project Article). This is why C++ improves on C by allowing variables to be declared anywhere not just at the start of a compound statement.

Many Coding Standards work against the advantages of DIRE, by prescribing:
  • order of declarations in a header file
  • order of functions in a source file
  • order of members in a class

Design

Most design practices and patterns do not intentionally contravene DIRE. However, they almost invariably promote decoupling in some form or another. Decoupling by it's nature tends to separate things. Though essential in almost every design, decoupling can be overused or misapplied.

Often a better design can decouple in a different way and not isolate closely related things - a good example is provided by the Layer Anti-Pattern. Sometimes a particular design pattern is appropriate but a cumbersome interface can make communication difficult between two closely related objects. Finally, sometimes complicated design patterns are simply not necessary but are used due to a misunderstanding or zealous desire to try something new.


Management

Managers essentially do two things:
  1. help employees to achieve the goals of the organization
  2. show that the work is being done
The first of these is very much about using the DIRE principle; but the second can work against DIRE.

First, helping employees is mostly about removing roadblocks. Roadblocks take many forms but are essentially things that prevent people doing their job efficiently and effectively. Removing roadblocks simply means getting rid of something that isolates related entities - such as a developer and the information they require to do their job.

Another part of helping employees is to get them working towards a common goal. This is achieved by fostering communication between them - another good example of DIRE where the related entities are the team members.

The problems occur when a manager allocates tasks and tries to monitor the work being performed. To do this a project must be divided into smaller and smaller tasks. The problem is that dividing tasks, by definition, creates divisions. If the way that tasks are divided is badly done then related tasks are separated in time or done by different people when they should not be. The classic example of this is the Waterfall development methodology as discussed in the next section.

Finally, some managers directly contravene DIRE for their own self-interest. This is why developers are not allowed to communicate directly with users or why they are only given just enough information to do their job. By controlling access and knowledge managers retain power which reduces the likelihood that it will be discovered that they are unnecessary, or not as important, to getting the work done as they like to believe. Of course, this sort of behavior is bad (but still common in the software industry) since it has DIRE consequences.


Process

Development methodologies are one of my pet topics (and the reason for this blog). Here also there are good examples of the benefits of DIRE.

As I mentioned above the best (or worst, depending on how you look at it) example of this is the Waterfall development methodology, which tries to control and monitor the development process, but in the process isolates closely related tasks. The processes of designing, coding and testing of a particular features or piece of software are very closely related. The Waterfall methodology separates them (and not just in time) by splitting the project into phases. This contravention of DIRE is the reason for all the problems with the Waterfall methodology and the main reason for the invention of Agile (as explained below).
the Layer Anti-Pattern in design is analogous to the Waterfall model of development
It's only when you think of these things in terms of DIRE that an interesting analogy between software design and the development process becomes apparent. Splitting development tasks in the wrong way causes DIRE problems. In software design, as we saw above, splitting the design into modules in a bad way causes similar problems. A specific example is the analogy between the Layer Anti-Pattern in software design and the Waterfall model of development - splitting a design into a stack of layers is like splitting the work into a series of phases. In both cases the separation causes problems by isolating closely related entities.


How Do Related Entities become Isolated?


The whole point of DIRE is to bring together things that should never have been separated. But was there a reason for this separation in the first place? Sometimes things are separated by accident but usually it is done for a reason. Often there is a trade-off, where DIRE is sacrificed for some other perceived benefit such as reduced complexity (as in the Layer anti-pattern) or improved monitoring/control (as in the Waterfall methodology).


Divide and Conquer

The most common reason for isolating things is to reduce complexity, using the principle of divide and conquer. This is fundamental to software design and expressed in such terms as modularity, information hiding, decoupling, etc (see Software Complexity).

Although important, it can be used improperly. For example, how software is divided into modules is often done poorly. On the other hand a split *may* be appropriate but the method with which a module communicates (ie, its interface) may be too limiting. And sometimes different parts of code are separated prematurely or for no good reason at all.


Planning

Talk to almost any manager and they will say "You can't achieve anything without a plan". In fact many will believe this with every ounce of their being. I guess this is natural as planning is their raison d'etre. However, it is an exaggeration at best. In fact, there have been many worthwhile achievements in human history which had no plan.

Don't get me wrong - I think a little bit of planning can be a good thing. However, the tendency is to divide a project into many small tasks. Each such task has a well-defined deliverable so that the manager can closely monitor progress to determine if the project is behind schedule.

Unfortunately, detailed planning can interfere with actually achieving team goals. By dividing the project into tasks the implication is that each task can be completed independently, which can result in closely related activities being isolated. Even at a high level dividing a project into stages means that work stops after each major milestone, then gradually builds up before the next deadline. This stop-start work-flow isolates related tasks and leads to poor continuity.


Playing Safe

A common reason for separating related entities is avoiding risk. JIT is a perfect example -- when JIT is used in manufacturing the process becomes crucially dependent on input parts arriving on time. Hoarding inventory is safer but introduces the problems I discussed last week.

In software development a good example of the disadvantages of playing safe is demonstrated in the traditional way of designing software. Here the design is supposed to be fully specified and signed off by the customer before coding is begun. This allows for the developers to protect themselves if  when things go wrong and the customer becomes unhappy. Their usual refrain is "It's not our fault, you signed off on the spec."


Much better results are invariably obtained using the Agile approach of JIT-design (see below). Doing things the right way usually involves more risk which is the reason XP promotes the value of courage (see Extreme Programming). 

Retaining Control

Another, more insidious, reason that managers like to isolate, is to retain control. If developers start talking directly to users then the work may begin to get done without even involving the manager. And since knowledge is power then information is doled out on a "need to know" basis. If you have a boss like this then my advice is simply to look for a new job.


Over-design

Similar to the problem of over-planning is the problem of over-design, exemplified by the Gas Factory Anti-Pattern. Martin Fowler gives a very reasoned account of this in Is Design Dead?

Accidental

Sometimes things are separated for no good, or apparent, reason. This is a problem to especially watch out for as a design evolves.

Agile


DIRE  is at the core of the Agile methodology. Consider the four elements of the Agile Manifesto and how they exemplify DIRE...

1. Favor Individuals and Interactions over processes and tools.


I have always found this rule a bit vague, but I think it is about getting the team working together, rather than following some safe, bureaucratic procedure. Breaking down barriers and encouraging communication and brainstorming is a great example of DIRE.


2. Favor working software over comprehensive documentation.


This is a critical part of Agile and nowhere is it better demonstrated than in the use of Unit Tests. Unit Tests are working software. Moreover, they can work as a substitute for massive amounts of design documentation. In fact they have several advantages over design documentation (see the Documentation section of Unit Test Advantages).

How does this relate to DIRE? Well the problem with a design document is that it is isolated from the "real" design - ie the actual implementation. The only link between the two is through the mind of the developer(s). This has forever been a major source of problems since people misinterpret and misunderstand.
Unit Tests avoid this problem. Using Unit Tests as a substitute for design documentation allows you to take advantage of the DIRE principle. As documentation Unit Tests are directly connected to the code they are designed to document and test.

3. Favor customer collaboration over contract negotiation.


This is similar to rule 1. It is about getting developers to communicate and work with the users. As I said above, contract negotiation is symptomatic of playing safe.


4. Favor responding to change over following a plan.


Responding to change is at the heart of Agile development. Most Agile principles and techniques use DIRE to quickly and more easily respond to change:
  • JIT-design brings the design process and corresponding coding together
  • continuous integration is about bringing different modules together early to detect problems earlier
  • TDD brings coding and corresponding testing together to find bugs faster
  • getting developers and users/customer to work more closely together allows quick response to change
  • giving users working software earlier finds problems faster

Disadvantages of DIRE


There are no inherent disadvantages of DIRE that occur to me. The main problem is that it may be a matter of opinion the best way to split up a task or problem, such as the design of some software.

For a large problem, we necessarily have to split it up to reduce complexity. Using divide and conquer we can usually isolate and solve sub-problems one at a time until the whole problem is solved. This is the most fundamental principle in software design, and in most technical problems.  See my first post.

As an example, consider the Waterfall development methodology, which splits the process of creating software into phases (design, coding, etc). There are obvious advantages into doing the design of all the parts of the software at the same time, then all the coding together, etc. Doing all the testing at the end should avoid a lot of regression testing. However, experience (and awareness of DIRE) tells us that isolating the design of particular part of the software from the coding, and isolating the coding from the testing causes many problems.


Waterfall Methodology
The Agile approach is to divide the problem up in a very different way. First, quickly get a minimal working version of the software, then add to it over time. I have previously discussed the advantages of this evolutionary approach, ad nauseum, so I won't repeat them here.

Agile Methodology
The disadvantage of the Agile approach is the separation of related activities that occurs.

It's easier to design all the features at the same time since when you go back to design the next part you may have forgotten a lot of the original design. Similarly, most programmers will tell you that it is faster to implement Features 1a and 1b (of the above diagram) together. Finally, there will be a great deal of testing rework, since testing Feature 1b will probably entail retesting feature 1a and the core system again. 


Unit Testing

Fortunately Agile techniques, particularly Unit Testing, come to the rescue in alleviating all these disadvantages:

  • Unit Tests act as design documentation making it easy to do design and re-design at any time
  • Unit Tests make it is easy to add new features without fear of breaking existing behaviour
  • Much testing is automated with Unit Tests which means the re-testing is automated

Summary


DIRE is a simple principle that helps to create better software. It may seem obvious but keeping it in mind allows you to find better approaches to many problems. For example, DIRE is behind most of the principles of Agile.

Using a divide and conquer approach is natural and essential to most tasks and problems, but sometimes how something is divided may cause undesirable consequences. However, by careful application of the DIRE principle it is often possible to reduce the "isolating" effects of the division. Further, looking out for things that should not be separated may allow you to see a different way to do the division. A good example of this is the Layer Anti-Pattern.

Sunday, 6 April 2014

JIT

I first encountered JIT (short for Just In Time) when doing a course on SQA in 1993. JIT is an idea that originated in the Quality Assurance movement in Japan, in the area of inventory control in manufacturing (Toyota?). JIT was an idea that went against conventional thinking, which was to ensure you always had more than enough inventory (eg, of inputs into a manufacturing process). JIT took the opposite approach which was to have as little inventory as possible (or none at all) by relying on product arriving when needed (ie, just in time).
DIRE   

JIT is an example of the broader principle of DIRE which I will discuss one day soon.

The obvious advantage is that inventory costs are reduced. But there are other, not so obvious ones so let's list all those that I can think of:
  1. Reduced storage costs for inventory
  2. Greater visibility in the supply chain - this allows problems to be quickly found and fixed - see Visibility (below)
  3. Parts are less likely to have deteriorated (rusty, stale, etc) - see Cost of Delay
  4. Parts are less likely to be obsolete when needed - see Cost of Rework
In the software industry the first advantage is not at all important - a little bit of disk space costs a lot less than a warehouse full of car parts. However, the others are significant so I will give some examples.

Visibility

Traditionally software is created by building low-level components then gradually assembling them into more and more grandiose structures. Though design is usually done top-down, implementing the design is done bottom-up.

There are several problems with this approach. The main one, in this context, is that the creation of a low-level component will be isolated from its first use. Moreover, the initial use is unlikely to exercise all the features of the module. [Another problem is that the low-level module is typically over-designed and far more complex than is necessary for its intended purpose - see Reusability Futility.]

In other words a module is not even tried in a realistic environment until some time later, typically never fully used as intended. Eventually it's used and realized that it is too slow, has many bugs, or just doesn't do what's required. By then it is much harder to fix. In some cases it is easier just to start again.

A better way is to build everything at once. How can you possibly do that? It is usually fairly easy to build the minimal piece of software that actually works, without modules and just a basic design. Later, we can split parts off into modules and evolve the design. If it is truly minimal then every bit of the code will be used to satisfy the initial acceptance test. That way you know there are no surprises lurking in some piece of code that has not been exercised yet.

Cost of Delay

In manufacturing the JIT approach avoids the costs associated with parts deteriorating over time - what I call a "cost of delay". In software development the main cost of delay is due to forgetting how things work. As we saw above, introducing a delay between two "tightly-coupled" tasks interrupts one's train of thought.

This is a major problem in debugging. It's much easier to fix a bug if it's discovered while you are still working on the code. Fixing bugs months, weeks or even days later is tedious, time-consuming and error-prone. Further, you might have to do thorough regression testing to ensure that the change does not introduce new bugs - which is a cost of rework (see below). Also see (JIT Testing) for more on why it is better to test ASAP.

Cost of Rework

The design of manufactured goods changes occasionally to address design flaws, for reasons of fashion, and even to add enhancements. For example, if it is found that two parts do not fit together reliably then a change might be made to the way they are joined, such as adding a longer thread to joining two pipes. If the manufacturer has thousands of one of the parts in their warehouse then these will have to be thrown away and remade to the new specification.

In software development the problem of rework is much worse because things tend to change often. If you do something before it's needed you will probably have to redo it. I will now use the example of designing software (but the problem of rework can be found in other areas of software development such as the need for regression testing).
the "best" solution
to the problem
evolves
as developers
have more time
to consider alternatives

One of the most serious problems with the Waterfall methodology is the separation of the design and coding phases. Usually when it comes to implement the design it is no longer even close to what is really required. There are many reasons for this which I discussed in detail last October (see Change). In brief, the problem to be solved (ie, what is required) evolves over time as the customer gains more insight. Furthermore, the "best" solution to the problem evolves as developers have more time to consider alternatives.

Proponents of the Waterfall model say that the design is not cast in stone. Even so, the problem is that to maintain the design document according to the current best solution requires a continuous rework. Moreover, it is human nature for the designers not to want to throw away all or part of their original design upon which they labored so intently.

So with a large up-front design the alternatives are:
  1. put a lot of effort into maintaining the design which requires to a lot of rework
  2. put up with a bad design which causes major problems as the code is modified
Generally, in traditional waterfall projects the second option is used, not by any conscious choice, but because it is easier. This means that the design is obsolete before the code is written. In fact parts of the design may be obsolete before the design is finished. In Agile development this is addressed by postponing design until it is required - sometimes called JIT-Design.

Summary

The advantage of JIT is in bringing together activities that are related. In the manufacturing industry (where it was invented) this means bringing the activity of producing a product closer to the activity of producing its component parts.

In software development it is similar but it can be applied in more ways. One of the most important is taking the Agile approach of not having a separate design phase followed by a coding phase but instead using JIT-Design. A similar one is to test code as soon as possible, of which TDD is the extreme example.

JIT is one example of a more general principle that I have called DIRE (Don't Isolate Related Events or Entities). I will discuss DIRE in detail next week.

Sunday, 16 March 2014

Unit Tests Summary

I have written a lot about Unit Tests in the last few months. It's time to summarize.

Agile

The reason I have talked so much about Unit Tests (recently and in the past) is that I believe it is the core enabling technology of Agile methodologies. Trying to do Agile without Unit Tests is like trying to build a ship without a hull - it just doesn't work. (This is one thing I don't like about Scrum - it does not require any technical procedures like Unit Tests.)

One of the guiding principles of Agile is that the system evolves in response to change. Unit Tests allow that to happen by:
  • detecting bugs immediately after changes are made
  • giving developers the confidence to make changes properly
  • avoiding the need for a lot of regression testing which would kill frequent changes
  • avoiding all of the problems that makes software hard to modify
  • allowing the initial design to start out simpler - avoiding unnecessary complexity (see Reusability Futility)
One important point that is often missed: responding to change is about a lot more than how the software behaves. Just as important is the ability to refactor the design to take advantage of new ideas, knowledge and technologies, without even affecting the external behavior. And, Unit Tests allow you to fix bugs in a way that does not compromise the original design.

Design

There is a lot of evidence that using Unit Tests results in a better initial design.
Unit Tests allow
you to...
take advantage of
new ideas and technologies

Further, without Unit Tests the design of the software will degrade over time (see Why Good Programs Go Bad). With Unit Tests changes can be made properly, so the design does not need to degrade.

Finally, the design can even improve over time, since Unit Tests make it easy to make changes without fear of introducing new bugs. As software is developed and evolves it is inevitable that better ways are found (new algorithms, new hardware, new third-party libraries, etc). Unit Tests allow you to refactor the code to take advantage of new ideas and technologies.

Documentation

There are other advantages to Unit Tests (see What's So Good About 'Em). A major one is that they act as a substitute for (and improvement on) design documentation. Unit Tests have these advantages over written documentation:
  • documents have mistakes - Unit Tests always agree with the code
  • Unit Tests are never out of date as long they are maintained and run often
  • verifiable - just run them to check that they agree with the code
  • more easily written - since developers like coding not writing
  • more easily understood by other developers
  • if not understood you can always step through the code in the debugger

Implementation

Of course, creating Unit Tests is not without its challenges. First of all it can take some time, especially if you need to build test doubles. Here are the major points on creating foolproof tests:
  • the system being tested must have a good, verifiable, modular architecture
  • tests are written by the developer at the same time as the code being tested
  • tests should use knowledge of the implementation (see White Box Testing)
  • but tests should not depend on the implementation (only the interface)
  • tests should be complete (use code reviews and a code coverage tools)
  • tests should only test one concept (see Unit Tests - Best Practice)
  • tests should be short - move set-up and tear-down code to sub-routines
  • tests should be independent (of each other and the environment)
  • use test doubles for any external dependencies that may cause problems
  • use test doubles to simulate errors and other hard to reproduce situations
  • don't use a test double if the emulated module is OK (fast, reliable, etc)
  • tests should be easy to run and fast - and run often
  • use TDD to avoid many of the problems encountered with Unit Tests

Saturday, 1 March 2014

Arguments Against Unit Tests

Here are all the arguments against Unit Tests that I have ever heard (and a few new ones I just discovered). Many of these I believed myself in the past but have since found to be wrong or exaggerated.  I have tried to refute most of them. I do concede that, in a few rare cases, Unit Tests are not useful.

I wrote some Unit Tests but they did not find any bugs at all. It was a waste of time.


Unit Tests
are not about
finding bugs!
This is a common trap that I initially fell for too. Unit Tests are not about finding bugs! - at least not initially. The real advantage is that you can check at any time that there are none. Plus, Unit Tests also have many ancillary benefits such as documenting how the code is supposed to behave. (See here for more on the benefits.)


Our framework makes it impossible to use Unit Tests.


It's true that many frameworks and languages are not Unit Test friendly. Luckily this is changing. If you have an existing application with this problem then maybe you can't use Unit Tests as much as you would want (see next argument). If you are creating new software, make sure the language and other software you use is amenable to Unit Testing.

We are trying to add Unit Tests to an existing application. The code has no obvious design. We can't work out what to do.

Unfortunately there are times when it is too difficult to create Unit Tests. The reason is nothing to do with what the software is designed to do and certainly not a limitation of Unit Tests per se. Instead, it is caused by a bad design.

Our code interacts with web services, databases, servers, hardware, etc. It's impossible to create Unit Tests for this sort of code.


It's not impossible. It may be hard or take some thought and effort. On the other hand you may be able to use tools like a mock library, whence it may be much easier than you think. Even if it does take a lot of time, it is worth it.

The software is highly user-interactive so we can't use Unit Tests.

It may appear to be mostly GUI but there is a lot of other logic in there. Again, this gets back to a good design (including any GUI framework you are using). You need to decouple your logic from the user-interface.

Many modern development frameworks promote good design by separating the user-interface from the rest of the code using a thin UI layer on top of everything else. For example, XAML allows you to create the GUI often without any programming at all. In fact, the user-interface can be (and often is) created by someone with no coding skills. Moreover, it can be modified at any time without code changes.

The MVVC is a design pattern which separates the View (user interface) from the View-Controller (code behind the user interface). Unit Tests can be inserted at this interface by talking directly to the View-Controller. (Though more commonly a user-interface scripting language is used which allows anyone to create tests without modifying the code - these are usually called acceptance tests not unit tests.)

We are working to a tight deadline. We don't have time to create Unit Tests.

This one I really hate! You need to find the time somehow to add the unit tests, or renegotiate the deadline. The "decision makers" should be made aware that trying to save a little time now will have much larger costs in time and money in the future. If they cannot be convinced then I recommend you look for a new job.

One of the reasons that unrealistic deadlines are forced on projects is that there is a perception (not entirely unwarranted) that people don't work at their full capacity until the deadline approaches. An alternative is to take the Agile approach and make several releases so that everyone can see that something is happening.

CASE STUDY
About a decade ago I was given the task of adding a GUI to an existing piece of software which was just being run via the command line (using dozens of confusing switches). The deadline for completion was two months. I thought it would take six months to do properly with full Unit Tests.

I tried to negotiate for more time but it was decided that if we spent any more than two months it would be impossible to make any money out of the project. It was to be a one-off "quick and dirty" project. We were told to cut corners to save time -- there was definitely no room for Unit Tests. The other limitation was that we could not drop any of the major features.

However, I felt the real reason for the short deadline was that, in the past, for any project longer than a couple of months, the developers had lost focus somewhat. I was just getting into XP and thought the Agile approach of multiple releases would assuage that concern. My alternative proposal (eventually accepted) was that a minimal working version be created, then have fortnightly releases to enhance it until the right trade-off between capability and profitability was attained.

We quickly created a basic initial running version without cutting corners and with full Unit Tests. In the next two months we added most of the desired features though some were changed and others were left out. Even though it went three weeks past the original deadline everybody was happy with the process and more than happy with the end result.

The software was more successful in the market than anyone imagined. There were several later enhancements which were easily implemented due to the fact we had worked "slow and clean" instead of "quick and dirty". Maintaining this code was when I really came to appreciate the advantages of having a comprehensive set of Unit Tests.

I have a gut feel for the amount of testing I need to do and have usually been proven correct. Unit Tests are often overkill.

Your "gut feel" may be misleading you. How can you be sure you've generally been correct in the past? Adding more Unit Tests may have produced even better results than the ones you got.

I also have a tendency to trust the feelings I get from decades of experience. However, my initial "gut feel" was way off when it came to Unit Tests.

The moral is trust your instincts, but be open to the possibility that they are fallible. It may surprise you how much time Unit Tests can save in the long term..

Bugs were slipping through the Unit Tests. We didn't have time to maintain something that was not working. Eventually we stopped using them.

OK, it sounds like your tests were not very good to start with. Then you stopped adding new tests. There was a snowball effect -- the tests had less and less value until they become worthless.

There was a alternative possibility, where the snowball was pushed to the top of the hill and ran away down the other side. Some people call this "reaching the critical mass". By pushing a little harder initially to get good tests, people would have eventually recognized the benefits. You might also look out for them yourself and point them out. Once the programmers are aware of the benefits they will be encouraged to add more and maintain them, which will add even more value.

In summary, you need to start off writing good tests. Good tests have 100% code coverage (or as close as is feasible). Also, don't forget that in code reviews, it is just as important to review the tests as the code (if not more so).

We tried Unit Tests but found no real benefits so we dropped them.

This is a common cry. It usually comes from someone who was forced to use Unit Tests, despite not being convinced. When you look closely, either the design/tests are not well done OR the many benefits were there but not noticed.

Doing things properly involves having a good (modular, maintainable, verifiable, etc) design and having good tests (see next item).

Sometimes the benefits are not obvious, but still significant. And sometimes you have to be a little patient before you start to see the advantages. One day you will realize how much easier and pleasant many aspects of your job have become compared to when you didn't use them. Here are some with/without examples:



1. Someone introduces a nasty errant pointer bug. There is no Unit Test for this specific problem but the overnight build/test causes all sorts of bells to ring because the bug has a follow-on effect that causes several Unit Tests to fail. Though the errors are far from the source of the problem we know that the previous days changes caused or triggered the problem. Hence it is relatively straightforward to track down and fix.

Without Unit Tests this might not have been noticed for some time, even slip into production. Some users may have experience odd behavior, and a very small number may have had their system crash. These sorts of bugs can be very hard to track down unless you know when they started happening.
2. An enhancement is proposed from way out in left field (ie, strange and completely unanticipated). Implementing the change with the current core engine would create a solution that was grossly inefficient. However, an efficient solution is found which requires completely rewriting the core engine. Unit Tests allow us to take this approach and be sure the system is has no new bugs and is backward compatible.

Without Unit Tests the only viable solution is to completely rewrite the system allowing for the new requirement, but this would have not been possible due to the cost. Moreover, such a rewrite would have other problems (like missed undocumented features that users have come to depend on).
3. We had developed a problematic library which performed complex financial calculations. The trouble was it had grown large and cumbersome over many years of modifications. Many parts were poorly done and almost impossible to understand and modify. Fortunately, the overall design was acceptable and additions and bug fixes had been accompanied by Unit Tests.

One of the programmers proposed a table-driven approach that grossly simplified the design. In fact changes that were difficult in the past could be made with simple configuration changes (no code changes at all :). It took some time but the library was completely rewritten. The first new version failed most of the Unit Tests but after a few days work we were confident that the replacement had the same behavior as the original (and incidentally was much faster).

Without Unit Tests we would have just had to limp on with the problematic library. Any rewrite would have introduced so many subtle differences that it would have taken years to find and fix all the bugs. Further, some bugs would have probably still slipped into production with the possibility of large financial losses to users.
4. A new programmer joins the team. After being pointed in the right direction she is immediately able to make a contribution. She found some tests related to the area of code she was enhancing, stepped through it in the debugger to check her understanding, then wrote a new function (and a new test) all in a few hours.

Without Unit Tests a new guy would take days or even weeks of reading documentation and experimenting before being able to accomplish anything useful.

We stopped using Unit Tests because we found it too hard to maintain them.

This happens due to poorly written tests. It's not easy to write good tests if you have not been taught. In general, you should not need to modify individual tests much - most of the time you just add new ones or delete ones that are no longer valid. More commonly you would need to modify support code, like set-up routines.

Many novices try to cram as much as possible into one giant test. With a large cumbersome test a simple change to the SUT (system under test) can invalidate the whole test requiring the test to be completely replaced. If instead, many small tests are used then a simple change may be all that's required, such as to a single test or a set-up function.

I already discussed this and many other bad test practices (see Best Practices). In brief here are some reasons tests may be hard to maintain:

  • trying to check too many things in the same test - one concept per test
  • lots of tests with similar or duplicate setup (and tear-down) code - DRY
  • tests that depend on other tests having already been run - make tests independent of each other
  • tests that depend on the environment being set up correctly - make tests independent of the environment
  • tests that depend on the SUT's implementation - only test through the public interface (see also White Box Testing)
  • poorly written code - generally tests should be of same high standard as normal code
  • use of external dependencies with complex behavior - use test doubles

I made a simple (legitimate) code change and I got 42 red lights. (The programmer who made this comment then stopped running the Unit Tests.)

In this situation it's either a problem with the recent code change or a problem with the tests (or a combination of both). Neither is a good reason to stop using Unit Tests.

In the former case, it usually happens that a code change has been made without fully understanding the consequences. This can be easy to do, and it is one of the advantages of Unit Tests that they often tell you when you may have done something wrong. However, Unit Tests are no substitute for understanding what you are doing as well as having someone review your code.

The other possibility is that the tests make an assumption about the implementation which has been invalidated by your change. It may be overwhelming to get a large number of red lights but the fix may be as simple as updating a setup function used by all the failing tests.

The point is you have to fully understand the code (and the tests). Have another programmer review you code (and tests) too.

It's too hard to write a test that checks what happens when the server is down. It's easier to just pull out the network cable.

It may not be fun, but writing a test double that simulates the server will save a lot of trouble in the long-term. A test double can be used to simulate any sort of error -- after all communications to the server can be lost for more reasons than an unplugged cable. Having a comprehensive suite of Unit Tests using this test double allows you to easily test many situations that are rarely (or never) seen using manual tests.

Without a Unit Test for the off-line problem(s) then some silly programmer will introduce a bug that causes the software to crash when the server is disconnected. One day, perhaps weeks or months later, somebody will find this problem (pray it is not a customer). By then it will be a complete mystery as to what is going wrong. It will take much longer to track down the cause of the problem than if you had a Unit Test that flagged it immediately.

We use several complicated hardware devices that are always being updated and reconfigured. Creating test doubles for all of them would be impossible.

I feel for you, mate :( I have worked in several environments like this in the past. First of all I will say that if you are working with a complex hardware device and you do not have a simulator for the device you are already behind the eight ball. Even with normal testing there are situations and error-conditions that you probably cannot generate using the real hardware. You need test doubles even if you don't use Unit Tests.

First and foremost, you need a simulator for every hardware device. If a device (and associated drivers) is provided by a third party then you need to ask them for a simulator or work with them to create one. Of course, this needs to be maintained to always be up to date. Many hardware suppliers will have simulators for their own use which they will provide on request.

If you are writing the hardware drivers/library for some hardware then you also need to create a simulator. With a bit of thought and inventiveness this can be easy to do. The main thing is to update the simulator at the same time as changes are made to the real hardware/drivers, or even before; often (remember DRY) they can share a lot of code. In fact it can be very useful to update the simulator to test new functionality even before the real hardware is available.
CASE STUDY
About 15 years ago I wrote a C-library that controlled high-capacity SCSI tape drives. At the same time I wrote a simulator that performed all the same operations (using a temporary disk file in order to store large amounts of data that was "written" to the tape.)

This made it very quick and simple to write tests for many scenarios that would have been tedious using real drives. For example, with a real tape drive it might take hours to fill up a tape in order to test what the software does when the drive returns an EOT (end of tape) early warning. Even rewinding a tape can take several minutes.

Creating a simulator for hardware in this way is essential for writing Unit Tests for software that interacts with the hardware. If a real tape drive was required then the tests would take too long and nobody would run them. Moreover, the simulator allows testing even without the presence of a physical tape drive.

Our tests are out of date. We don't have enough time to catch up, since we're flat out just doing the development.

You've dug yourself into a hole. To avoid this situation in future I recommend you use TDD, where you write the tests before you write the code. In fact there are many other advantages to TDD (see xxx).

We have the best developers. They don't make mistakes, so we don't need Unit Tests.

To me this is like saying "Nobody around here lights fires so we don't need fire extinguishers".

Everyone makes mistakes, especially when modifying code they haven't looked at in months. One day you will spend days or weeks tracking down a problem that would have been spotted months earlier (and fixed easily) if there had been Unit Tests. Disregarding this there are other advantages to Unit Tests...

Without Unit Tests, your brilliant developers will have to document the design of the system, and very few brilliant developers can write well. Those that can are often more interested in coding. And those that are capable and willing invariably can't explain things in a simple enough manner for us ordinary developers to understand. Finally, even if you have a good designer, who can write, wants to and can explain things simply, they will not have time to update it every time the software changes (particularly if they leave the organization).

Far better, to get them to do what they do best - write code (ie, Unit Tests). These will act as documentation and living proof of how the software currently works.

Also, what do you do when your best developers leave and all you are left with is mediocre ones? Do you get the new programmers to write Unit Tests for all the existing code, which they don't even understand (despite reading the doco 10 times). In fact there will be too much inertia to even start writing tests for new code. After all why should they have to write tests when the previous developers did not have to?

Quite simply, if you really have the best developers then they will be pushing to have Unit Tests for all their code. First, they will realize that they themselves are not infallible. They will also want to give their code its best chance of surviving and prospering once they are gone.

I can't test the real-time behavior of my code using Unit Tests.

You can probably use test doubles in your Unit Tests. Most likely you will need to simulate the system clock. You may also need test doubles for other hardware devices.

If there are aspects of the environment that are non-deterministic then you probably cannot check for correct behaviour using Unit Tests (see next argument).

Our code is multi-threaded so we can't use Unit Tests.

You can't use Unit Tests to test interactions between threads since that is non-deterministic. (You need another approach to avoid/detect race-conditions, deadlocks etc, which is outside the scope of this article.) However, these sorts of interactions should be minimized and isolated from the rest of the code.

You can still use Unit Tests to test everything else.

If you use formal proofs of correctness then you don't need Unit Tests.

Good luck with that.

Summary

In conclusion, if you are starting a new project there is absolutely no excuse for not using Unit Tests. Almost everybody has some sort of problem using Unit Tests initially, but with a sensible design and properly implemented tests (as described above and in previous articles) they will give enormous benefits particularly for software that needs to be modified frequently.


If you have heard any other arguments against Unit Tests please feel free to share them.