Customer Management

There is often a disparity between what people want and what they need. (One example is the propensity to eat unhealthy food.) Fortunately, people are generally rational and well-educated in avoiding things that work to their detriment. (Most people have a reasonably healthy diet.)

However, there is still one industry where customers seem to be unaware of the dangers of impulse buying. That industry is the software industry. Admittedly, customers (the people who pay for the software) have a much better understanding of what software can and can't do than they did a few decades ago. However, they are generally oblivious to what goes on under the hood which is a great shame since with greater understanding they could make better decisions.

The Challenge

It is in everyone's best interests to educate and involve the customer in the software development process. (Conversely, developers generally need to understand more about the business side of things, but I will leave that for another blog.) The customer needs to:

• understand how software is developed and the importance of maintainability of the code
• concentrate on why features are needed and not insist on a specific implementation
• realize that short term thinking can incur a long term technical debt
• become involved in the day-to-day development process
• provide continuous feedback to ensure the project is moving in the right direction

The challenge is to achieve the above ideal. If it is done at all, it is usually left to developer(s), such as analyst, system architect or technical manager, to educate the customer, but this can be a slow process. Getting the customer involved in, or even aware of, what the developers are doing can also be difficult. I make some suggestions later.

YAA (Yet Another Analogy)

In this sort of discussion, inevitably, (as in the under the hood reference above) I come back to the analogy with buying a car. I hope to show that when people buy a car they make reasonably informed decisions; when people (or companies) buy software, or software enhancements, they often make poor decisions.

Almost everyone has a basic understanding of how a car works internally. When purchasing a car people will do some research or ask a knowledgeable friend for help. When I talk to my friends and colleagues, most of whom claim to have little knowledge of cars, they still have a very good understanding of the strengths of different brands, models and even the year of make. In fact everyone I have asked agrees on the most reliable brand.

Despite the common joke, nobody really buys a car for the cup-holders.

When buying software on the other hand there is far too much emphasis on the cup-holders, like GUI bells and whistles. Even most developers are oblivious to many things, such as trade-offs between different software quality attributes (see below). So it is no surprise that the "professionals" that customers consult (managers, analysts, salespeople, etc) have no idea about what is really in the customer's best interest.

The Homer   

This also reminds me of an episode of the Simpsons. The episode had Homer as the designer of a new car model. (His long-lost half-brother owned a car company and wanted to give the average person, like Homer, the car that they really wanted.)

I can't remember much about the actual car except that it had two separate "bubbles" so that you could not hear the kids in the back making a racket.

As you can imagine, "The Homer" was a disaster, as giving a person what they want (or what they think they want) is entirely impractical.

This reminds of another old joke:

Q. What's the difference between a car salesman and a computer (software) salesman.
A. A car saleman knows when he is lying.

Quality Attributes

Way back last February I discussed the two different types of software quality attributes. (See The Importance of Developer Software Quality Attributes.) The main point of the post was that focusing on the customer quality attributes can be detrimental to the customer's interests, especially maintainability and especially in the long term.

Customers, and non-technical stakeholders, need to understand the importance of developer quality attributes. Sometimes the developers have more important activities than adding more cup-holders!

Feedback

In the software industry the customer is not always right. In fact they are almost never 100% right and most of the time way off track.

Of course, I am not saying that developers should arrogantly assume they know better than their customers. I am simply saying that most customers often only have a vague idea of what they want. And what they say they want is not what they (or their organization) actually needs. Further, what they say they want often evolves or even changes drastically. Different representatives of the customer may have very different views also.

This is where regular feedback is so useful. It helps the customers and developers quickly hone in on exactly what is required. Unfortunately, generally developers get little and irregular feedback.

Avoiding Disasters

Just after I wrote the above I was watching one of my favorite TV shows - Air Crash Investigation. (This is the name in Australia. In other countries it may be called May Day or something else.) It is basically a documentary series about how planes crash. I like this show as it has a lot of information about how things go wrong which I find can often be applied to the design of software. Some of the accidents (actually surprisingly few) even have software problems as a contributing factor.

In the episode I just saw, the aircraft was fine (ie all flight controls were working perfectly) except that the instruments were feeding them incorrect altitude information. Since it was at night and they were flying over the ocean they also had no visual feedback on their vertical position. To cut a long story short the plane flew into the sea and everyone died. There have also been other crashes and incidents where the aircraft flight controls were functioning perfectly but a problem was caused by instrument(s) feeding incorrect information to the pilots or the computers.

This really drove home to me how important feedback is. The moral is if you don't want your software project to crash and burn you had better obtain accurate, frequent feedback from your customer.

Customer Focus

One of the principal tenets of software quality assurance (and QA generally) is a focus on the customer. It is correctly observed that many organizations become focused on what they are doing, not why (or for whom) they are doing it. Customer focus is a good thing!

For a developer, obtaining customer focus is a problem, as most developers are many layers removed from the actual users of their software. Moreover, many developers take customer focus to mean intently responding to every request of the organization paying for the software (or usually one or two individuals of that organization), no matter how impractical or pointless they are.

Instead you need to work with the customer to first identify what they need, and find the best way to give it to them. In the process you may need to persuade them to alter their perspective on what is important and consider the long term effects of what they are asking.

developers do their customer a disservice by pandering to every whim of its representative

Many developers do their customer (ie, the actual organization) a complete disservice by pandering to every whim of its representative (ie, the employee of the organization). This is not what is meant by customer focus!

Employee Focus

There is an old saying (from the quality assurance world) designed to emphasize the relative importance of the customer (the client(s) who pay the bills), the employees (ie staff) and the owners (ie shareholders). It is that CEO stands for:

    Customer
    Employee
    Owner

My own rule of thumb is to use the 80:16:4 ratio. That is an organization should spend 80% of its effort in giving the customer what they want, 16% in employee assistance and development such as training, and 4% on investor relations.

However, an idea that has recently gained some favor is the idea of employee focus rather than customer focus. This relies on the assumption that if you treat your employees well then they will in turn treat the customers well. In my experience, this does not always hold, but I guess that depends on the employees.

A Sad Tale

Speaking of keeping employees happy reminds me of an experience that I had in my 3rd year of work as a programmer. Unfortunately, I have oft seen similar things many times since.

I was working for a big company (at the time the largest company in Australia). Soon after I started, our section began a new project that were we told was of strategic importance to the company and that was critical to be delivered on time to coincide with a new marketing campaign. The project was driven by one person from the "business" side of the company upon whom we depended for our direction and to whom we reported progress. However, we also interacted with his "big boss" and other representatives from the "business" side of the company from time to time.

Although the project was fairly well specified compared to earlier projects there were inevitable problems and delays. There were also extra undocumented requirements added that were deemed critical to the success of the project by the "big boss". Our own project manager also added his own ideas on how and what should be done.

Of course, a month before the deadline we were behind schedule but due to long hours and dedication we felt that there was a good chance of finishing by the immovable deadline. Then the project was simply canceled by the "bigger boss". No proper explanation was given. One concern was that perhaps that one of the "bosses" did not believe we would deliver on time. Eventually the explanation given was that there had been a "strategic change in direction".

Obviously, this upset a lot of people who had worked very hard on the project. The "business analyst", who had put her heart and soul into the project, quit on the spot and several team members left soon afterward. Most of these people actually worked for a software contracting firm hired by the big company, so it was this company (rather than the "big company") that lost out due to loss of experienced staff and the effect on morale. (Though to be fair the "big company" was a major shareholder in the contracting company.)

Customer Management

There are three important aspects to what I call customer management.

First, you need to educate the customer so they can make informed decisions. This can be a slow and painful process but the benefits are enormous. I provide a list of some of the things the customer needs to learn in the What Every Customer Should Know section below.

Also, sometimes you need to say "no" to a customer request if it is a bad idea. (I talk about how to spot these bad ideas below in the section on The Customer's Customer.) Finding out the reason for the request (or the reason for the reason for the reason ...) can lead to a deeper understanding of what the customer needs rather than what they think they want. If the customer understands something of the development process (see above) they will be more likely to understand your point of view.

Finally, you need to involve the customer in the development process for several reasons:

1. so they learn more about how the software is developed (which assists their education);
2. so they appreciate the effort involved; 
3. (possibly most importantly) so they can provide feedback that drives the project in the right direction;
4. last (but probably not least importantly) for developer motivation.

I also discuss some ideas for getting the customer involved below.

What Every Customer Should Know

• between 50% and 90% of software work is maintenance = to save time and money there needs to be effort spent on making software maintainable
• unit tests can add to the initial cost of developing software but the long term benefit is enormous
• bugs can be a good sign! - they may indicate that the developers are refactoring the code to improve maintainability
• but having good unit tests mean that the code can be refactored with almost no risk of bugs
• sometimes things just take time
• immovable deadlines can result in bad (quick and dirty) changes being made to the code
• continual bad code changes (without a chance to refactor) can destroy the software
• having good unit tests means that changes can usually be made more quickly with minimal chance of bugs
• if a developer is reluctant to change what is working then it indicates that the code is in a poor state (or the developer does not know what they are doing)
• a team without customer feedback is like a ship without a rudder
• provide feedback about your goals, not preconceived ideas about how they should be achieved

The Customer's Customer

Sometimes a customer will insist on something that you are not sure about. To get a different perspective it is often useful to consider the customer's customer.

In the excellent book 97 Things Every Software Architect Should Know, tip number 82 is called Your Customer Is Not Your Customer. The example given is developing software for a web site and the client says not to worry about the security (such as not using SSL). In this case the correct thing is to say no to the customer.

When your customer asks for something that does not seem right, you need to consider the customer's customer(s). This allows you to clearly explain what you are doing or why you must say no to a request. Again this goes back to giving the customer what they need, not what they ask for.

Getting The Customer Involved

An important part of agile methodologies is to have the customer regularly use the software. For example, in Scrum there is a Sprint Review at the end of every Sprint. This is the best way to get the customer involved, by having working software which shows what has been recently accomplished.

However, it can be difficult to get a disinterested client to even come to sprint reviews. Sometimes an incentive, like free chocolates, or just making it a bit more fun can help. Of course, a few reminder emails will not go astray.

It is also worthwhile keeping all stakeholders in the loop, with regular emails or even a newsletter.

An important thing is not to isolate developers from the client and users of the software. Encouraging this interaction not only gets the customer more involved in the process, but they also learn to appreciate how the software is developed and the difficulties that the developers have to deal with. Moreover, most developers are highly motivated by interacting with the actual users of their work!

developers are
highly motivated
by interacting
with the users
of their work!

Note that one potential problem with a lot of interaction between the client/users and the developers is the possibility of unauthorized changes being requested. It should be made clear to all, that all change requests are to go through the proper channels (eg, the Product Owner in Scrum).

Scenarios

Just to emphasize the points I invite you to consider these scenarios that are all based on real-life experiences.

1. Your company is considering buying a smaller software company for a particular product that it has developed. Your boss insists that you make sure it is a good buy so their are extensive discussions with the two developers. You also have several people "road test" it to expose bugs and problems of usability, reliability, efficiency, etc.

At no time does anyone run metrics on the source code or inspect it with regard to maintainability, etc. This is like buying a car without opening the bonnet and checking the log books. Actually I consider it to be even worse than that since maintainability of software is usually its most crucial attribute.

2. The customer has an urgent requirement. There is not time to understand how to implement the change correctly. A quick and dirty fix is implemented. No time is allocated for repairing the band-aid solution later.

The result may be a happy customer in the short term. However, there will be bad to horrendous consequences in the long term. If software is continually modified like this, and not refactored, it eventually becomes prohibitively expensive to maintain. I have seen code that has degenerated to such an extent that seemingly simple changes can require ten (or more) times the effort than they should.

The customer needs to be made aware that this sort of development style is not in their own long term best interests. This is what is meant by technical debt - saving time and money in the short term means you will be paying interest which has far greater costs in the long term in the form of much higher maintenance costs.

3. The customer asks for a significant change and is very pleased when the changes are delivered ahead of schedule. They next ask for a more important but apparently much easier change and are very disappointed when it takes many times longer than expected or can not even be delivered.

The customer has no idea how the software works internally. They need to realize that sometimes apparently simple things need a major re-factorization to accomplish.

4. Your software stores a large database of customer details in a web site. You know there are potential security holes which could allow someone to access customer information or even modify it.

It is in the best interests of the user of the web site to have good security and to encrypt customer details. Of course, you would never even consider storing credit card details unencrypted, would you?

Conclusion

The idea of customer management sounds a little condescending, but the point of this post is about finding how to give the best value possible to the customer, in the long term, despite what the customer does, or says they want.

Customers need to be educated to understand that there is more to software than the GUI. Insisting on strict deadlines and intolerance of bugs can have dire consequences for the maintainability of the software.

Customers also need to realize that what they initially believe they want may be nothing like what they need. Often the only way to obtain a good result is to use a process of iterative refinement based on continuous feedback as described in various agile methodologies.

Continuous feedback means that the customer needs to become more involved in the development process. It is essential they know what is happening under the hood in order to make the best decisions.

Iterators Through the Looking Glass

I have been talking a lot about STL (the brilliant C++ template library) lately because it is brilliant, but also because I have been teaching a class on it at my son's local high school - NBHS (Normanhurst Boys High School).

Several boys at NHBS have been winning programming competitions recently - for example, three went to the Informatics Olympiad training camp which takes the top 15 high school students throughout Australia to be trained and selected for the International Informatics Olympiad. My son Daniel's team also recently won the open round of the UNSW Prog Comp (see https://cgi.cse.unsw.edu.au/~progcomp/open/prov.php).

However, they do all their stuff in Python. I like Python (a lot) but I am trying to convince them that C++ and STL is more flexible (and efficient) for a lot of problems. Most of these competitions are won using C++ or Python.

One of the questions that recently came up in the class was to do with reverse iterators. There is some confusion about reverse iterators so I thought I should explain... (I promise that this is the last post on STL for a while.)

Missing STL Algorithms?

Recently in the class we looked at some STL algorithms. I mentioned that one way to find a character in a string is to use the find() algorithm, which is analogous to the C standard library strchr() function (except that it can be used to search a range of any type not just a string of chars).

// C

const char * s = "Hello world.";

const char * p = strchr(s, 'e');      // returns pointer to 'e'

// C++ using a string as an STL container

std::string s("Hello world.");

auto p = std::find(s.begin(), s.end(), 'e');

The obvious question is then: how do you search for a character from the end of a string instead of the start? That is, what is the analogous function in STL to the standard C library function strrchr()? (Note the extra 'r' denoting a reverse search.)

If you are looking for an STL algorithm that searches for a value from the end you will probably immediately think of find_end(). As it turns out there is a function called find_end, but it searches for the last occurrence of a sub-range within a range. You could use it to search for the last occurrence of a sub-string within a string. Ie, it is like search() [which is analogous to the standard C function strstr()] but searches from the end. Obviously it should have been called search_end not find_end.

It turns out there is no STL algorithm to do this. Is this an oversight? Hardly, since it is trivial to do a reverse find using a reverse iterator. However, it is not so simple to use the returned iterator unless you understand how reverse iterators work.

Reverse Iterators

Reverse iterators are exactly the same as normal iterators except in reverse. When you increment a reverse iterator it moves to the previous element in a range. When you dereference a reverse iterator you access the element before the address.

My first use of reverse iterators was very pleasant. I was working on software that manipulated 3-D geophysical data and I had a huge array (actually a vector of vectors of vectors) of magnetic data that was indexed by Cartesian coordinates (x, y, z). To process this data I had a 3 nested loops (for each of the 3 dimensions). At some point I realized that I was processing one of the dimensions in the wrong direction. It was trivial to simply change the iterators used for that loop to reverse iterators. Doing the same thing with array indices would have taken time (maybe 10 minutes) and would have been a little more error-prone too.

Rather Backward

Reverse iterators are perfect if you just want to use or modify the value they "point" to. The problem occurs when you want to do something like erase the value from its container. It is easy enough to erase from a container given a forward iterator. Unfortunately, you can't erase using a reverse_iterator - container erase() methods only take a normal iterator.

The other problem is that you need to think carefully about whether you want the "forward" value or the "reverse" value given an address.

Let's look at an example. The following code searches for the first occurrence of a left bracket and adds a space before it.

std::string pal("A(good) test.");

auto first = std::find(pal.begin(), pal.end(), '(');

pal.insert(first, ' ');   // "A (good) test."

Note that I use the new C++11 keyword "auto" to save typing "std::string::iterator". Also note that I am using a std:string as an STL container but I could just as easily use a std::vector, std::list, etc.

That's OK. Now how do we insert a space before the last bracket? The obvious thing to try is:

std::string pal("A(good) test. It(really)is.");

auto last = std::find(pal.rbegin(), pal.rend(), '(');

pal.insert(last, ' ');   // compile error

But std::string::erase() takes an iterator parameter not a reverse_iterator, so this does not even compile. Luckily, you can convert a reverse_iterator to the corresponding iterator by using reverse_iterator::base().

std::string pal("A(good) test. It(really)is.");

auto last = std::find(pal.rbegin(), pal.rend(), '(');

pal.insert(last.base(), ' '); // "A(good) test. It( really)is."

But this is also wrong as it inserts the space after the bracket. The problem is that an iterator "points" to the next element, whereas a reverse_iterator points to the previous one. See the diagram.

In this case we want to insert before the address of the element pointed to by the iterator, so we need to decrement the reverse iterator before we convert it to a forward iterator.

std::string pal("A(good) test. It(really)is.");

auto last = std::find(pal.rbegin(), pal.rend(), '(');

pal.insert((++last).base(), ' '); // "A(good) test. It (really)is."

In the Looking Glass

So to use a reverse iterator you may need to increment it before converting it to its corresponding forward iterator. However, this is not the case if the natural order of the data is reversed, in which case a reverse iterator reflects the correct processing direction. An example, would be a text editor for a language with right to left reading order - if you insert a character in such a text editor then the existing character at the cursor (and all those to the left of it) gets pushed left. This is analogous to "inserting" a space at the right bracket in the above example string like this:

last = std::find(pal.rbegin(), pal.rend(), ')');

pal.insert(last.base(), ' '); // "A(good) test. It (really) is."

However, even in data with a natural reversed order deleting an element (with the container's erase() method) requires adjusting the reverse iterator first as erase() only takes a forward iterator and consequently deletes the next element, not the previous one as we would like.

Conclusion

Reverse iterators are useful and essential to be used with some algorithms when you want to work from the end instead of the start.  (An alternative is completely reversing the order of the range first using reverse() but that may be inefficient.)

The problem with reverse iterators is that many functions only use normal iterators. You can use base() to convert a reverse iterator but you need to be careful you use the right slot. Most of the time you want to use the slot before in which case you should increment the reverse iterator before calling base() on it.