Scrum Problems

Having used Scrum and read a lot about it I have found it has some problems.  If you try to explain these problems to an proponent of Scrum then the reply is always "you don't understand it" or  "you aren't doing it properly".  To an extent this is true but some teams try and try and eventually give up on Scrum.  There must be problems inherent in Scrum -- even if just the problem that says some people just can't learn it or can't do it properly.

Here I give an account of some problems I have experienced with Scrum or similar agile methodologies.  For some I give methods to fix or avoid the problem.  For the others I am open to suggestions.

Management

The common problem that Scrum (or any improvement requiring cultural change in an organization) encounters is lack of management commitment.  Often management think all they have to do is spend some money on Scrum training and they will have an immediate boost in productivity and responsiveness.

Many companies spend a small fortune on Scrum training for the team (the "pigs") but give managers and other "chickens" no guidance.  It is most important that management understand how Scrum works.  In particular, managers (and even the scrum master) need to refrain from interfering with the process and letting the team "self-organise".

Management also needs to be responsive to requests from the scrum master.  If the scrum team perceive that management are not committed to the process then Scrum will be doomed to failure.

Product Owner

Most product owners do a good job but they can be the weak link.  There can be major problems if the product owner cannot do their job properly.

One problem is that the client may not be able to or willing to provide a product owner.  In this case it is up to the team to provide their own proxy product owner - typically someone with a title like "business analyst".  Even when the product owner is provided by the client they can be so far removed from the actual users of the software that they are not in touch with what is really required.

Of course, the whole point of Scrum is to make things visible.  If the product owner is not doing a good job then, even as early as the first sprint, problems should be obvious.  In practice, however, everybody tends to trust the product owner and nobody else (ie, chickens) checks how the team is progressing.  The project may proceed for many sprints with the product owner perfectly happy, but it's not until a version is released to real users that anybody realises that the product owner's understanding of the project is very different to the users or other stakeholders.  This problem may have been visible but there was nobody there to see it.  Again, if managers (see above) take an interest (eg, attending sprint reviews) then this situation can be avoided.

A related problem is when the product owner cannot make proper decisions due to conflicting priorities from stakeholders.  It is common for different employees of a customer's organization to have different agendas.  (Of course, this is not a problem of Scrum but may occur with any type of project.)  It is the responsibility of the product owner to communicate with all stakeholders and resolve any differences.  If necessary, the customer's senior management may need to be called in to resolve any conflicts.

Finally, the most common problem I have encountered with product owners, is their tendency to vacillate.  I am not sure why this happens but I suspect that there is an element of megalomania having the team bend to their every whim.  Some product owners are oblivious of how their decisions can result in major headaches for the team.

The main trouble here is that *when priorities keep changing the team has no clear focus*.

As an example of how a bad product owner can cause chaos I will relate a story that happened to me.  The product owner had created a product backlog which was topped by some simple but crucial tasks, followed by a large (but less crucial) enhancement.  Since the crucial tasks and the large task would have taken too long to complete in one sprint we resolved to finish the crucial tasks and a few other lower-priority tasks and to do the large task in the next sprint.

When it came time for the next sprint the product owner had reorganized the backlog and the task we had all been expecting had been pushed way down the list.  The first problem was that we had been mentally preparing ourselves for this large task by thinking about it and even doing a bit of research (grooming).  Worse was the demotivating effect of the product owner making seemingly arbitrary changes.

It's important to make sure the product owner understands Scrum and the development process.  The scrum master does have the authority to "fire" the product owner but this would typically be due to their interactions with the team.  The scrum master could be largely oblivious to the above-mentioned problems.

Backlog

Another problem related to the  product owner  is the prioritization of the product backlog.  A major rule of Scrum is that the  product owner orders the product backlog according to value to the business or business functionality.  I think this is a bad approach.

What I really hate about this approach is that it tends to ignore any user interface niceties that make the program enjoyable to use.  Many Scrum teams produce software that could be at best described as Spartan and I would just call plain ugly.

However, a worse problem is that most product owners have no appreciation of the development process.  Because the backlog is ordered on business functionality there is no room in the process for the most important work such as re-factoring to maintain the integrity of the design and prevent it becoming an unmaintainable "ball of mud".

The whole emphasis in Scrum is on "user quality attributes" not the more important "developer quality attributes".  This is covered in more detail in my previous blog post "The Importance of Developer Quality Attributes".

Team

As I have learnt more about the "proper" way to do Scrum I have discovered many things that surprised me ...

The first time I used Scrum all the team members were basically programmers.  We did not have separate analysts, designers, testers, etc.  Everyone contributed to the design to some extent, though of course, the more experienced team members did most of the architecture and high-level design.  Analysis was done by the team together with the  product owner .  Everyone tested their own code (though the customer had acceptance tests and other tests).

Everyone in the team was involved in all the steps towards delivering a product to the customer ready for their rubber stamp (acceptance tests) and subsequent release.  This was very empowering and motivating.

I have since learnt that the team in Scrum is supposed to be *multi-disciplinary*, which most people take to mean that the traditional roles of analysts, designers, architects, programmers, testers, etc are preserved.  This to me goes against the principles of empowerment and reverts to the "production line" mentality of Waterfall models.

It also means that there is far less flexiblity for teams members to self-organise.  If all team members have assigned roles then the work is effectively assigned to each member simply by the nature of the task - the programmer(s) write the code, the tester(s) test it, etc.  And what happens if your tester(s) are away - who does their work?  It also makes sprints like "mini-waterfalls" where at the start of the sprint there is nothing for the testers to do, and by the end the analysts and designers are twiddling their thumbs.

Another problem I have seen is that Scrum teams can become insular.  Although communication within the group can be greatly improved, communication with people outside the group can decline.  This is a common misunderstanding of how Scrum works.  Though Scrum restricts how outside influences affect the team this does not mean that the team should not talk to people outside the group.  In fact, for the sake of visiblity, a fundamental tenet of Scrum, they should do so.

Other

I think those are the major problems I have encountered.  Please feel free to comment on why these are not really problems or how they could be avoided.

There are other problems which I could mention, but they will have to wait for a later time, but I will briefly mention them.  One is simply that some types of projects are not suitable for an agile approach.

Another difficulty is deciding on the amount and type of documentation that needs to be produced in Scrum (or any agile methodology).  Generally, I favour code over technical documentation for many reasons, but mainly because code is much more likely to be correct and complete.  I will explain all of this in an upcoming article on documentation.

On thing I just rememebered is what I have always felt is a major failing of Scrum, which is that it fails to discuss any sort of technical techniques and processes - the sort of things that XP goes into in depth.  The most important technique (and one I have been promoting since 1996), is the idea of unit tests.  Now, with agile methodologies the use of unit tests is crucial, but, amazingly, none of the books I have read on Scrum even mention unit testing.