Saturday, 18 April 2015

Team Collaboration

Collaboration is at the core of problems experienced with implementing Scrum (or any team work). As I mentioned last week, consideration of personalities is important when composing a team. This is a big/important enough subject that I decided to devote a whole post to it.

Developers are simply not good at working in a team. In the past, good programmers could get away with working fairly independently in their own specialized area. Nowadays, larger projects, more varied project types and different development methodologies means that good team-work is expected of everyone. Moreover, highly collaborative (ie, Agile) teams are more likely to produce a better outcome, more efficiently and with less risks.

You can try to hire people with the right personality, but it's not necessary (and sometimes you simply don't have that choice). Luckily, using Scrum and some tips that I mention below, most developers can learn to work as part of a team, and even enjoy working with others for a worthwhile goal.


Why Teams Don't Collaborate

Poor collaboration is most obviously seen in conflict over technical issues. In three decades in the industry I have seen this many times. When you get to the heart of the matter the problem is rarely purely a technical issue but also due to personality conflict, poor communication or interpersonal skills or even emotional disturbance.

Differences of opinion on technical topics such as software design, coding standards, etc are inevitable. Often this is simply a sign that different people have different priorities - eg, different team members may focus on different aspects of the quality of the code such as efficiency vs maintainability. (See Importance of Developer Quality Attributes for more on software quality attributes and their relative importance.) When the team is collaborating, and pulling in the same direction these issues can disappear, or become much easier to resolve, since team members are more amenable to compromise.

Managers often realize there is more behind the disagreement, such as a clash of egos. But they fail to dig deeper to find that some of the team may feel disempowered, threatened or generally dissatisfied. Again, a Scrum team overcomes these sorts of problems by empowering and encouraging communication and collaboration.


Personality

Typical Developer DISC Profile
To get an idea of why collaboration does not happen let's look at the personality of a typical developer. Last month I mentioned the DISC system of personality assessment (see Team Composition). Developers tend to fall in the S-C-D range and are predominantly of the C personality type (see diagram at right). This means they are conscientious and competent, striving for accuracy and quality. However, they prefer to work independently with little social interaction.

It seems that the software industry tends to attract people who are averse to collaboration. (In fact DISC assessors recommend software development as an appropriate career for those with a C personality type!) This is the major problem when attempting to get a Scrum team to collaborate and self-organize.

So do we give all new recruits a personality test and only hire developers with the right personality type? No -- that's not a good idea. It would greatly reduce the pool of potential candidates, some of whom may be otherwise ideally suited to the role. The fact is almost all developers (even the extreme case of someone with Aspergers) can learn to collaborate, and even enjoy working in a small self-organizing team.
My Internal DISC Profile

I believe this non-collaborative behavior is primarily a conditioned or learnt response, as I explain later. This is clearly shown with DISC personality assessments of a typical developer. DISC assessments give two personality profiles - called internal and external. An internal profile (see diagram at right) reveals a person's natural tendencies, whereas the external profile (see diagram below) shows actual behavior in the work environment. In disempowering environments developers typically suppress their collaborative tendencies.


My External DISC Profile
For example, last year I had a DISC assessment which showed that for my Natural/Internal personality my S and C styles were about average. However, for my Adapted/External personality my S style had disappeared. (The S personality type is supportive and likes working closely with other people.)

What was it that suppressed my natural collaborative tendencies? I think there are a number of factors which I cover in the next few sections. Further, I also believe that proclivity to working in a small team is a part of human nature as I explain later in the section below entitled Homo Habilis.

Training

Training in the industry tends to reinforce the notion that software development (at least design and coding) is a solitary activity. For example, in all the courses I did at uni. just one (of the dozens) of modules I took involved working as part of a team - my 3rd year course on "Software Engineering". Things have undoubtedly improved since 1983 but I still believe there is a problem.

Conditioning

Even more significant is non-collaborative behavior that is reinforced by many organization's culture or managerial practices. Many managers, by their attitude and behavior, promote rivalry and even conflict between team members. This is symptomatic of a command-and-control management style as I discussed previously (see Production Line Mentality).

Managers often realize that what they do and say has detrimental effects and try to change their behavior. However, this can be difficult when  motives are questioned. For example, common advice is "Praise publicly, criticize privately" - but this can backfire. Of course, you should never criticize publicly - it is demoralizing and humiliating - but (too much) public praise may not be seen as genuine and may trigger envy in other team members. A sudden change in behavior will also be viewed with suspicion unless there is evidence of a change of attitude. Above all never say one thing and do another.

Praise privately   

It's commonly said to "criticize privately but praise publicly". My own policy is to give all feedback to employees privately - both criticism and praise.

Of course (unless you are very good a deception :) it should also be honest, otherwise it is simply seen as manipulative.

Admittedly, it is nice to ensure it is known publicly when someone has done a good job.
In addition, many companies have policies that work against team work. For example, such things as Annual Performance Reviews and Employee of the Month awards usually have little effect apart from causing resentment. But when they do have an effect it is one that promotes competition and rivalry which actually undermines collaboration.

Why Teams Should Collaborate

We have seen how the personality, training and years of conditioning make it difficult for developers to work together - so why don't we give up on the idea of an highly-collaborative, self-organizing team?

Don't give up! If we can overcome the above problems then there are big advantages. I have been expounding these advantages (and the disadvantages of a command-and-control no-collaborative approach) in all my posts since last December (except for Lean is Not Agile).

In summary they are:
  • more creative ideas are generated
  • less coordination overhead
  • avoids many pointless activities
  • greater focus on giving the customer what they need
  • less specialization leading to greater productivity
  • specialists cannot hold the organization to ransom
  • greater adaptability to change
  • greater job satisfaction
  • greater productivity
In essence a good Scrum team will produce better results. Further, the results are often produced more quickly - or there is, at least, the perception of increased productivity because the team is focused on what is important to the customer - in the short term. In the long term there are definitely large productivity benefits.

Disadvantages

I researching this post I found quite a few web pages arguing against small self-managing teams (eg: Does XP/Scrum Violate the “Agile Manifesto”?), Some of them have a point, but some are simply incorrect. I refined it to six basic points which I will now address.

1. Having many small tasks done by different people results in inconsistencies.

This will happen in any team, but in Scrum team members are actually more likely to work together to avoid these sorts of problems. For example, in almost all jobs I have had there have been coding standards "in use", but I have only ever seen them followed with any rigor in a Scrum team.

2. Group decisions are often not good decisions.

The team may be prone to "group think", but this is not due to Scrum. In fact, Scrum empowers individuals to make a contribution. The focus should be on finding the best solution, not going along with everyone else.

There will be occasions where there are differences of opinion on technical issues, but in my experience, when all team members are aligned, these are easy to resolve. Moreover, decisions are often not group decisions since generally Scrum teams have experts in different areas to whom they defer.

3. Decisions are slow because they are made by committee.

First, it is important to remember that a Scrum team is not a committee. Most committees are composed of members with very different agendas; whereas in a Scrum team all members should have the same focus.

Also, it may appear that a team is indecisive because they are using an important Agile practice of deferring decisions until necessary (see JIT). When the current backlog task dictates that a decision be made then a good Scrum team will not hesitate to decide (or give it to the expert in that area to decide).

4. There is no single person with the authority to enforce design consistency.

This is related to the previous points. The argument is that there is nobody in control who sees the big picture, which results in haphazard decisions and inconsistencies in the design. Common opinion is that it's better to have a single person with the vision to direct the project.

The truth is that this approach does not work well with a team of more than a few people. The people actually creating the software are continually twisting and changing the vision for many reasons. The visionary is flat out ensuring that her vision is implemented.

Scrum encourages a better approach where everyone sees, or at least has a grasp of, the big picture. It does not always work perfectly, but it works better than the single visionary approach.

5. The more you divide a task between people the greater the communication burden.

Again, this is simply a consequence of working in a team of more than a few people. Using Scrum and feature teams (an example of DIRE) actually reduces the communication burden.

6. There is no individual ownership and pride in work.

I talked about this last week (see the section on Code Ownership). You may believe, as I do, that you write perfect code, but with input from other team members it can be even better. Moreover, there are other disadvantages of ownership such as specialization, increased risk, etc.

Of course, taking pride in your work is a good thing. Knowing you have done a good job is immensely satisfying. What I didn't expect with Scrum is that I can be even more proud when my team does a good job.

Homo Habilis

One fundamental advantage that humans have is the ability to work in small teams. Humans are inherently social - far more than any other species. Much of this behavior (or tendency towards the behavior) is built into our DNA, due to our recent evolution over the last few hundred thousand or more years.

The formation of small teams was an important early step in human evolution probably beginning more than two million years ago when Homo Habilis began hunting in small self-organizing teams.

Many developers work for years, even decades, almost in isolation, which can be alienating. Many report having an immense feeling of satisfaction when they first work on a collaborative, high-performance Scrum Team. The problem is that not all teams are like that - in the next section we will try to fix that.


How to Encourage Collaboration

Collaboration is fundamental to Agile and Scrum in particular. Lack of a collaborative spirit is usually the reason that Scrum fails. So how do we fix it?

Scrum Practices

Scrum practices encourage collaboration. The visibility provided by the daily standup provides a feedback mechanism that keeps everyone on the same track (as well as providing motivation and peer pressure - see the discussion of the Ringelmann Effect in Scrum == Communism). The product owner, and particularly their product backlog, keeps everyone focused on the one thing - what the customer needs. Empowering the team gets them working together to come up with creative solutions.

XP Practices

Other Agile (not specifically Scrum) practices also help. Here are some examples from XP.

A common open work area for the team encourages communication. (See XP:Space)

A customer representative (eg, the PO in Scrum) works closely with the team. This not only allows the developers to obtain timely help, it also gives the customer invaluable insight into the development process as I explained in Customer Management(See XP:Customer)

Continuous integration means the developers are always working together to ensure that the system works and that code modifications are compatible. (See XP:CI)

One of the best signs of a good team is when programmers are happy to work on other's code and have other's modify their code. At the very least all code should be reviewed by at least one other person. If people get defensive about "their" code then that is a problem. Pair programming is the ultimate aim in this regard, and has the added benefit of producing better code. (see XP:Pair)

Related to pair programming is the idea of "collective code ownership". Nobody owns any piece of the code (or anything at all, actually), so anyone is free to modify any part of the code. Note that I agree that often only the original creator(s) of a piece of code may understand all nuances of the code, which is why Unit Tests are essential to ensure that changes do not introduce bugs. (See XP:Ownership)

Enhancing Team Spirit

Unfortunately, some teams just never "click". Usually the problem is the "mind-set" of one or more team members, as I discuss below. First we look at building team spirit which can be the first step towards changing a mind-set.

Here are 12 suggestions:

  1. encourage open and honest communication
  2. empower the team to make decisions
  3. communicate about strategic planning/goals
  4. encourage team members to trust each other
  5. make the most of the talents of the team
  6. never punish or humiliate team members for mistakes
  7. avoid behavior which promotes rivalry
  8. never compare team members against each other
  9. reward the team not individual team members
  10. encourage the team to be polite to each other
  11. encourage the team to eat together
  12. encourage the team to see the customer's point of view
Changing Mind Set

I guess the whole point of this post is that many developers have the wrong mind-set. They become defensive when their mistakes are pointed out. Alternatively they are reluctant to give feedback when someone does a bad job for fear of offending them or getting a bad reaction. People *forget* to give positive feedback when someone does a good job.

Somehow you need to encourage the team to be open and honest with each other. A collaborative environment where people bounce ideas off each other will always produce the best software.

Here are 12 tips for team members:

  1. trust team-mates to do their job
  2. be open to new or different ideas
  3. take responsibility for committed tasks
  4. always find a way to make a contribution
  5. be open to helping in areas outside your expertise
  6. look at things from customer viewpoint
  7. don't be defensive when someone points out a mistake or a better way
  8. don't hide your mistakes
  9. don't be afraid to give negative feedback BUT
  10. don't be offensive in pointing out other people's mistakes
  11. be polite and respectful
  12. never put personal ambition ahead of the project
Culture

As I discussed a few years ago (see Why Scrum Fails) the root cause of the failure of Scrum is the culture of the organization. The team will not change their mind-set if the culture discourages it.

First, managers need to relinquish their control, so the team can effectively self-organize. Rewards should encourage collaboration - ie, team rewards, not individual bonuses. Encourage the team to identify and fix their own problems.

Most importantly, it is important to drive out fear. People won't take risks if they are fearful of the repercussions when things don't go to plan.


Conclusion

Scrum is centered around the team - a small, stable, self-organizing, self-managing, collaborative team. When the team works well together Scrum works very well. When the team does not collaborate Scrum does not work well.

Unfortunately, many developers by their nature are not prone to collaborative behavior. Luckily this is not an inherent limitation but is a mainly a conditioned response due to years of working in an oppressive environment.

The advantages of removing an untrusting, blaming culture and overcoming developers defensiveness to develop a collaborative team are big. As explained above there are many benefits but in essence better results are produced, often more efficiently and with greater satisfaction to customer and developers.

Many of the practices of Scrum help to encourage collaboration such as the visibility provided by the daily standup and the focus provided by the backlog. Feature teams as discussed last week are also important. Further, other Agile practices such as continuous integration, collective code ownership, customer focus, collocation, etc are very useful.

The most important thing is to change the environment to one where the developers are empowered to do their best and rewards are team-based not individual.

Further, developers need to be open to the change and fight their defensiveness and preference for working alone. It may be surprising that working as part of a highly-functional small team can be deeply satisfying.


Friday, 3 April 2015

Scrum Team Composition

Last week we saw how important it was to get the size of a Scrum team right. This time I will look at another issue that is just as important - team composition or who to put on which team.

Of course, if you only have enough people for a single team this blog may not be relevant to you. However, there are things of interest such as a look at personalities of team members and how that affects collaboration. [Late note: I have split off the bit about collaboration into my next post, so I only cover it briefly here.]

Personalities

The biggest problem I have found with the successful adoption of Scrum is team members who seem incapable of working effectively together. This can mostly be put down to personalities. First, different people sometimes just don't get along, for no obvious reason. This is why a bit of consultation, judgement and experimentation may be required when composing teams.


Typical developer personality profile
(provided by DISC Personality Testing)
More significant is that most developers generally have a personality type that rails against the collaboration inherent in Agile methodologies. If you are familiar with the DISC system of personality classification and assessment you may already be aware that software professionals are generally of the S personality type something like in the diagram. Importantly, for our discussion, it means they prefer to work alone. (However, I believe this is, at least partially, a learnt or conditioned behavior, as I discuss next week.)

This causes the sorts of problems you have probably seen, such as continual disagreement about approaches to technical problems. Less obvious is the developer who is uncooperative with Scrum practices or other areas of their work that they are unhappy with - they may be ostensibly open and helpful, and may not even be aware of their behavior and how it causes problems. Most important is how personality affects collaboration between team members.

Many companies adopting Scrum attempt to hire developers of the desired personality. This is why a lot of job ads emphasize the "team player" aspect. Some companies go as far as performing personality testing (such as DISC) on job candidates. I think this is the wrong approach as most developers can be encouraged to be collaborative given the right environment which I discuss next week.

Team Structure

I already talked about the different skills required (and not required) in an Agile development environment - see Agile Roles. What I did not cover is how to structure the teams, basically which people to put together in the same team. The basic rule is that you need all the skills in a team required to complete the tasks in the backlog of that team. I will elaborate on this below but first we look at why it is important.

Coordination

A major consideration when composing teams is to reduce the costs of communication by putting those people on the same team that need to work closely together.
Component vs Feature Teams   

In Succeeding with Agile, Mike Cohn calls teams structured around the architectural design of the software Component Teams. He contrasts this with teams structured according to customer requirements which he calls Feature Teams.

See Succeeding with Agile [ISBN-13: 978-0-321-57936-2] pages 183-188.
The simplistic way to do this is to group people working on the same sorts of things - so you might have an analysis team, an architecture team, coding, testing, installation and support teams, etc.

The developers too are split into sub-teams based on different parts of the software they work on, Examples I have seen are a database team, services layer team, display engine, AI, comms, backend server, web interface, etc. In Agile jargon these are called Component Teams (see note at right).

Component Teams do not work well in an Agile environment, as we will see, but even in a traditional non-Agile project they cause problems like:
  • silo syndrome (see Production Line Mentality)
  • unnecessarily complex designs (see below)
  • contorted designs due to changes made in the wrong place
  • modules with unneeded features due to perceived future needs
  • over-specialization
  • uneven workload
  • dependence on multiple teams to meet a deadline
  • code ownership problems (see below)
  • narrow focus on a specific task/module
  • lack of focus on delivering what the customer wants
  • isolating people who often need to communicate
  • need for lots of "interface" documentation
  • a lot of integration testing work
  • requires a lot more planning
An alternative is to be customer-oriented when composing teams. Each team can focus on one or more aspects of delivering something useful to the customer - ie, Feature Teams. Feature teams are focused on completing tasks on the product backlog. Typically these would be about different aspects of the user-interface, but other things may be important to the customer, such as data-compatibility with their systems.

Software Design

Talking about software design may seem completely off topic here, but the design of a piece of software is closely linked to how the people creating it interact. Some say that organizational structure determines software architecture. It could be the other way around. In either case it is important for those involved in the system architecture to also be involved in the creation of the teams.

Interestingly, this relationship might also explain why the Layer Anti-pattern is so common. I have noticed the tendency of organizations with many layers of management to also develop software with lots of layers. The Layer Anti-pattern demonstrates poor or unnecessary separation of concerns which interferes with the ability of modules to communicate and adds to the complexity of the software. Analogously, a layered organizational structure interferes with collaboration. Both are examples of how a simplistic layered approach can work against the DIRE.

DIRE

This whole discussion can also benefit from a consideration of the DIRE principle. Conventional thinking is that people doing the same things should be on the same team - a Component Team. Using the DIRE principle we compose teams in such a way that people who need to work closely together are on the same team. (In this case the team members are the "related entities" - see DIRE for more info.)

Further, this is a result of the Agile emphasis is on delivering useful features to the customer, by performing the tasks for a particular feature at once (or as close in time as possible). This is also an example of DIRE - in this case DIRE stands for "Don't Isolate Related Events".

Component Teams

Before we look at feature teams in detail let's cover how component teams work. Many of the deficiencies of component teams are due to developers employing "silo" thinking. See Production Line Mentality and Silo Syndrome for more on this -- but here I just discuss problems specific to software development.

Note that even teams that deal directly with the user interface may still be component teams. For example, the "GUI" team in the diagram below is a component team (as, of course, are the other two teams) since the development of most features would typically involve at least one other team.

Component Teams
In the following discussion I will discuss a theoretical component team with one or more dependent teams -- such as the Math Team in the diagram, which has the Services Team and the "GUI" team as dependent teams.

Complexity

Using component teams tends to lead a project towards the dark side called BDUF (big design up front), which I hope you are aware is very bad and the antithesis of the Agile approach -- if not see Agile Design. Components developed like this are always overly complex, with many unneeded facilities. This makes the code harder to understand, harder to verify, and makes it less amenable to modifications. In my experience, the worst thing is that it's harder for the users of the component (ie, dependent teams) to understand how to use it (see Gas Factory Anti-Pattern).

premature decoupling
is a common problem
The Agile approach is to develop the bare minimum that is required to allow the customer to use the software. The very first usable version of a program may be ready in just a few days and have no discernible components. It will later be refactored by decoupling different parts of the design when necessary, and, significantly, when it is more obvious how to best do so. Premature decoupling is a common problem even in Agile development.

If the team is certain of the best way to decouple (and that decoupling is even necessary) then it can be useful to create a simple component early in the evolution of the software. This simple component will initially be very small (perhaps only a few lines of code) but will be added to and refactored over time, as backlog items dictate.

Over time, components tend to be "owned" by one team (or even one individual). If other teams need changes to the component then they depend on the owner to deliver those changes. The component team then tries to anticipate what dependent teams will require to avoid the dependencies interfering with sprint commitments. As I have mentioned in the past, lots of other unnecessary stuff creeps in when the "owner" is focused on the component and not the backlog - eg: things that are "trivial to add and nice to have", "will definitely be needed eventually", etc. Moreover, these "bonus" features add to the debugging, testing, and planning burden, but most of all they make it harder to refactor the code and make the changes that are actually needed.

components tend
to be owned
by one team
Apart from BDUF, there is the problem that component teams naturally tend to focus just on the component, instead of the backlog items (ie, what the customer actually wants). They will have an idealized picture of what the component should do that is out of tune with what is really needed. They need to focus on customer requirements not component capabilities.

One consequence is that the dependent team(s) often have to make do with a component that is not really suited for what the customer needs. This results in a misshapen design which inevitably leads to bugs, complexity, and difficult to maintain software. This is a major reason why software degrades over time and has to be written from scratch (see Why Good Programs Go Bad).

CASE STUDY 1

I once worked for a company which had developed a world-leading piece of software that was heavily reliant on the display and manipulation of 2-D graphical objects. The company had developed a remarkably useful library for 2-d computational geometry. They had the good fortune to have a team of mathematicians who were also good C++ programmers who had created this component.

One of the most powerful features of the software was the display and manipulation of text. Among the text options was the ability to display text on many different base-lines: straight lines, lines at any angle, along a Beziere curve, etc. One of the base-line options was the ability to place the text on the inside or outside of a circle or ellipse. Of course, the geometry library provided an ellipse class which was used for many things and was the obvious thing to use for this type of base-line.

Like all the graphical objects you could apply an affine transformation to the ellipse class - for example, to move, rotate, enlarge, etc. The problem with using an ellipse as the base-line for text was that when it was transformed it was impossible to determine how it had been transformed. Even if you knew the before and after you could not determine what transformation had been applied. For example, an ellipse that was elongated in the X direction could be transformed into an ellipse elongated in the Y direction either by rotating it 90 degrees (or 270 degrees) or by reducing the X scaling and increasing the Y scaling.

In order to transform the text to match the base-line you had to know where each point on the ellipse had moved to. This was impossible to do if you only had the before and after ellipses to work with.

Unfortunately, the "geometry" component team were unwilling to modify their perfect ellipse class to facilitate its use as a base-line. The result was a kludge at a higher level in the code which caused bugs and maintenance problems.

Code Ownership

Component teams can become very protective of their work. I have seen some go as far as not making the source code available outside the team. Even if they do release the source, it may be strictly forbidden for anyone outside the team to make changes to the component, or even submit code to be included in the component. Change requests often need to be submitted in writing and can take months to be considered, approved and (if you're lucky) implemented.

As you can imagine, and may have even experienced, this can be very frustrating to a dependent team dealing with a customer who expects responsive changes. The component team may be unavailable, too busy, or simply not interested in making the change. A common problem is that the component team is unwilling to corrupt the purity of their component by acceding to the whims of the customer! Does this sound familiar?

One of the principles of XP (the guiding light in these areas of Agile development) is that there should be no code ownership (see Collective Code Ownership). Any team should be able to make changes to any of the code that is available to them. Of course, that is not possible with third party components without source code - but that is just another argument for the use of open-source libraries.

The key thing is that, if modifying the component requires specialist knowledge then each team needs at least one developer (even if shared) with that knowledge, or needs to acquire the knowledge. (Of course, the need for lots of specialists is symptomatic of a non-Agile environment as I discuss below.)

CASE STUDY 2

There was once a company with a software product that dominated it's market throughout the world. It was used by some of the largest companies. Most of these clients required a high degree of customization of the software to suit local markets, regulations, etc.

This was a large piece of software based around the template design pattern - essentially a "core" system with many extension points. The "core" component was tightly controlled by the main development team at the head office of the company. Several on-site customer teams used the extension points to customise the software to individual customer requirements.

* This configurability came at a large cost. The main problem was that it added a great deal of complexity making it hard to understand and modify the system. This also had knock-on effects such as poor performance.
However, despite it's immense configurability* there were many customer requirements which could not be accommodated, and required (or should have required) core changes. As you can imagine, it was extremely difficult to get timely changes made to the core system, if they were done at all. Scheduling conflicts meant that requests for changes needed to be submitted many months in advance. The teams at client sites found the core team arrogant and uncompromising. The core team were upset at the barrage of stupid and conflicting requests.

Inevitably the integrity of design became compromised. Many facilities were simply tacked on to client systems rather than being properly implemented in the core. This contorted the design and created maintenance problems. Moreover, the same thing was often implemented by different client teams in different ways - an obvious violation of DRY.

Planning

Component teams require a lot of coordination and planning to ensure that the components work well together and the development of all components is on schedule. (A great advantage of Scrum is that it requires much less planning work.)

Integrating components and integration testing is also required, but if coordination (and documentation - see below) between teams is inadequate then integration tasks can blow out enormously.

Using component teams also makes it far more likely that deadlines will be missed. It only takes one team to be late for the whole project to be delayed.

Component teams are also more likely to suffer from uneven workload. Sometimes they have little to do; other times they are overloaded due to different dependent teams waiting for critical functionality. Component teams will invent work if they have none, which can be good if it is refactoring the code to improve its quality (maintainability, efficiency, etc), but it is often unrelated to any current customer requirements.

A common scenario with highly-specialized component teams is that they are formed near the start of a project then quickly complete most of the required functionality, whence the team needs to be down-sized or even disbanded. This can lead to valued developers leaving the organization because it seems that their work is complete and they have nothing much of interest to hold them. Alternatively, the organization will start inventing work for the component team - but this work tends to become less and less customer-focused.

A project using component teams often needs to be reorganized which is counter-productive. The truth is, it's hard to build a good team that works well together - at least partly due to personalities (see above). Once you have a well-functioning team you don't want to change personnel too often. Many Agile teams can go unchanged for years, pumping out quality software.

Agile teams can
go unchanged
for years
A big problem for a Scrum team working in a large organization is "external dependencies". Scrum teams will strive to finish their sprint backlog by the end of the sprint but are continually frustrated by being dependent on people outside the team. If you have this problem then it indicates that the teams are not really feature teams.

Documentation

Component teams tend to produce lots of documentation, since they need to explain the interfaces so that other teams can use their component(s). Of course, as you know, Agile favors "working code over documentation". We all know the problem of documents (time-consuming to create, difficult to understand, incorrect, out of date, etc), so how can we avoid this problem?

Another Agile principle, continuous integration, means there is far less need for documentation. If the component and the code that uses it are developed at the same time (by the same team) then this avoids the need for documentation, hand-overs, integration testing, etc. Continuous integration works far better with feature teams than component teams.

Continuous integration and feature teams work together to reduce the need for large amounts of documentation.

Feature Teams

Feature teams are a different way to structure a project. Here we try to avoid having a team that is dependent on any outside person or team in order to complete their backlog items. After all it's only the members of the Scrum team who are completely focused on meeting the sprint objectives.

feature teams
can and
should specialize
Note that there is some dispute about what constitutes a feature team. This is caused by some people insisting that feature teams do not specialize. The fact is that feature teams can and often should specialize.

For example, a common definition of a feature team (which I believe is inadequate) goes something like this:

A cross-functional team that has all the skills required to complete items from the product backlog.

This definition implies that all feature teams are equivalent since they must be able to complete many or all items from the product backlog. It assumes that the product backlog has, and always will have, items of a similar nature.


Feature Teams
The problem is that real-world projects have backlog items of amazing variety. Consequently, different feature teams can, and probably should have different capabilities. I discuss this below under Specialist Feature Teams.

The diagram at left takes the members of the three component teams (in the component teams diagram above) and creates three feature teams. Note that there is a specialist "Charts" feature team which is the only team that requires developers with knowledge of the geometry library and the charting component.

Specialist Feature Teams

A feature team can specialize as long as it specializes in things that are important to the customer. As an example you might have a feature team that does not deal with the user interface; instead it handles interfacing with other client systems since it may be critical for the customer to get up-to-date information (eg, inventory levels). A team concentrating on this area of the software would have specialist knowledge very different to that of feature teams dealing with the GUI.

teams
should be
long lived
Of course, even such a specialist feature team should not be so specialized that they can't work on other general backlog items.

We also need to keep in mind that feature teams should be long lived. Before creating a specialist team it should be fairly certain that they would have an ongoing supply of high-priority backlog items to suit their speciality.

Scrum Jargon       

Product Backlog: prioritized list of tasks for the project

Backlog item (PBI): a single task on the product backlog
General item: a backlog item that can be taken by any feature team
Special item: a backlog item to be taken by a particular feature team
Team Backlog: Subset of product backlog allocated to a specific team
Sprint backlog: the items that a team commits to complete in a sprint
Specialist team: preferred team to deal with special items

Collaboration

Feature teams are a great example of DIRE. They promote collaboration by getting people who are responsible for a feature to work closely together.

This collaboration has the advantage that the team members are all focused on a common goal rather than their own small area of interest. Moreover this focus (on high-priority backlog items) is of direct relevance to the customer.

This sort of collaboration helps avoid the Silo Syndrome (see Silo Syndrome in Production Line Mentality) and over-specialization (see below). It also addresses coordination overhead by greatly reducing the need for written documentation, etc.

Evolutionary Design

The basic Agile approach is that changes are made in small increments, so that the software gradually evolves into what the customer wants. By taking "small nibbles", slow but steady progress is made. The overhead of large hand-overs is dramatically reduced and integration problems eliminated. Further, the customer is happier because they can see what is happening. (There may even be a perception that the development took less time than it actually did.)

evolutionary design
is easier
using feature teams
The only advantage to taking "large bites" instead of "small nibbles" is from economies of scale. However, (as I already discussed in Agile Design) this is far outweighed by other problems like reduced visibility. To continue the analogy, there is a tendency to bite off more than one can chew.

However, each small nibble may possibly involve changes in many different areas of the code. Hence a team requires all the skills to make the changes. That is, evolutionary design is easier using feature teams.

Specialists

A big problem with a lot of software, especially much legacy software, is that it can only be modified by a small group of specialists. Whether this happens intentionally or not, it works as a form of job preservation for them. Due to a poor design and a lack of documentation only the "gurus" who understand how it works can modify it; these are usually those who wrote the software in the first place. (Note that by "poor design" I may simply mean that it is unnecessarily complex, though typical legacy applications have many other problems.)

These specialists are only required due to the complexity of the design. It's usually not that the component is performing an inherently difficult process; it is simply the way it has been designed and built (and not documented).

Feature teams are far less likely to create such components since there is no code ownership. An underrated advantage of feature teams is that they reduce dependency on these "specialists".

Finally, consider the worst case scenario. This is where somebody (usually an existing component team) decides to build an in-house component, rather than use a perfectly adequate off-the-shelf or open-source one. Typically the project must then deal with an expensive-to-implement, poorly-documented, hard-to-use component for little, if any, advantage. A feature team is focused on giving the customer what they want, as simply as possible, so is much more likely to go with the open-source solution.

Feature Team Problems

Using feature teams does increase the likelihood of conflicting code changes, but this is not a major problem especially if you take my advice and allow some team specialization. Often conflicting changes can be merged with little or no problem. Sometimes it may require a few team members from different teams to get together and work out how to combine the features. Occasionally the teams may need to go back to the PO(s) to point out conflicting requirements.

One thing that worries many people is that they think chaos will ensue if all developers can modify any of the code - the fear is that the quality of the code will rapidly degrade. This simply does not happen as long if there is someone on the team with the skills needed to do a proper job. (If a team member must make changes outside their area of expertise then it should be carefully reviewed by the relevant expert.)

One problem is if there are not enough specialists for all the teams. In this case specialists should be shared between teams, and encouraged to pass on their knowledge. (But remember, as I discussed above, feature teams do not tend to create components that require specialists anyway.)

Task Allocation

Once you have multiple feature teams the next problem is how to assign tasks to the different teams. This depends on your PO(s) (product owners). If you have a single product owner for all the teams then you need a single product backlog from which all teams derive their own team backlogs.


Even with multiple PO's it is beneficial to maintain a single product backlog for the project. Of course, this requires the PO's to meet to work out the relative priorities of all items and assign them to different teams. (This can be difficult with vastly different items, like comparing apples and oranges.) As an example, the diagram above shows a product backlog split between three teams.

In doing this it is important to carefully consider the task assignments, depending on the work-load and experience of the teams. For example, the same team should probably perform related items from the same Epic. In the diagram you can see that the Charting Team has enough high-priority Charting Tasks, but if they didn't they would need to work on other tasks like the General items. The PO(s) should ensure that each team has enough tasks in their backlog for at least two sprints (based on rough estimates given in grooming).

One obvious, but important point, is to ensure two teams do not accidentally end up working on the same task.

CASE STUDY 3

This case study concerns a Scrum team (one of several) in a large company, tasked with supporting and maintaining a legacy application running in thousands of locations. The software was highly configurable (and regularly configured). Apart from the PO and SM the team members were:
  • 5 testers
  • 4 configuration experts
  • 3 developers (coding in 3 different languages)
  • 2 business analysts
Most of the PBIs (product backlog items) were configuration changes - requiring configuration experts, testers and often a business analyst, but no developers. Apart from config. PBIs there were also PBIs for small software changes and occasionally large developments - mainly requiring coding but with possible involvement from anyone in the team. Even if there were no software enhancements in the product backlog there were always low-priority (usually historical) bugs to fix.

The product backlog typically contained mostly configuration and general tasks with a few low-priority bug fixes (see Typical Product Backlog in the diagram). For a fortnightly sprint the sprint backlog would be composed of the highest priority tasks (mainly config) plus a few development tasks to give the coders something to do.


However, there were sometimes high-priority developments, in which case the product backlog would look very different (see "Development" Backlog in the diagram).

The major problem that the team experienced was due to the mix of tasks. Most of the time the high priority tasks were configuration and of little or no consequence to developers. Sometimes there was a major development that was of no interest to most of the configuration experts. In most meetings (planning, standup, grooming) there was a lot of discussion that was only understood by a subset of the team, which meant there was a huge amount of time being wasted by people in meetings that were not important to what they were focused on.

Component Teams
There were other problems such as many tasks being highly dependent on external people - indicating that this was not really a feature team. Another thing was that the configuration guys preferred to have a standup at 8:30am, while the developers preferred 10:00am. It was also demoralizing for the developers to be continually handed low-priority "scraps".

It was obvious (not just because the team was too big) that the team should be split into multiple teams. So how would you split up the team?

It was suggested to use a team structure of Testing, Configuration and Development teams.  This was similar to the teams that existed before we adopted Scrum. See the Component Teams diagram above.

Feature Teams
A better approach would be to create two (or even three) feature teams to deal with the types of backlog items that were commonly encountered. The approach I recommended was to have a Configuration Team that dealt with configuration backlog items and a Development team that would deal with programming tasks (and other tasks if there was not enough development). See the Feature Teams diagram at right.

Each team would have their own backlog derived from the product backlog (see diagram below) and would not have to deal with meetings and discussions about tasks that they did not need to focus on. (Of course, that did not mean whatsoever that communication between the teams was discouraged, when necessary.)



Conclusion

An important part of implementing Scrum is to carefully choose the composition of teams.

First, it is important to consider personalities when building teams. This requires good judgement and a measure of luck to get right. Don't be afraid to experiment and change teams around if things are not working.

From an Agile standpoint it is also important to get the skill mix right. Scrum teams should be formed around customer needs not around the software architecture -ie, use feature teams not component teams. One of the most frustrating things for a Scrum team is repeatedly failing to complete sprints due to external dependencies.

As well as creating external dependencies, component teams also:
  • need more planning and coordination effort
  • tend to produce overly complex designs
  • create and depend too much on specialists
  • focus on their own small world instead of the big picture
In contrast, feature teams encourage collaboration and avoid the problems of component teams. Amongst their advantages are such diverse elements as:
  • keeping all team members focused on the customer
  • making specialization less likely (see Production Line Mentality)
  • putting together team members that need to work together (see DIRE)
  • allowing teams to work without being frustrated by external dependencies
  • focusing on only delivering what is required now (see Fifth Element)
In brief the rules/attributes for feature teams are:
  • the team has all the skills to tackle tasks on their backlog
  • every team member could make a contribution to each backlog task
  • specialist teams are useful if the backlog has different classes of PBIs
  • you anticipate enough special items to justify creation of a specialist team
  • feature teams are stable and long-lived
  • feature teams are highly collaborative and focused on the customer
Next week I will look at personalities and why teams do not collaborate.

Monday, 30 March 2015

Scrum Team Size

One of the most important aspects of a Scrum team performing at their best is to get the size and composition of the team right. Size is simple, but it's importance is not generally recognized or explained. So I have taken it on myself to explain it here.

Composition is a bit trickier. Its not just about skill sets (which I already covered last year - see Agile Roles) but also about personalities and how members get along. Another consideration is how and how often teams should be reorganized. Due to lack of space and time I will leave the discussion of team composition until next week.

Scrum Team Size

I have worked in teams from a size of 3-4 to a team in the range of 14-18. In my experience a good Scrum team size is about five with a maximum of about seven. Depending on the people and the type of project you might be able to get away with a team size of nine.

I have read many books and articles on Scrum and the absolute maximum team size is always specified as either seven or nine. For example, I am currently reading an excellent book called Succeeding with Agile by Mike Cohn who says that the accepted rule is a team size of five to nine, but recommends (page 178) the less regimented rule of a team that you can feed with two pizzas. Mike mentions that this is a convention used at Amazon. (I'm not sure if this means that you can have a team larger than nine if you get larger pizzas or smaller developers:) Later on I will tell you how to split a team that is too big.

Scrum rules also give a minimum size of five, though I have also seen three mentioned. In an extreme case a small software company may have to make do with a team of three - two developers (one of whom acts as a part time Scrum Master) and a Product Owner (hopefully provided by the client).

Every Scrum article, book, expert and consultant emphasizes the importance of the team size, so don't question it! If you insist on questioning it, despite my advice, then I can give you several arguments which I explain in the next sections.

Social Loafing

Social loafing has been extensively studied starting with RingelMann about a century ago - see Ringelmann Effect.

I talked about this at length in my previous post.  In summary, social loafing is the effect that as more people work together to perform a task then the less effort each individual contributes to the task. The effect increases as the number of people increases but is measurable even with two people. Most studies show a pronounced drop when the group size reaches around seven to twelve people depending on the type of task.

Try to limit the size of Scrum teams if for no other reason than this.

Visibility

An important feature of Agile that counteracts social loafing is visibility.

Visibility was originally designed to assuage the fears of the customer. The PO (and other stakeholders) can, at a minimum, see that progress has been made at the end of each sprint. If they want even greater assuagement (is that a word :), they can examine the sprint board on a daily basis. Of course, the PO does not, and need not, know what each team member is doing but they can easily see progress as tasks move to "done" status.

peer pressure
counteracts
social loafing
However, visibility is also important within the team. In Scrum everyone in the team should know what everyone else is doing. This is essential if the team is to self-manage and self-organize effectively. It becomes very obvious when someone is not pulling their weight. The resulting peer pressure counteracts social loafing.

Obviously, as the team size gets bigger it becomes harder for team members to understand what everyone else in the team is doing. Studies have shown that even the most skilled multi-taskers can at most keep track of nine separate activities. My experience, and that of many others, is that a team of nine is the absolute maximum, while even seven is getting too big.

I have heard the argument that a Scrum team is like a sports team (such as a rugby team), and since a rugby team has 15 players you should be able to have a Scrum team of that size. (First, I would say that a rugby team is composed of two cooperating teams - the forwards and the backs - but I don't know enough about the sport to argue that further.) It's true that a Scrum team is a bit like a rugby team but the main difference is that individuals in sports team have much greater visibility - it is much more obvious what each player is doing. Another difference with sports, is that typically only a one or a few players are actively involved in the game at any time, whereas in a Scrum team everyone is working diligently all the time.

the major benefit
of the daily standup
is visibility
Scrum practices are important. In particular, the major benefit of the daily standup is visibility. This is why it is very important to make sure the standup is working properly as I discussed here a few years ago (see October 2012).

Remember, visibility is essential for counteracting social loafing. The team size should be small to facilitate this visibility otherwise team members can easily fade into the background.

CASE STUDY :)

A colleague once worked in a Scrum team that was far too big at a size of more than 20. As an "experiment" he tried several different strategies to avoid talking or saying anything useful at the daily standup. Due to the size of the team he found it very easy. Sometimes nobody would notice if he did not even attend!  This demonstrates how important a small team is for visibility.

Team Spirit

Having a good team spirit is another way to counter social loafing. Many of the principles and practices of Agile encourage the team to work closely together which fosters team spirit.

To invoke another sporting analogy consider times in relay events such as swimming, running, etc. According to the Ringelmann Effect, times of athletes in relay events should be slower than their times in individual events. However, it is not unusual for athletes to perform as well or even better in a relay. This is entirely due to team spirit.

There are many aspects of Agile that promote team spirit.
  • focus on customer requirements
  • having a clear and common understanding of team goals
  • encouraging collaboration and having the team self-organize
  • rewarding the team, rather than the individual team members
  • encouraging open, honest communication
  • empowerment and self-management
  • motivation of team members to do their part
  • team members feel that they are making a valuable contribution
Notwithstanding Agile practices, team spirit may be affected by other factors, such as the personalities of the team members (something I talk about next week), competition with other teams, etc.

team spirit decreases
the larger
the team becomes
My point here is that team spirit diminishes the larger the team becomes which is another reason not to have a large team.

Competitive Spirit

Talking about team spirit I recalled that another way to get teams motivated is to encourage a bit of inter-team rivalry. I have actually only seen this work well once, where I was working with 12 to 16 recent university graduates. There was an interesting coincidence in that the group was almost exactly split between electrical/electronic engineering and software engineering graduates. I split the group into two teams based on their degree and tried to objectively compare the performance of each team, even displaying graphs of work completed. This worked well to motivate the teams, at least for a while.

However, I recommend caution in using this approach. First, you probably shouldn't do this if the teams need to work together in any significant manner - it does not encourage collaboration. Also you must keep it friendly and be entirely honest and open about it, otherwise it will be seen as manipulative.

Over-Specialization

Another problem with a large team is that team members become prone to Production Line Mentality. With many developers there is too much opportunity to specialize in one particular aspect of the team's responsibilities. (The same can happen with entire teams as I discuss next week.) This leads to developers who are unwilling, even unable, to perform any tasks outside their area of expertise.

This has many undesirable consequences. First, there may be times when the specialization is not required and the specialist has nothing to do. Conversely, if the specialist leaves or has too much to do then there is often nobody with the experience in the area who can take over or help.

But the worst problem, in my opinion, is that specialists tend to concentrate on the their own narrow area of focus, rather than being aware of, and striving to achieve, the goals of the team.

Don't get me wrong. It is often very useful to have developers with specialist knowledge. However, they should be willing and able to work in other areas as required (On the other hand you should not waste their talents unless there is a very good reason.)

Coordination

So Scrum helps to counter social loafing, but there is another reason for the Ringelmann Effect besides social loafing. For complex tasks, such as software development, coordination overhead becomes an even more significant problem.
Complexity Grows Exponentially
as Team Size Increases

There are two aspects to coordination. First, there are technical issues to do with building the software - these are not addressed by splitting into smaller teams because there will still be a need for inter-team communication. It might be useful if there is a logical separation of tasks based on feature areas of the software so that the new teams can concentrate on different areas, but this does not reduce the need for coordination, just contains it within the separate teams.

The other issue of coordination is getting everyone on the team to work together to knock off the sprint tasks. This is where having two small teams instead of one large one can reduce coordination effort.

Some developers try to take this argument to the extreme, arguing for a "team" size of one. It's true that the most efficient way to produce software may be just to have one good developer go at it full-time, without interruptions. However, there are disadvantages to having too small a team as I will discuss next.

Minimum Size

So much for the maximum size - what about the minimum?

By extrapolation of the arguments above you might conclude that a team size of one or two is best. It's true that a lot of problems, especially of coordination, disappear when a single developer (or even two) works on a project. There is actually some validity in this argument - for example, there are many great pieces of software that have been created by one or two people.

But I think the main motivation for this argument is that most developers have the personality type that prefers to work alone. Moreover, industry training and years of behavioral conditioning have reinforced this preference. I don't have enough space to elaborate here but this is important enough to warrant a future post.

You probably already concede that for large projects using one very small team is silly. It might be most efficient to have one developer take five years on a project but in that time priorities may have changed. Even if inefficient, it is likely preferable, for example, to assemble a team of eight and complete it in one year. There is also increased risk due to the single point of failure.

On the other hand, if a large project is tackled by many small teams, then this is effectively the same as having one large non-communicative team. Unless you are able to split the work into many independent parts then you must encourage communication and collaboration by building teams of at least four.

So I recommend not having a team size smaller than four or five. If absolutely necessary you could have a team of two developers (one of which is a part-time Scrum Master) plus a Product Owner (possibly part-time and possibly supplied by the client).

Splitting Teams

The obvious question now is if you have a large or growing team, how do you split it?

Once a team gets to a size of 8 then you should look at splitting the team into two. One way to do this with a team composed of one PO, one SM and 6 developers is to create two teams of 5 which share their PO and SM and have three developers each. If you already have teams with shared members (for example two teams of 7) then you can look at splitting into three teams of 5.

I have heard the argument that sharing a PO or SM between teams means more work for them. But working with two small teams should not really create much more work for the PO/SM than one large team. But it *will* save a great deal of time for the other team members if they do not have to attend long standups, planning meetings, etc (as well as having all the other advantages of small teams that I mentioned above).

Shared PO  

Having a shared PO between two teams should not result in more work than a PO with one big team.

However, I do not recommend having a PO shared by more than two or three teams. This would just be too confusing.
Another argument I have sometimes heard, for not splitting a team, is that there is no natural way to separate a large team as they are all doing similar work. It doesn't matter, split it anyway. Two teams can even work from the same product backlog, as long as they have a shared PO. Of course, it is probably best to have an equal mix of experience, familiarity with Scrum, etc on both new teams.

A crucial point when splitting is to resist creating "component" teams (teams that create components for other teams to use). Each new team should have all the skills to create new features. In other words each team should be capable of completing any (or a large proportion) of tasks from the product backlog. This is another good example of DIRE since it gets people on the same team to work together towards the same goal. Not only does it promote customer focus, but avoids problems like external dependencies, over-specialization (as above), etc as I will explain next week when we look at team composition.

Conclusion

There are several effects detrimental to large teams. Generally these effects are seen in teams of any size but start to really kick-in once a team reaches about seven or eight people. These include:
  • increase in social loafing
  • reduction in team spirit
  • coordination overhead
  • inefficiencies due to over-specialization
At the other extreme, having very small "teams" of one or two people may address these problems, but to do this the work must be divided into many different tasks for each of the "teams". This introduces other problems:
  • some tasks are not easily divided
  • some tasks may take longer than others
  • increased coordination between teams
  • single points of failure
Moreover, when you have many small teams working together on a large project you effectively have one large loosely-coupled team, whence the large team problems start to rear their ugly heads again.

It seems that, even with their drawbacks, teams of size five to eight are the best size. The good news is that team spirit and many Agile practices will minimize these drawbacks. In particular, the visibility provided by Scrum's daily standup greatly reduces social loafing.