Agile Roles

Back in May I wrote about how software design is done in Agile. I received some feedback which I promised to respond to in a future post (this one). Sorry for the delay.

Someone asked about the role of designers in Agile. They thought I was implying that specialist designers were not needed. Well, just to be clear, I will say it outright: No dedicated designers! All or most of the team should be involved in the design though typically some will have more input than others.

"No dedicated designers!"

When moving to Scrum people who just do "design" need to stop it. Designers traditionally are either technical (sometimes called architects) or business-oriented (sometimes called analyst/designers). I talk below about how these (and other) roles may change when moving to agile but in brief designers can move in two different directions - either to more development work or to more of a PO type role. Architects often become more hands-on - spending more time coding and reviewing. Analyst/designers can move back to development or if not from a technical background will need to move into a role involving mostly acceptance testing and customer interaction.

Another comment was: "If design, programming (and testing) are to be done together then should they all be done by the same person?"  In essence: YES. But before I get into that I need to talk a little about roles in Agile teams.


Multi-skilled Teams

Everyone agrees that Agile development uses multi-skilled (or cross-functional) teams. The problem is nobody really agrees (or even pretends to know) exactly what that means. Does it mean each of the team members have the skills to do all, or at least many, of the required tasks? Or does it mean that altogether the team has the required skills but each specializes in their own skill-set?

In my experience most Agile "experts" use the latter definition. For example, I was recently involved in the implementation of Scrum in a large company, and one of the things that the Scrum consultants emphasized is that a Scrum team (ie, "pigs" not "chickens") is made up of members with a wide variety of skills. However, I get the strong impression that this is mainly to placate those employees who feel that they may be redundant in the new order of things.

In reality, in an organization suffering from production line mentality (see below) adoption of an Agile methodology will mean that there will be less need for some existing roles. Less BDUF (see my post on Agile Design) means that there is no need for dedicated designers as design is performed continuously by the developers. Also, developers take more responsibility for testing so there is less need for dedicated testers doing black-box testing (like regression tests). This means that developers have more responsibilities (rather than just coding), but this is a good thing as it promotes DIRE (see DIRE) but also makes the work more rewarding and interesting.

If you need less people for other roles, such as designers, does this mean you need more developers? The simple answer is no. In my experience the same developers can actually be more productive, so you may even need less developers too!

Here I should note that I am talking about using Scrum in a pure software development environment.

I know that Scrum has been used successfully in other environments and that is great. However, whatever the environment all team members must have enough shared context to understand what everyone else in the team is doing.

Scrum Team Composition

Ideally all members of a Scrum team should be developers. That is, team members (apart from the Product Owner and perhaps the Scrum Master) should have the ability to write code. To understand why I will first recap on the advantages of Scrum.

Scrum (and Agile in general) has many benefits such as promoting team-work, working towards a common goal, quick response to change, etc. This is achieved by good communication and empowering the team to make their own decisions. The daily standup is particularly important as it allows everyone to know what everyone else is doing.

When everyone in the team can (and does) understand what everyone else is doing this avoids issues like:

• differences in understanding of what is to be done
• poor division of work
• duplicating work - such as two developers fixing the same bug
• forces pulling in different directions and introducing conflicting changes
• not realizing when someone needs help
• not being able to have someone else take over a task due to illness
• not being able to have the whole team focus on a critical problem
• inability to find another team member who understands what you are doing well enough to perform a review

Unless the team members are all developers they cannot really understand what everyone else in the team is doing (and how, why, etc). This is essential for Scrum to work (see Why Scrum Fails).

Of course, different individuals will have different levels of experience and different aptitudes. Over time, team members become familiar with each other's abilities. It is easy and natural for team members to take each other's skills into account when allocating tasks during the sprint planning.


Production Line Mentality

The production line was a major invention in the history of industrialization, since even unskilled workers can become very good at a simple repetitive task. However, its drawbacks were quickly discovered. Workers suffer from frustration and alienation. Once a task is mastered boredom sets in and mistakes are made, even for simple things.

Quality and productivity are greatly improved when workers are given a variety of challenging tasks. Performing different jobs as part of a small team working towards the same goal can be a tremendous motivator.

Another problem with production line mentality is that it usually results in post-production testing - ie someone inspecting the product coming off the end of the line. 

This sort of "Black-box testing" is not a good way to add quality particularly for software (see Unit Tests - White Box Testing).

What I call "Production Line Mentality" is where everybody has a specific well-defined job specification. This is common in an organization with a top-down management style where a worker's purpose is to impress their boss rather than work with others. In other words it does not encourage people to work together to accomplish their team's goals.

It is caused by a controlling (rather than empowering) management style where each role is kept simple to be easily understood and hence controlled. However, it results in a large number of specialized positions and a multi-layered management structure to be able supervise everybody in the organization.

This discussion actually ties in nicely with some of my recent posts. First, multi-layered management is a classic example of the Layer Anti-Pattern which is a pattern of bad software design which I have found can be applied in other domains.

Moreover, these layers form an excellent example of the DIRE principle. First, the managers at the top of the hierarchy are typically completely out of touch with the people actually doing the real work in the organization as they are isolated from them by all the layers. Perhaps even worse is that it encourages the workers themselves to concentrate on their own specific task without regard for the goals of the broader organization.

In an Agile development environment the developers are empowered to do whatever they need to do (within their bounds) to finish their tasks by the end of the sprint. For a traditional controlling manager it may be hard to deal with an environment where you can't even understand what everyone is doing let alone control it.

A controlling manager can be a real problem for the Agile approach. On the other hand, having developers who are happy to (or used to) being controlled like this is another major obstacle for the adoption of Agile. Years of manipulation (and even threats) means that many are happy simply to be given work as outlined in their job spec.


Traditional Roles

Let's look at traditional roles and how they change in an Agile world.

Team Leader

Scrum books simply say that the team leader role is no longer necessary. The team self-manages so the need for a manager within the team ceases to exist.

I disagree with this. In fact most of the same books describe a role for the Scrum Master which is almost exactly what I have always thought a team leader should be. (Perhaps they like the different title to emphasize that many team leaders are using a bad style.)

The team leader is now called the Scrum Master. Many team leaders also write code. That's OK too - you can have a part time scrum master who also codes.

However, if you think a team leader/scrum master is there to tell the team members what to do, then forget it. One of the main advantages of Agile is that teams are self-organizing and self-managing (within the bounds set for them by the PO and higher management). Small self-organized teams generally work much more efficiently and harmoniously than "centrally-organized" teams.

Designer

There is no longer any room for analysts/designers who spend weeks or months understanding a problem and trying (unsuccessfully, of course) to come up with the perfect, well-documented design. In an Agile world analysis and design is performed "just in time" (see JIT). The developers design, code (and test) all at once. This is not to say that what is designed is left to the whim of the developers - in fact how the design evolves is more closely controlled through the customer representative (product owner).

Traditional designers can move toward a role with industry knowledge, in particular how the software will help the business (or even the customers of the business). Perhaps they will be the actual PO (product owner), though I prefer someone even closer to the customer than that. Probably they will become a "business analyst" assisting the PO. Depending on the software (and despite the Agile preference of JIT design) there may be a need to mock up designs slightly ahead of time to garner feedback from the PO and/or customer.

Alternatively, or also, they may also do a lot of Acceptance Testing (see below). Whatever they do, their main focus (as for any Scrum team member) should be on completing the sprint backlog before the end of the current sprint.

Architect

There are just two points I want to mention about software architects using quotes from the excellent book 97 Things Every Software Architect Should Know.

" [the architect] should not sit in an ivory tower dictating the way forward but should be on the ground working with the team "

- from 19. Architects must be hands on by John Davies

"As well as being prone to incidental complexity and inertia, designing large systems upfront means larger projects, which are more likely to fail, more likely to be un-testable, more likely to be fragile, more likely to have unneeded and unused parts, more likely to be expensive, and more likely to have a negative political dimension. ...

"Design the smallest system you can, help deliver it, and let it evolve towards the grand vision. Although this might feel like giving up control, or even shirking your responsibilities, ultimately your stakeholders will thank you for it."

- from 97. Great software is not built, it is grown by Bill de hOra.

In brief: architects work with the team to evolve the design, spending most of their time coding and reviewing.

Programmer

In an Agile environment programmers now do everything that is required of the team (analyze, design, code, test, etc). They do whatever it takes to complete the sprint tasks by the end of the sprint.

This is in stark contrast with some traditional environments where programmers have just been coders - they take a design from the analyst/designer and code it for "QA" to test. This is demeaning for programmers who are usually capable of a lot more. Even if you care not for the morale of developers giving the programmers free reign to think about the design and even experiment at little will produce a much better design than any analyst/designer (no matter how clever) can invent in their head.

Of course, this does not mean the programmers can do whatever they want. They are closely guided by the product owner who, after all, prioritizes the backlog effectively deciding on the sprint task(s) that are the focus for the team.

Tester

If waterfall development is disempowering to programmers it is much worse for testers. As compensation, testers are often given the impressive title of quality assurance experts, despite the fact that quality assurance has little to do with testing.

You may not believe this but using Agile development (especially Unit Tests) there is a greatly reduced need for dedicated testers. How great this reduction is depends to a large extent on how much testing the developers do already. At one extreme I recently worked in an environment where there were twice as many testers as developers! The software contractor had there own tester(s) who worked closely with the coders. Then the customer had several testers, plus an "offshore" test team to perform regression testing, as well as an automated testing team.

There are many different names for the types of testing (and testers) that are used in the software industry. In Agile there is just two general types of testing:

1. Unit Tests: automated tests written by the developers of the code to test individual (low and high level) modules
2. Acceptance Tests: performed by the PO, customers, or anyone who wants to try the software

Note that Unit Tests are always automated, but acceptance tests may be automated too. Often the developers will add a simple scripting system to make it easy for the PO and others to create test scripts.

This is useful for testing the current changes. But don't go overboard and add lots of tests for existing functionality as they will most likely fail when changes are made (and even act as a disincentive to change). Leave the regression testing to the Unit Tests.

To understand this better consider the testing that an aircraft manufacturer does. First, individual parts are developed (or purchased). The developers of these parts (for example, spark plugs) will design, build and test them themselves and provide a hopefully reliable product. Some parts may be made from a combination of other lower-level parts, but it is still up to the developer of any part to thoroughly test it. Of course, larger, more complicated, parts may require more elaborate testing - testing of an aircraft engine may require a huge test rig. In the end it is always up to the maker of a part to ensure it performs to the requirements of the plane manufacturer. Testing of parts is the equivalent in software development of Unit Testing.

When the first plane (or a prototype) is finally built it is then up to a test pilot to perform various flights and manoeuvres to ensure that the plane performs as expected. This is the equivalent in software development of Acceptance Testing. This analogy is not perfect though, since Acceptance Testing of software should usually be performed much earlier than test flights.

If a Scrum team requires a dedicated tester at all then perhaps one or two members (at most) could be required for Acceptance Testing on behalf of the PO. They should spend much of their time understanding what the software is to do by discussing it with PO, business analyst, customer and any other stakeholders - something that an analyst/designer traditionally does. They might also attempt to automate much of their work using acceptance test scripts.

Whatever, they do their main focus should be producing working software at the end of the sprint. This means giving the users what they need as opposed to testing for conformance to any written specification.


Conclusion

If you are about to, or have been trying to, move toward an Agile development environment then you should ignore the advice given by many Agile coaches and experts that most roles will not change. There is no longer any need for software designers and less need for testers. Instead the whole team is involved in analysis, design, coding and testing - and this is all done "just in time" (see JIT). (Of course, if they are willing to adapt a role can be found for anyone -- it may be more challenging but will probably involves better use of their abilities.)

Most importantly, managers have less control of the team, which is now self-organizing. The major benefit is simply that a creative team can organize themselves much better than any manager. It also makes for a more rewarding and enjoyable working environment. The developers concentrate on creating good software and overcoming hurdles rather than making themselves look good for their annual performance review or trying to shift the blame when something goes wrong.

It may be hard to overcome the unthinking production line mentality (I do what the boss tells me to do) or vendor mentality (I do what the customer tells me to do). But please try!