Scrum takes its name from a play in rugby where the players bind together in teams and try to take back possession of the ball. Injuries are common.
‘Scrum’ as in Software Development
The description above is an apt metaphor for Scrum development. It is a lightweight team-based Agile framework that is focused on getting the ball down the field in the fastest, most efficient way possible. Scrum is used for developing complex products and services with a focus on developing incremental units of business value within short iterations. It requires honesty, communication & adaptability. Just like the Rugby scrum it can be painful, but incredibly effective.
Now, let’s breakdown the components of Scrum. It is comprised of roles, ceremonies, and artifacts.
The Product Owner (PO) is the individual who makes the decisions on anything related to the “product”. The PO’s key responsibility is to make sure the team is developing products and features that deliver business value. The PO represents the voice of the customer and other business stakeholders. They drive the vision of the product and its features; create and prioritize the product backlog; define acceptance criteria and ultimately accepts the output from the development team. The PO is essentially an implant from the business world into the scrum team, as they are considered the product champion with the complete product know-how.
The Scrum Master (SM) is the servant leader for the development team. The SM makes sure that the team is on track to deliver the product in increments and within the time-boxed sprint schedule (more on sprints in a bit). They facilitate through devotion to the scrum process, and by swiftly eliminating scrum team’s impediments. They uphold the structure of scrum, and inculcates the principles of agile within the scrum team. The SM is a facilitator for the scrum team, and makes sure that the development team is aware of what needs to be accomplished and when, and that they have the ability to self-organize to accomplish the development tasks.
Team Members / Development Team
The Team Members / Development Team (DT) are those responsible for the implementation of the product or features. These are the set of people who are actually doing the what that has been proposed and conceived through the product backlog and grooming sessions. The DT is usually a cross-disciplinary group of individuals including engineers, UI developers, Analysts and Quality Assurance. The DT self organizes and decides the best way to accomplish tasks they pick up during planning. They provide estimates on stories within the product backlog as well as provide feedback on the acceptance criteria to the PO to help keep development on the right track.
The Product Backlog is a prioritized list of epics and stories created and managed by the Product Owner (PO). An epic is a large story and there may be smaller stories contained within. The story describes the feature in a way that makes the business value and use for the customer clear to all stakeholders so everyone can understand what is being delivered. A story is an increment of work that can be delivered by the team within the sprint cycle. Priority is set by the Product Owner, and the PO dictates the order in which stories are delivered by the team.
The Sprint Backlog are those stories that have been committed to by the team for the upcoming or current sprint. In order to qualify to be a sprint backlog candidate stories must have enough detail (meeting the Definition of Ready – DoR) that the team is comfortable estimating the size of the story (story points) and stakeholder input/approvals have been included. The number of stories in the Sprint Backlog are determined by teams capacity built over sprint over sprint, what is also known as their velocity.
Burn Down Chart
The Burn Down Chart is the means by which the Scrum Master, team and other stakeholders keep track of progress throughout the sprint. The ideal chart would have the team incrementally completing stories throughout the sprint and thus “burning down” the Sprint Backlog to land at 0 story points left at the end of the sprint.
Sample Burn Down Chart above – Blue line represents ideal progress, red line represents actual progress
Product Backlog Refinement (PBR)
Backlog Grooming is the process by which the Product Owner gets their stories Sprint Ready. It involves setting the vision, proving the business case, meeting with customers and other stakeholders, conferring with the team, working with UX & visual designers. Care should be taken to get enough detail into stories that the team is clear on what needs to be delivered and why, but not so much detail that analysis paralysis occurs. Grooming sessions with the team should be time-boxed to make sure their time is spent optimally and mostly devoted to getting actual work done delivering items for the current sprint. A typical grooming session lasts for about 2-4 hrs depending on the length of the sprint, and the development team has to make sure that each story meets their DoR as an input to the Sprint, and the acceptance criteria and assumptions meet Definition of Done (DoD) for the final approval of the story.
The Sprint is time-boxed cycle that is 1 – 4 weeks in duration. At the end of the Sprint an increment of the product should be completed and ideally delivered to the customer. Sprint Planning is the session run by the Scrum Master where the team decides how many stories they can take into the upcoming sprint. The team selects stories in priority order set by the Product Owner. A team that has been working together for several sprints or more will have a historic record of how many story points they can take into a sprint. This is referred to as their velocity. Ideally their velocity will increase over time as the team builds. The objective of the sprint planning is to commit stories for the upcoming sprint to the PO, and identify all activities required for the development of the story to meet the DoD.
Daily Standup (Scrum)
The Daily Standup or Scrum is facilitated by the Scrum Master on a daily basis at the same time everyday without fail for the development team. It is time-boxed to only 15 minutes. Each member of the development team comes on time, and answers three questions – what did they work on yesterday, what are they focused on working today and if there are any impediments in their way. Impediments are noted by the Scrum Master and taken to be discussed offline, and are addressed following the standup.
The Retrospective is a crucial ceremony in Scrum that invites Development Team Members to give an honest assessment of their team’s performance in recent sprint. It is facilitated by the Scrum Master, and may or may not include the Product Owner. Some teams may need the Retrospective as a place to discuss issues with the Product Owner and in that case may need them to be excluded to allow for open communication. Team Members each provide a list of the following categories: “Start Doing”; “Stop Doing”; & “Continue Doing”. These may be framed in different ways but basically the conversation should be open and constructive in identifying room for improvement; issues; as well as celebrating successes.
These are the basic components of Scrum. The best way to learn is to practice and each team will grow and modify the techniques to meet their unique group dynamics & development environment. Other good sources are –
The roles and responsibilities of the Scrum Master may vary based on the distribution environment and team structure, but there is always a component that seems to be common for all cases, and this is ensuring that the team is following Agile practices. It becomes imperative in the distributed environment since most of those practices were initially designed for the collocated teams. As a Scrum Master, she/he is responsible for coaching the team and helping them overcome those challenges.
Distributed teams can adopt not all Agile practices; some have to be significantly modified, and some will require specific tools, which means that the team will have to invest in some learning time to adopt them. One of the classic examples of those modifications is pair programming. In distributed Agile environments, pair programming is replaced with code reviews. (Personally, I have found code reviews more efficient that pair programming even within the collocated teams).
Another practice that is critical in a distributed environment is continuous integration which will ensure that everybody is working on the same code. The implementation of this practice can be challenging from the technical point of view but is well worth the investment. It also requires that all team members understand the importance of daily code check in, even though the particular feature they are working on may not be finished. If the code is throwing exceptions or prohibiting any previous functionality from testing, it should be commented out, but still checked in. The Scrum master is responsible for communicating the importance of Agile practices to all team members.
One of the other Scrum master responsibilities usually includes tracking iteration progress. In collocated environments. Iteration tracking is visualized by sticky notes on a wall so that every team member can see the current status of the particular issue, and update the status on items assigned to him during the daily standup meeting. In distributed environment, you need to use something more advanced to visualize the progress. There are fairly large numbers of tools available today in the market which do an excellent job of visualizing iteration tasks, keeping track of backlog items, and generating burn down charts.
The best way to promote Agile to senior management is to explain its numerous benefits and cost and waste reduction methods. Once key players in the organization are made aware of Agile’s benefits, it essentially sells itself. And, the best way to sell a methodology is to demonstrate its value by delivering quantifiable and visible business benefits, but to even get there, you first need to find a project that you can implement using Agile, and this is a challenge in itself.
The process of selling usually starts with a presentation to the key decision makers, which should at least cover the following areas:
Changes required in this particular department (team, tools, meetings)
Overview of Agile process (two to three slides)
Overview of user stories and how they relate to the requirements
Overview of tools required
Overview of general Agile benefits with a focus on how this particular company/department/project will benefit from Agile (reduced documentation overhead, better progress tracking, improved code quality, faster delivery, more efficient analysis of scope changes, customer satisfaction, short feedback loops, etc.)
Another very important step at this stage is to determine general expectations for Agile from senior management and to make sure they understand that Agile is not a magic solution. Some non-biased measures, as well as the success criteria for implementing Agile within a specific project, have to be defined.
Let’s return to the benefits outlined previously and see how they can be measured since some of them are very tricky (for example, it’s not always easy to measure customer satisfaction or the ability to react to changes in the requirements) here are a few general recommendations:
Reduce documentation overhead
Better progress tracking
Once you’ve obtained a general approval to test Agile you need to find a project that will demonstrate all the good things you promised in your presentation. Finding the right candidate is extremely critical and not an easy job to any extent.
One of the biggest misconceptions about Agile is that architecture is not required in the Agile development. ‘We‘re Agile; we don’t need architecture’–is something that everybody involved in Agile has heard at least once.
Let’s start by establishing a common understanding of what the software architecture is, to which everyone can agree. The definition of architecture is quite broad, and the roles and responsibilities of software architects vary dramatically from company to company.
Here is how Martin Fowler identifies architecture in Patterns of Enterprise Application Architecture:
“Architecture” is a term that lots of people try to define, with little agreement. There are two common elements: one is the highest-level breakdown of a system into its parts; the other–decisions that are hard to change.”
I find that we all can agree on those two common elements. Do we need the highest-level system break down? Absolutely. Do we need huge documents and long design stages? No. Agile is not against the architecture–it’s against useless, bulky documentation that nobody reads anyway.
From the perspective of change, the role of architecture in Agile development becomes quite clear – A good architecture is responsive and will support agility; a poor architecture will resist it and reduce it. And, since one of the benefits of adopting Agile is a better response to changes in the requirements, it’s obvious that flexible and extendable architecture is a key to this.
The biggest issue that I’ve noticed is the very thin line between architectural design and software design. I’ve seen companies where the different implementations of the following practice were used: Architects created design documentation and developers were responsible for writing the code. This introduced a myriad of problems, starting with developers feeling that they were not fully trusted. This also gave developers an excuse not to really think about the design. ‘We’re just coders, not responsible for the design and we do only what we are told to do.’ is a common attitude that I have witnessed. In Agile, the developer is responsible for the code he writes (and unit tests) as well as the design since nobody else will provide him with it.
Ideally, the high-level software architecture is completed before coding starts. And I really have to be careful here – completed doesn’t mean written in stone; it can change, but with an understanding “this is the best of what we know right now.” This doesn’t necessarily include a database design or class diagram, and the level of details really depends on the approach you will be taking moving forward. I found that for certain systems the Domain Drive Design (DDD) is extremely useful and has made my life far easier. Therefore, I like to have a domain model and a basic set of domain classes and their relations defined, but not to the level of methods and attributes.
Personally, I prefer projects to have a design stage; this is when the high-level business domain model and user stories are created. At this stage the main architectural decisions are made – the technology that will be used, the database server, the application type (for example, Mobile, rich client, or service), the architecture style (client server, layered architecture, SOA) is selected, and the architectural frame that will be applied is selected as well. The document created during this phase is not solely architectural effort, it is a collaborative effort of business analysts, developers, and network administrators. The output of this design stage is not only a high-level architectural document. (This is not an attempt to make fixed predictions of the future or create a detailed software design upfront as this approach places all the significant decisions at the point of least knowledge in a project’s lifecycle). This is simply a way of getting and sharing the common understanding of the system we are all about to develop.
This is an excerpt from the forthcoming book, The Art of Being Agile.
Feedback occurs when the return of information concerning the results of a process or activity takes place (http://www.thefreedictionary.com.). This event is part of a chain of cause-and-effect that forms a loop onto itself. Feedback comes in two forms: good feedback and bad feedback. Without feedback, the Agile process of inspection and adaption could not occur. An Agile process thrives in an environment with constant change. Because of this variability, adaptation and adjustment points are made on a regular basis. If we can shorten the amount of time elapsed between these calibration points, then we can more quickly adapt to these changing realities. In short, this is the feedback loop in our process and environment.
Types of Feedback
There are, however, different forms of feedback, which are listed here for the reader’s reference and throughout:
Communication feedback (e.g., onsite customer, open team workspace, ubiquitous language)
Technology feedback (tools we use to give us quick feedback, like Cruise Control, mocking, BigVisibleCruise, CCtray, Resharper, TFS real time compilation/warnings, and confirming that we use the right technology)
Requirement feedback (when a customer need realizes in a demo or the production environment)
Market feedback (to see how the market reacts to a new story/feature/module, frequent and numerous deployments)
Defect and testing feedback (the quicker we find out about bugs, the quicker we can fix them, deploying often and always, test driven development)
Operational feedback (process, methodologies, practices and how to improve them)
The quicker we can get these forms of feedback from the source, the faster we can validate our progress and adapt to the information received.
A company’s ability to deal with change and adapt accordingly to changing conditions will improve its competitiveness in the marketplace. Companies that struggle with a slow feedback loop will find themselves caught up in trying to solve problems that have already changed or are not important anymore.
Effects of a fast feedback loop
Having a fast feedback loop allows dominance of a company during market changes. For example, one of our clients using our health and safety management system had a deadline for submission of reports. Approximately 800 companies assigned to each performed various tasks and submitted reports showing the work completed. There were some features and latency problems that the client wanted to be fixed, and the deadline was one week away. We were able to get quickly the high-priority features added, and latency issues resolved three days into the week using our engineering practices, automated testing, and automated deployments. We went live before the deadline to get much-needed feedback on our changes in a real production environment. If we had waited until after the deadline, we wouldn’t have obtained the actual feedback from the end users re the added features, nor the feedback from an environment production perspective, since the client wouldn’t be using the system until the next deadline, which was months away. Using this feedback from deployed features in a production environment allowed us to make more improvements so that the next period time would go even more smoothly.
Being able to perform a full cycle of development from client request to production deployment in a few days helps ensure the company can quickly adapt to changing market conditions.
(This is an excerpt from the mini book series “Agile from the Trenches: The Feedback Loop”)
Product backlog refinement, or PBR, is an integral component of successful Scrums. This process of continuously reviewing product backlog items, to ensure that teams know exactly what to work on in the sprints, cannot be done without. It keeps the teams and the product owner on track. To understand why PBR is critical, it is necessary to first understand the details of what PBR is, what we expect from it, and the strategic value it provides.
What is PBR?
Product backlog refinement is the process in which the product owner, ScrumMaster, and the development teams review the product backlog items and define the stories that they will need to work on during the immediate sprint. Epics or unclear stories are split apart into smaller stories and are detailed in this process, while unnecessary backlog items are removed. In so doing, the backlog items are analyzed in terms of how much time and work each one will require, and the requirements for each item are clarified. After the PBR session is complete, each story should be valuable and testable.
PBR is conducted in each sprint planning meeting to address the tasks for the immediate sprint. Thus, for each project, PBR is conducted at least as many times as the number of sprints. New Scrum teams, or those with sprints of more than two weeks, may find it useful to conduct more PBR meetings than the number of sprints in their project; additional meetings would be conducted outside of the sprint planning meetings, and these could compose up to ten percent of the teams’ time. The frequency with which PBR should be conducted is due to the volatile nature of Scrum product management. The completion of each sprint reveals more details regarding the product, which results in the need to alter stories — by adding or deleting certain aspects of the stories — and update.
Expectations from PBR
The process of PBR is conducted with the intention of thoroughly prepping development team members about the sprint’s tasks, such that the teams know precisely what to work on and achieve by the end of the sprint. It also is conducted to assist the PO in getting the top-priority backlog items ready for the sprint planning meeting. Essentially, product backlog refinement occurs with the purpose of clarifying each sprint’s tasks and ensuring that they are in sizable chunks that can be accomplished.
Each PBR session is intended to provide team members and the PO with opportunities to update their sprint tasks accordingly, while additional PBR sessions outside of sprint planning meetings enable them to work on detailing and refining a larger number of backlog items.
Vision and strategic value of PBR
Product backlog refinement assists in ensuring progress toward the project’s objectives. The process succeeds in keeping the PO and development teams on track toward the project’s main objectives, as it is a way for the PO, ScrumMaster, and development teams to maintain a clear vision of the sprint tasks, especially in light of product feedback, the emergence of new ideas, and changes in project needs — aspects that can occur throughout each sprint. Scrum product management involves constant shaping of the product, which necessitates redefining the backlog tasks in each sprint. PBR optimizes such redefining of backlog tasks, thereby preventing the PO and teams from losing sight of the work to be completed in each sprint. In so doing, PBR keeps the PO and teams on track during the project.
Not only does PBR keep the PO and development teams on track but it also provides strategic value in that conducting PBR prevents a number of issues from arising later on in the sprint. A main issue that PBR can help the PO and teams to avoid is delivery of stories that do not meet the recipient’s full needs. Without PBR, teams work on stories whose details are not clarified, leading the teams to complete the stories unaware of certain requirements. This results in a loss of time and effort. Another related issue that PBR can make avoidable is a volatile team velocity that results from running ambiguous sprints. If a PO and the development teams choose to forgo PBR, they also risk completing sprints where the stories they worked on were not the most valuable stories. Had a PBR meeting been conducted, the more valuable stories could have been noted and addressed accordingly.
The value and benefits associated with product backlog refinement are large, as are the consequences of forgoing the process. On top of the value it holds, PBR can be easily integrated into each sprint. Therefore, there is little reason to pass over this process that is critical to Scrum success.