It's hard to believe that the agile methodology officially turned 20 years old last year. What once was an outlying practice for startups collaborating in colocated spaces with stickies and whiteboards is now a sophisticated, scalable, and widely used set of agile software development processes and tools.
There’s a rich history behind agile development and why organizations use agile methods such as scrum and kanban to modernize applications, improve customer experience, and implement digital transformations. There’s also a tremendous body of knowledge around these methodologies and their intersections with design thinking, product management, and devops. Fewer people today ask, “What is agile?” More are seeking guidance for how to align their teams on agile best practices.
This article is a primer on agile methodologies, starting with people, teams, processes, and tools. You'll also learn how agile connects to devops, and about best practices that help organizations cultivate an agile culture and deliver better software.
Roles in agile methodology
An agile software development process always starts by defining the users for a particular product and documenting a vision statement for the scope of problems, opportunities, and values to be addressed. The product owner captures this vision and works with a multidisciplinary team (or teams) to deliver on it. Several roles are involved in an agile development process.
Users
An agile process always begins with the user or customer in mind. Today, we often define user personas to illustrate different workflow roles or types of customer needs and behaviors.
Product owner
The product owner is tasked to be the voice of the customer, including any internal stakeholders. This person distills insights, ideas, and feedback to create a product vision. Product visions are often short and straightforward, but they nonetheless paint a picture of who the customer or user is, what values are being addressed, and a strategy for addressing them. I imagine Google’s original vision looked something like, “Let’s make it easy for anyone with internet access to find relevant websites and web pages with a simple, keyword-driven interface and an algorithm that ranks reputable sources higher in the search results.”
Whatever the vision, the product owner is responsible for defining it and then working with the development team to make it real.
To work with the development team, the product owner breaks down the product vision into a series of user stories. Each user story should identify the target users, their challenges, why the solution is needed, and what constraints and acceptance criteria define the solution. The product owner prioritizes these user stories and reviews them with the team to ensure they have a shared understanding of what is being asked of them.
Software development team
Teams should be multidisciplinary and include a diverse group with the skills and backgrounds to get the job done. In addition to developers, agile development teams should include quality assurance automation engineers, data engineers, user experience (UX) designers, and other roles depending on the type of software project.
Agile focuses teams on delivering working software, so they must complete end-to-end functioning applications, integrations, and other deliverables that impact users—not just the technical components. Team members must align on what they are building, who is doing what, and how the software will be developed.
Agile teams often have other roles assigned, including the following:
- Tech or team leads partner with the product owner on architecture, non-functional acceptance criteria, sequencing, dependencies, and other technology and security considerations. Tech leads have broad responsibilities that might include estimating stories and planning implementation details with the team.
- Scrum masters often coach new teams on agile processes, responsibilities, and tools. Scrum master responsibilities can include resolving blocks that impede progress, reviewing approaches to improve the agile team’s velocity, and grooming backlogs.
- Business analysts partner with the product owner. The analysts’ responsibilities often include creating wireframes, documenting user stories, and reviewing test results. Business analysts are especially helpful when software development teams are developing microservices and other technical products, and where the business analyst has more software development knowledge than the product owner.
It is up to organizational leaders to decide how to staff agile teams and how big to make them. Many follow Jeff Bezos’s best practice of constructing two pizza-size agile teams to maximize the collaboration between teammates.
Scrum and kanban
Once a product vision and team (or teams) adopt agile principles, starting with the ones identified in the agile manifesto, the organization must select a process methodology. Scrum and kanban are the primary agile processes.
Some organizations start with kanban because it’s relatively easy to explain and implement. Kanban works as a fan-in and fan-out process where the team pulls user stories from an intake board and funnels them through a workflow until they are marked done.
But many organizations implement scrum, which organizes the work in cadences called sprints, usually lasting one or two weeks. The product owner writes the requirements as user stories, then prioritizes them in a backlog based on their business value. The team reviews the backlog and commits to the top user stories they can complete during the sprint.
Scrum includes several standard meetings (sometimes called scrum ceremonies or scrum rituals) to help teams commit to sprint priorities, complete the work during the sprint, and end each sprint successfully. These meetings usually include a few common elements:
- Sprint planning is where the product owner shares priorities, and the team decides how much work it can complete during the sprint.
- Daily standup meetings help teams discuss the status of user stories; teammates share their daily goals, and anyone can escalate blocks that impede the team’s progress.
- Sprint reviews are demo meetings at the end of the sprint, where the functionality is shown to the product owner to gain acceptance on completed work.
- Retrospective meetings are where the team discusses what went well and what needs improvement in their agile and software development processes.
It should be noted that these practices are adaptable to agile hybrid work models.
Scrum improves a team’s performance by empowering the team to commit to an achievable amount of work rather than having a product, program, or project manager specify the expected timeline and scope. The user story forms a microcontract that separates business need, the acceptance criteria (or what agile teams sometimes call the definition of done), and then enables teams to self-organize on how to implement. Sprint reviews are one type of feedback loop, and product owners are encouraged to realign priorities and redefine requirements before each sprint. Sprint retrospectives help the team improve collaboration and initiate process improvements.
Technical best practices for agile organizations
Scrum forms the basic process for teams collaborating, planning, and delivering, but it doesn’t speak to technical best practices, organizational standards, or defining and driving agile cultures.
Today, many technical best practices include defining the software development lifecycle (SDLC) and implementing devops processes. The SDLC provides guidelines on writing code, managing software assets, and developing technical standards. Devops automations like CI/CD, Infrastructure as Code (IaC), and continuous testing enable a more reliable path to production. Other practices, including shift-left security practices, observable microservices, feature flagging, canary releases, and AIOps, provide a more flexible and reliable delivery model.
The combination of empowering self-organizing teams, agile methodologies, devops automations, and modernizing to cloud architectures helps technology organizations evolve their culture. Longer development cycles are replaced by continuous delivery models that enable releasing features and improvements faster. Automations address many gaps between developers seeking autonomy and speed with operations responsibilities around performance, reliability, and security. Combining these practices helps agile teams make smarter architecture decisions, drive experimentation, become more data driven, and correct mistakes quickly.
Other practices like integrating design thinking with scrum, implementing value streams, developing product management practices, and implementing continuous planning help agile teams collaborate with customers, end-users, and business stakeholders.
Agile teams typically deploy tools like Jira Software, Azure DevOps, and Digital.ai to collaborate on agile backlogs and kanban boards. These tools help agile teams prioritize work, capture requirements, complete user stories, review burndown reports, and automate workflows using version control, CI/CD, and other tools.
Conceptual frameworks and guides such as SAFe, Enterprise Scrum, LeSS (Large-Scale Scrum), the Spotify Model, and StarCIO Agile can help drive agile principles, standards, and practices across many collaborating teams.
Most coaches recommend starting agile practices with well-defined business objectives, a few selected teams, and limited, optimally chosen tools. The challenge for organization leaders is to find the right balance of diverse teams, self-organization principles, standards, tools, and integrations that enable their organizations to build, extend, scale, and maintain technology capabilities.