Let's take it one step at a time, being agile does little or nothing to change the need to document requirements. It doesn't matter the name or the structure we use. We call them epics, user stories or requirements. We use them within the context of operations, projects or innovation. Documenting these structures, our needs, or how we are to exploit the opportunities we identify is fundamental to any initiative. And I don't mean by this the sole purpose of communicating to others what we think, I also include the process of documenting for the future. Documenting is a process of contemplating writing, and in doing so we often unintentionally structure our thoughts better.
Some history to start with. Software development is today, and has always been, a great challenge of communication between people. Since the days of punched cards, programmers, those capable of creating programs for computers, and those with ideas, those incapable of creating the programs, but with good ideas for taking advantage of the computing power of machines, have had trouble "understanding" each other.
Requirements
Many tools have been created to solve this communication problem, and the basic concept behind it is known as a software "requirement".
In systems development engineering, a requirement is a documented requirement for the content, form or functionality of a product or service. It is used in a formal sense in the systems engineering, software engineering e requirements engineering.
Wikipedia (April 2020) - https://es.wikipedia.org/wiki/Requisito_(sistemas)
Why is it so difficult to write requirements?
There are many theories about the complexity of requirements. However, to illustrate my point I want to use a concept that I have always found fabulous: conceptual integrity from Fred Brooks. This refers to the ability of one person or a very small group of people to contain the entire design of a system, product or model.
In one of his books - the design of the design - raises conceptual integrity with an example: think for a moment about who invented the electric light bulb, the telephone, the telegraph or the airplane. In all these examples it is easy to identify the inventor. However, when the complexity of the system under design increases substantially, it is almost impossible for a single person to keep in his or her head the detail of all the components. At that point, maintaining conceptual integrity is a real challenge.
If you're still not clear on the problem, think of a book you've read and discussed with someone else. What's the likelihood that your mental picture of each other's characters and motivations is exactly the same as the other person's? NONE! And we're talking about books that are hundreds of pages long.
A student once said to me in class when we were looking at this concept: "the problem [of conceptual integrity] is that other people are given to thinking". While I understand your personal motivation and the frustration that can generate that different interpretations are in no way aligned, we can not defend the idea of "stop thinking". What we seek is precisely the opposite, we want each individual member of a team to think.
Collaboration or blame game
Another aspect to consider when determining which tools to use to facilitate communication between "those who have the ideas" and "those who can realize them" is the extent of such collaboration. In other words, how much collaboration and trust there is between team members and within an organization.
Trust requires psychological safety within team members and within an organization or company. Psychological safety is a term coined by Amy C. Edmondson, a Harvard Business School professor. Edmondson, professor at Harvard Business School that impacts the way we communicate and - in particular - we do so during creative processes or during the construction of new solutions.
In an environment with low psychological safety, members of a team or work group do not trust each other. When a threat arises, whether it is a problem, a mistake, or a misinterpretation, individuals seek to "offload responsibility" onto others. If this situation happens regularly, the requirements become the "contract" that regulates the relationship. The result ends up expanding in details and explanations the documents that moderate the conversation between the parties. Like trying to "document" whose fault it is if something goes wrong.
Use cases
They are the step-by-step documentation of a user's interaction with an information system. The description of this step by step, click by click and moment by moment, gave way to Use Cases. A UC - or Use Case in English - contains detailed information of the interaction flows that a type of user (Actor) has, which, almost always, is presented as a conversation:
- User performs action X
- The system under design (SuD) performs the AND action.
- The user performs the action Z
- …
I think they got the point. To this conversation between the user and the system, they included:
- Exception paths - what happens if something goes differently in one of the steps,
- Test cases - discussions on how to validate each of the steps,
- Descriptions,
- Images
One of the best books for documenting good Use Cases came to me from the hand of a great entrepreneur - Alex Torrenegra - when I was still trying to code for a living: Writing Effective Use Cases by Alistair Cockburn.
Pros
- UC is very valuable when the systems we design (SuD) are extremely complex.
- When the team assigned to development is unfamiliar with how SuD works. For example, when we have new team members, new teams or we incorporate new suppliers to our development structure.
- They provide a clear level of detail about what is expected in the screens, sequences and processes of a complex system.
- They leave little room for creativity and divergent thinking.
Cons
- UCs leave no room for proposing improvements, being creative or generally having divergent thinking.
- They do not allow open discussion of new usability, functional or performance ideas.
- UC has a high dependence on Conceptual Integrity.
Innovative environments for user stories
In environments of greater uncertainty, where there is no clarity in the requirements, having a detailed level of detail can be difficult or simply impractical. To give an example, when working on the discovery or design of new products, defining requirements in such detail can be a headache or very limiting for the team.
User stories
To encourage the participation of all members and promote divergent thinking, new models of documentation emerged, much lighter and oriented to promote discussions that add value to the solution. eXtreme Programming (XP), an agile software development process that was born in the 90s, proposed the use of User Stories (US).
User stories have the same purpose as use cases, but they are not the same. They are used to create time estimates for the release planning meeting. They are also used in place of a large requirements document. User stories are written by customers as things the system should do for them. They are similar to usage scenarios, except that they are not limited to describing a user interface. They are in the format of approximately three sentences of text written by the client in the client's terminology without technicalities.
eXtreme Programming (April 2020) - http://www.extremeprogramming.org/rules/userstories.html
Unlike the Use Cases, the US promotes several things:
- Focus on the objective (the opportunity or need to be solved) rather than the how.
- Keeping focus on the target promotes innovative solutions
- Simplifies that users, customers and stakeholders don't need to know the system to contribute ideas or request new features
- Conversation, discussion and debate about how to solve or complete a US is welcome - which in UC can happen, but almost always seems more like an "explanation of the flow" than a true discussion of equals.
Pros
- The US promotes divergent thinking and creative solutions.
- They allow different - and certainly more efficient - proposals to the problems or needs posed.
- Increase team productivity - mature and well-coordinated teams
- Promote customer - and other key stakeholder - engagement by being simple to write (which reduces barriers to entry to participation and input)
Cons
- Unaligned teams can violate designs or application architectures designed on a large scale - in large organizations.
- Forcing the use of US can be a real headache - because of the very lack of detail. For example, I once saw a team trying to use US to describe BI and Analytics operations.
- Many believe that US cannot coexist in the same Backlog - prioritized list of requirements or work to be done - with other instruments such as tasks, use cases or technical requirements (NFR).
If you want to learn more about how to write good user stories I recommend reading Mike Cohn's book "angular" User Stories: User Stories Applied. I make a well-deserved mention for two Colombians (Lucho Salazar y Jorge Abad) who did an excellent job with their publication User Stories: A Pragmatic View.
Agile Modeling
Agile modeling refers to the use of other instruments and tools to document requirements - mostly software. They can be understood as a sort of "Extension Pack" to frameworks like XP or Scrum. In my experience, the most useful complementary tools of this time are:
- Agile People
- Wireframes and Mockups
- Story maps
- Empathy Maps
Scale and drill down, beyond user stories
However, using a single instrument within the Backlog - or describing them all on the same scale - is inefficient when in the same instrument we have different time horizons. That is, the Backlog includes requirements that are so clear and important that they must be executed and completed in the next few weeks; and it can also include elements that are just vague ideas.
These "annotations" to the Backlog seem more like a "reminder" for future discussion - in my very personal opinion, and perhaps influenced by computer management tools, one should always write down all ideas and concepts with potential to become reality in the Backlog and not rely on human memory.
Consequently, the magnitude of effort and complexity of an element within the Backlog can consider many requirements - in the form of stories or use cases, or simple tasks - here I separate myself from the radical purists. In their eagerness to name different scales of detail and magnitude, and somehow make it clear that such elements require further breakdown, concepts such as Epics, Features and Capabilities were created. Epics, Features y Capabilities.
Practical advice: Use the names Epic, Trait and Ability as you see fit, but don't interchange them so as not to create confusion. You can use one, you can use several, you can use these and more. Don't get complicated, just give each term a special context and use.
Epic
An excellent idea that the team of Scaled Agile Framework was to define specific purposes for each instrument - and also objectives within the planning and execution time horizons. Without being the only way, it seems to me the best way (so far) to differentiate each scale.
In this frame of reference, the epic are instruments to define portfolio investments - for marketing issues and distancing from the ever lagging PMI they decided not to use the word "project". Epics then are components of a portfolio or program that include validation of technical or business hypotheses - similar to an Agile Business Case. They insist that epics and projects are not the same, but, just as I have my personal differences with the "PMI philosophy" I also have them with the "SAFe philosophy".
Within an Epic we can contemplate many things, some oriented to reduce risks, others to enable our organizational or technical capabilities, and others 100% oriented to the fulfillment of strategic objectives.
Features - Features
However, an epic defined in this context seems far removed from the work that agile teams do. That's why we need intermediate instruments that connect the strategic scale with the operation and realization of that business value. This is where, much to my dismay, they used terms that are very closely linked to product development: the features.
Features two things, a high-level requirement, somewhere in the middle between Epics and Stories, and therefore also represent a set of Backlog items that is a close instrument to the team - although in SAFe they define other high-level Backlogs. What's interesting is the timing relationship they try to take care of between stories and features.
Capabilities - Capabilities
In SAFe, there is an additional aggregation tool known as capacitywhose most notable difference from the features is that it is completed thanks to the work of different ARTs - a term for a large-scale agile team focused on a product, service or Value Stream.
Planning instruments
Recall that backlog elements are instruments for identifying value and maintaining alignment between purpose and delivery - a concept that is gaining momentum in the world that promotes the "flow of business value". Likewise, these elements are very useful for establishing time horizons and, like it or not, business milestones.
Having elements that represent different scales of effort is very useful for establishing delivery cadence. To cite an example, we can look at the proposal from Scaled Agile Framework:
- Two-week iterations (longer ones seem unnecessary to me, I like one-week ones better) should deliver several user stories.
- Cycles of 5 or 6 iterations or Sprints to deliver various Features and Capabilities (Program Incrementalism in SAFe)
- One year could deliver, solve or service several Epics (Lean Portfolio)
Conclusions and considerations
The challenge is the same, to maintain conceptual integrity among the stakeholders - the wonderful, diverse and dispersed human beings - and to facilitate and promote the development of "best possible solutions" - technical, economical, functional and user-friendly. There are tools that limit creativity and divergent thinking - very useful for solving complex problems where teams do not have the expertise and/or knowledge, or where it is simply impractical to propose different solutions or promote interpretation - such as advanced calculations with data, reports, and tightly regulated business processes. There are tools that promote divergent thinking and innovative solutions, that contribute to the creation of new and better products, and that propose mental alignment mechanisms - such as Agile Personas - to strengthen conceptual integrity.
These tools, such as user stories, are also useful according to the maturity stages of the team and the members' knowledge of the system itself. Imagine a group of "newcomers" with no previous experience trying to design a system within an organization with strong design constraints or technical limitations - such as legacy systems - at the end of user stories, it would be frustrating!
Conversely, imagine an experienced group of close colleagues trying to solve a challenge by following in the footsteps of someone who is not with them - boring! - We must always balance skills with challenge and these tools can be useful at every stage of every cycle.
Other tools exist only to allow us to manage different scales of effort and/or time, such as epics and characteristics. They are useful for managing multiple teams at a high level and communicating intentions, purpose and value objectives.
As a final thought I leave you with this short video on the concept of Conceptual Integrity. Nothing as deep and radical as Amy's video.
Surely in the future we will see new ways of communicating and documenting our needs, and I hope to be there to use them and make the most of them.
