Work Item Estimation Guidelines

This document outlines:

1. Why and how we estimate work items
2. How to update estimates throughout the project lifecycle
3. How to run effective estimation meetings
4. When to use alternative estimation approaches

1. At the Sales/Discovery phase, the titles of work items are captured along with some detail around what each feature involves.
2. Items are given initial sales estimates - using the Initial Estimate Low and Initial Estimate High fields on the Product Backlog Items within Azure DevOps.
   1. Note that as part of the sales process, single numbers may have been agreed - in which case both these numbers will be the same or only one field will have been used.
3. Work is planned and sold based on these estimates.
4. When the delivery phase commences, requirements are fleshed out by the Product Owner, supported by the Scrum Master and/or the Business Analyst.
5. Once work items have their acceptance criteria are signed off, the delivery team hold regular estimation sessions which result in us agreeing the actual estimates and capturing these in the Effort field in Azure DevOps.
6. Work is planned and committed to sprints based on these Effort numbers and the available capacity per sprint.

As part of the agile/Scrum process, the scale of work is estimated by the delivery team prior to items being committed to a sprint.

A common misconception is that the only purpose of conducting these estimates is to help us forecast how long it will take to deliver each work item. In fact, the true benefit of estimating work items is two-fold:

1. Reviewing the work items together helps ensure the team have a shared understanding of what must be delivered.
   1. Additional understanding will have been gleaned since the original Sales/Discovery estimates, so this review can be done with additional insights.
   2. Engineers will have opportunity to add value that someone playing a sales consultant role might not.
   3. Critically, the team should be committing to delivery against estimates that they own, rather than being expected to align to an external party’s view.
2. Estimates allow us to forecast how much work can be completed within a given time period.
   1. This allows us to effectively budget for project delivery, plan the resource schedule and manage customer expectations.

The former is often more important than having a perfect view of the latter. Lack of shared understanding can easily lead to accidental scope creep, rework or errors in interpretation.

Use of effort points

We use “effort points” for estimation - as opposed to other common estimation methods such as T-shirt sizing.

This involves assigning a number to each card that gives a relative sense of scale for this item. The numbers we use are loosely based on the Fibonnaci sequence.

Numbers used would therefore be taken from the following list:

0.5, 1, 2, 3, 5, 8, 13, 20, 40, 100

The way this works is as follows:

1. The team agree a baseline that they will use such that a relative sense of scale can be established. An example of this would be:
   1. “A simple page with a form that allows us to create and update a basic user is 3 effort points”.
2. Once this baseline is established, all other items can be scaled relative to this. If a card is perceived as taking twice as long as a simple “create, read, update” work item, then it’s a 6.

What are we estimating?

The estimate covers completion of the work item such that it meets the Definition of Done. This means it must include all effort required to develop and conduct QA on the work item, as well as supporting any bugs that emerge from User Acceptance Testing. If the Definition of Done includes releasing this work to the Production environment, the effort for this should be included too. A typical Definition of Done can be found within the controlled documents repository in COM012 - Statement of Work Template v1.0.

A note regarding time, complexity and effort

Lots of agile/Scrum learning resources create a confusing picture around what “story points” or “effort points” are intended to represent.

These resources might argue that “time” is not the only component we are trying to cover when assigning the number of effort points. These resources will suggest that “complexity” should be factored in independently of time. They might suggest that “risk or uncertainty” should be considered independently.

While the theory of this is debatable, for the purpose of Audacia project estimates, we keep this simple; effort points are an abstraction over time only. The rationale is simple:

1. If something is more complex, it takes more time.
2. If something is more risky or uncertain, it could take more time.
3. We need to build resource schedules and sell time based on a forecast of how much time the work will take us.

The Scrum Master is responsible for planning and running estimation sessions on a regular basis.

Assuming we are using Azure DevOps to track work, this would involve doing the following:

1. Create invites in calendars for the regular estimation sessions.
   1. The development team conduct estimation - so Aattendees will include both developers and test engineers.
   2. The client Product Owner and e.g. client SMEs do not attend the estimation sessions.
   3. A good length for an estimation session in 45 minutes. People start to lose focus after this.
   4. Ideally, estimation sessions should happen well before the sprint in which the items in question will be delivered. This helps avoid a situation where new questions are being raised at the last minute that could block delivery.
2. Identify work items that will be delivered imminently that have their acceptance criteria signed off that do not already have an effort estimate.
3. From the Backlog view in DevOps, select the items to be estimated, right-click and select Estimate work item(s).
4. Give the estimation session a name e.g. 2024-03-27 and create the session.
5. In an ideal world, share a link to the estimation session to the team in advance of the meeting, such that they can review cards at their leisure, capture any queries as applicable, and consider the effort involved.
6. Run the estimation session as per the section below.

Note that it is not dictated that people must use the Azure DevOps Estimate extension for running estimation sessions. They may use a Jira equivalent, some other planning poker tool, or planning poker cards if estimating in-person.

It is normal for requirements to change during the course of delivery. If requirements change, the previous estimate should be removed and the work item should be re-estimated.

Summary

The estimation process will be tailored to the project in question and this process provides an example of how this could work, rather than this being prescriptive.

1. The Scrum Master chairs the estimation session.
2. All team members should access the estimation session link.
3. At the start of the estimation session, the Scrum Master should remind people:
   1. This is not just about agreeing numbers. This is about verifying that we have a shared understanding of the business requirements.
   2. If team members do not feel confident providing an estimate, they should be encouraged to say so rather than being rushed into providing a number without feeling they have the required context. It is better to take away queries and estimate as part of a future session than to accidentally commit to delivering against a timescale for work we don’t understand.
4. Each card should be reviewed in turn. This essentially involves reviewing the acceptance criteria and verifying that the team have a shared understanding of the objective. Often, this will result in queries being captured.
5. If queries are captured that could impact the effort involved, the item should not be estimated - instead the Scrum Master should take this item away for clarification.
   1. This is generally done by capturing the queries in the Questions and Clarifications section of the work item.
   2. Queries are a good thing. The PM should not take this as a criticism of the work they’ve done to define the work required. Instead, feedback should be encouraged such that a clear consensus of understanding is established.
6. If the team identify an opportunity to split the work item into smaller chunks, the item should not be estimated - instead the Scrum Master should take this item away, split it, and present the split items at the next session.
7. If it has been verified that the team have a shared understanding and no impacting queries are raised, each team member will supply an estimate for the work item.
8. If estimates are consistent i.e. everyone picks the same number, the Scrum Master can simply allocate this number to the work item.
9. If estimates sit in a wide range, this implies that team members do not have a shared understanding of the work involved. The Scrum Master should challenge on this and should ask searching questions e.g.
   1. “You’ve gone for a lower number - why do you think this is simple?” or “Have you considered the testing effort involved?”.
   2. “You’ve gone for a higher number - which aspect of this feature do you think would be tough to deliver?”.
10. If the estimates in the first round sit in a wide range, once the searching questions have been answered, be willing to estimate again.
11. The Scrum Master will populate the Effort field with the agreed number.
    1. Note that while each team member will supply an effort estimate using the list of numbers in the not-quite-Fibonnaci sequence, it is perfectly reasonable to put an average on the card. If 2 team members estimate an 8 and the other 2 team members estimate 13, allocating 10 or 11 effort points to the card is reasonable - provided searching questions have already been asked to help close any gaps in understanding.

Different people estimate in different ways

When supplying estimates, people approach this in various ways that can be loosely grouped into the following categories.

1. Honest, individual perspective
   1. Some people give their honest, individual view on the size of the work item. They focus on how long it would take them, rather than trying to predict how long it might take others.
2. Pre-empt, then fall in line
   1. Some people attempt to pre-empt the number of effort points that other members of the team will estimate, then adjust their own number such that they are not seen as being an outlier.
   2. Example: 3 developers are estimating a work item. Developer 1 thinks that for them, the item being estimated would be an 8. However they suspect the other developers will estimate 5, because they have experience delivering similar features - so they fall in line and estimate 5. This is likely driven by a desire to avoid exposing themselves as having less knowledge or experience in this area.
   3. This is a problem, because the effort point number chosen as a result of estimation should reflect the team average - or it might be that this developer has identified an area of complexity that others haven’t.
3. Pre-empt, then attempt to manipulate the average
   1. Some people attempt to pre-empt the number of effort points that other members of the team will estimate, then adjust their number such that the overall average of the estimates falls in the range they think it should be in.
   2. Example: A Test Engineer and 2 Developers are estimating a work item. The Test Engineer suspects that the Developers will estimate a 5. The Test Engineer thinks this should be an 8 because testing effort is significant. They therefore estimate 13 in order to increase the average.

The above are all natural behaviours. The individual leading the estimation session (usually the Scrum Master) should be cognisant of the different approaches that people may use. They should encourage people to use the first approach, and correct for people using the second or third approach.

Estimating the time required to complete a certain work item is intrinsically difficult. When estimating complex or poorly defined features, this difficulty increases.

Estimating using ranges is an excellent way to acknowledge and address this fact, such that energy can be focused on refining the features that could have the largest impact on project viability.

As an example, let’s say we have two items to estimate and the team of 3 people estimates as follows:

In both cases, we could reasonably say that the effort number that should be assigned is 13.

However, by using this single number, we throw away incredibly useful information. It is hard to understate how important this is. The reality is that for “PBI 1” we understand the work well and can predict delivery timescales with a high degree of confidence. For “PBI 2”, we either to not understand the requirement well, or we have a range of opinions on the required solution and/or the effort involved. “PBI 2” is a low confidence estimate. Knowing this is useful information that should not be thrown away.

This becomes particularly critical at the Sales/Discovery phase, or when estimating new tranches of work that have only been defined at a high level - since this generally involves estimating work items that we don’t understand in-depth.

To illustrate this with an example, let’s look at two example backlogs.

Backlog A

Backlog B

The implications are obvious. Let’s assume that in both cases our predicted team velocity is 2.

For Backlog A:

1. We believe we understand the work well.
2. We have a high degree of confidence that we can deliver within 32.5 working days.
3. There is no argument for a large contingency budget.
4. Because of the above, it is very easy for the client to predict their return on investment.

For Backlog B:

1. We do not understand the work well.
2. We have low confidence in delivery timescales. This could take us 20 days, or it could take 50 days.
3. If we sell this to the client using the average (70 effort points, 35 days), we do them a disservice, because we hide the fact that it’s highly probable we won’t hit this.
4. There is a strong case for adding a significant contingency budget.
5. The return on investment calculation should arguably be done against the 50 day cost, not the 35 day cost.
6. If this contingency budget and level of uncertainty around the RoI is undesirable, there is a case for doing more work on refining requirements prior to commencing the project.

Because of this, range estimates are an incredibly useful tool at both the sales stage, and when estimating new poorly understood work items that are added to the Product Backlog during the delivery phase. This allows the Product Owner to focus on refining requirements for those items that we have low confidence in which could result in large numbers, as this is the best place to invest energy in order to de-risk delivery.

Once the work has been better defined, the range can be converted to a single number - this must happen prior to any work item being committed to a sprint.

This section covers some supplementary discussion and is not considered “critical reading”.

This is best expressed using an analogy. Let’s say we have a building with 4 equally sized rooms and we ask a decorator for a quote for painting these rooms, they might tell us it will take 4 days total.

Estimates would be:

Room 1 - 1 day
Room 2 - 1 day
Room 3 - 1 day
Room 4 - 1 day

In practice, we might find it takes 2 days to pain Room 1. Rooms 2, 3 & 4 are still in our backlog and in order to plan effectively and assuming all other things are held equal, we can reasonably assume the remainder of our backlog looks like this:

Room 2 - 2 days
Room 3 - 2 days
Room 4 - 2 days

This creates admin overhead, because we have to update the forecast for all remaining work items every time we deliver a work item that doesn’t match the original estimate. Using effort points is superior because it avoids us having to do this. To illustrate this, let’s say our original estimate is as follows:

Room 1 - 3 effort points
Room 2 - 3 effort points
Room 3 - 3 effort points
Room 4 - 3 effort points

Our predicted velocity is 3 effort points per day. At the end of day 2, because Room 1 has taken longer than planned, we don’t need to update any subsequent work items. We simply say “our target velocity was 3, but our actual velocity is 1.5. To predict the remaining work at the end of day 2 is straightforward:

Effort points remaining (9) / Velocity (1.5) = Days remaining (6)

Avoiding this admin overhead is just one reason. Other reasons include:

1. If someone is asked to estimate in days/hours they inevitably struggle to keep this pure. They start trying to add additional corrections e.g. “This would take me a day, but I have lots of meetings next week, so I’ll estimate 2 days to try and account for this”. This creates inconsistency in estimates, so using an abstraction helps prevent this.
2. When the client is given an explicit number of days/hours, this will generally be interpreted as a commitment as opposed to an estimate.
3. Using the “effort points” abstraction allows us to plan changes effectively e.g. to forecast the impact of adding additional development resources, or changing our working processes such that we achieve a higher velocity.