Search
Close this search box.
Search
Close this search box.
Search
Close this search box.

Comparing Estimation Techniques

Facebook
Twitter
LinkedIn
Pinterest
WhatsApp

Which Estimation Units Should Your Teams Use For Work Items? 

Why estimate in the first place?

  1. At the feature level, you want estimates so you can figure ROI (Return On Investment), so the business and product owners can prioritize one feature against another
  2. At the roadmap & backlog level, most organizations want to be able to forecast, be it at the feature level or release level, so you need to be able to understand what is your organization’s capacity, and translate it into plans with dates and deliverables, milestones, etc.
  3. At the finer-grained, work-item level (typically User Story),  you want to help teams understand their capacity so they can take on the right amount. In particular in teams that use a frequent planning cadence, such as that prescribed by Scrum
  4. Still, at the team level, team members setting expectations on the amount of work they plan to complete can surprise themselves, learn, and improve.
  5. At the team level, you want to understand whether a work item is small enough, might want teams to discuss estimates as part of creating a common understanding about work and aligning their views

What attributes should you look for in units you use for work-items-level estimates?

I have listed here some attributes, prioritized based on my view (most important first):

  1. Intuitive – The units should make sense to newcomers, and to stakeholders outside the team itself. If it is opaque, it hurts transparency and makes it hard for team members to get their heads around it.
  2. Encourage team thinking, which is another way to say seeing the big picture. One of the main points in agile methods is the emphasis on the team, rather than the individual expert. Additionally, Lean Thinking puts an emphasis on optimizing the whole system, meaning that if I’m a software developer, it’s better for the system if I focus on getting user stories done-done, rather than focusing on finishing as many of my programming tasks as I can.
  3. Just enough precision – Estimation can be wasteful, in particular, if we do it too early and too detailed. Estimates will be wrong anyway (the very definition of “estimate” is “an approximate calculation”). Think about estimating the duration of a trip from Vegas to LA next month. Does it really make sense to argue if it is going to take 4:00 hours or 4:10?
  4. Be indifferent to the expertise of individual team members – If there are two back-end developers in the team, one is very experienced whereas the other is a newbie, they are bound to provide very different estimates if asked for hours to complete this. If we can somehow find a way to neutralize that effect, it would be swell.
  5. Scales up to multiple teams – In cases where a product or project requires the work of multiple teams, you will want to aggregate estimates, and combine teams’ velocities, so you can plan and track the whole effort, and coordinate dependencies using the same language in terms of progress and size.
  6. Minimize the risk of confusing effort and duration – One of the struggles of software development estimation, planning, and tracking, is the common practice of using time-based estimates (as in “this will take 5 days”). The problem here is that the very same unit is used for duration. After all, planning in software development is a type of rate problem, like the time and distance problems your kids hate in middle school. Think about discussing a trip from Vegas to LA. How far are they? (distance) – 4 hours away? No, it really depends on your speed and whether you stop along the way, etc. If you take a plane, the duration will be about 30 minutes. If you drive on Saturday (peak time) and you stop for a shopping spree at Ontario Mills, it may take well over 12 hours. The distance, however, is around 260 miles. It would be confusing to say that the distance is 4 hours, yet, this is exactly what we do in software development. Can we find units that sidestep that problem?
  7. Promote ingestion of small work items – One of the main struggles of teams moving to agile, is the need to break down work into small chunks. If the estimation “language” we use can challenge us to slice those work items thinner and thinner, that would be a benefit.
  8. Easy to reconcile with feature-level estimates. You will typically use a hierarchy of work items. From the large initiatives and roadmap items, down to features (or MMFs – Minimally-Marketable-Features), to team-level work items, the building blocks we typically call User Stories. If you use one unit for the features, and another unit for work items, you want to have an easy way to map from one to the other, so you can track progress without suffering a discontinuity.
  9. Creates a fair economy of velocity – This is a culture thing. Measuring productivity in software development teams is impossible (there are saner alternatives, though), and any naive and misguided attempt to do this using numbers amounts to senseless bean-counting that is pretty much guaranteed to achieve the exact opposite of what you want. Ideally, the unit you use would make it hard for the naive manager (or executive) to go down that path. Your goal with work-item estimation units is to help the team do their work and plan their capacity, not help executives wring the last bit of “productivity” out of the team.
  10. Good at accommodating a variety of item sizes – Some teams find themselves handling both tiny user stories one team member can bring to completion in a single day, and large user stories they can barely complete in two weeks, even if multiple team members work on them. Ideally, the unit we use can accommodate both extremes.
  11. Help validate capacity allocation for different skills in the team. For example, if all the stories the team takes on are very heavy on back-end development, and the team has a limited amount of back-end development expertise, the total estimate of all those stories won’t be a good representation of the team’s ability to complete these.

What are your options?

  1. Task-level time. Work items are broken down into tasks, and each of those tasks is estimated by the expert in the matching area. The breakdown can be done by expertise, where tasks look like “design”, “back-end development”, “front-end development”, “and testing”, or can be broken down in a more fine-grained level like “Add field to API”,  “Document API change”, “Write unit test” and so on. Task estimates are then rolled up to the work-item level, and you have a total estimate, in addition to skill-based estimates (e.g. this User Story will take 12 hours of back-end development, 3.5 hours of front-end development, and 4.25 hours of testing). This is often the default for teams that are just getting started with agile, and they will often keep doing it this way. Note that some teams use one of the methods listed below, and later on (just before the sprint/iteration starts, as part of some pre-planning) will perform that task breakdown.
  2. Story-level ideal-days. The team just estimates the total amount of effort needed to complete this work item, including testing, development, design, etc. The effort is calculated in days of work. Ideal in the sense that we are assuming that a single person is working on this work item at any point in time, without any interruptions and distractions.
  3. Team-weeks. This is equivalent to the Ideal Days, but it emphasizes the team aspect by asking “If the whole team could, and would work solely on that work item, how many weeks would elapse before it was complete (done-done)”. This does take into account the normal amount of interruptions and distractions. Numbers will typically be less than one, assuming a large enough team and small enough work items.
  4. Story points. This is a relative, unitless indication of effort. The team identifies a small (possibly the smallest) work item, and, looking at other work items, asks “how many times much effort is needed to complete this work item, relative to that small item”. This relative sizing has a few advantages.
  5. No-estimates – This approach, proposed by Vasco Duarte, suggests that estimating is dysfunctional by nature and that counting the number of stories that can be completed in a period of time is the best predictor of progress. A variation proposed by Yuval Yeret proposes using an “Equivalent # of stories” at the feature level, and perhaps using this as well for unusually larger stories

Any other options you know of? (I hope you won’t mention Function Points or KLOCs)

Okay, now let’s compare those:

 

Task-level time (hours or days) 

<bStory-level

Ideal Days

Team-week

Story points

No-estimates (just count stories)

Intuitive?

Definitely, this method is very common even pre-agile

Yes

Somewhat

No

Story points is an abstract concept which is hard for some to
grasp. Definitely not intuitive to business stakeholders who see that as a chinese wall for teams to hide behind

Somewhat

Encourages team thinking?

No, counting hours by expertise (testing, back-end…)

Somewhat

Estimate aggregates all expertise
needed to complete this item

Yes, the estimates reflect overall effort for the team

Yes, the estimates reflect overall effort for the team

Yes

Just-enough precision?

Very precise… too precise to be used until the work is
about to start. Requires a lot of effort in estimating

Yes

Yes

Yes

Uses Fibonacci series which provides a
variable resolution

Absolutely.

Indifferent to individual team member’s experience?

No

May require assignment of tasks to individuals so they can
estimate how much it will take them

No

For me it is 4 days, for you it may be 1 day (you have more
experience, knowledge of that area…)

Yes

The team is an amalgam of skills and level of expertise, whose
totality is reflected in this estimate

Yes

Regardless of whether you are an ace developer or a newbie, a
5-pointer should be around 5 times more effort than a 1-pointer

Yes

Scales up well to multi-team level?

Somewhat. Can easily be translated to days or weeks of work,
although it is likely to be tedious

Yes

Yes

Not very well. The whole concept is based on a team deciding
what does the 1-point story represent in their world. When you scale to the
release train and value stream if is very hard to aggregate the
story-points-based velocity of teams into a value-stream, release-level
velocity

Somewhat.

Story-based velocity can be aggregated across teams.

Same as story points. My typical story is not necessarily your
typical story.

Risk of confusing duration and effort?

Yes

Bean-counting at its best

Yes

“Why does that 5-days story take two weeks? You have 4 spare days
in your sprint, can’t you squeeze this one in?”

Yes

No

No

Easy to reconcile with feature-level estimates?

Yes

Time arithmetic is straightforward

Yes

Somewhat

At least at the single team level, it is fairly straightforward

Somewhat

Whether you use story points as well for features, or T-shirt
sizes, you can map these to the story points you use in stories

Somewhat. For a specific team, you’d need to estimate features
as “number of stories”, which is feasible.

Creates a fair economy of velocity?

No, teams are managed by the numbers, creating a pressure to
deliver hours, and likely to create a burn-out and tech-debt

Somewhat

Somewhat

Mostly

Makes it harder to

Yes, team focuses on getting things done, and getting it done
the right way, vs. just “making good time”

Good at accommodating large variety in work-item sizes?

Yes, anything that can be broken down to tasks can be estimated

Yes

Yes

Somewhat

The scale goes to infinity pretty fast, and you need to revert
to using a different scale for large items

No

At the sprint level, if a team has a significant variety in the
size of work items, # of work items won’t be very helpful in deciding how
much can the team take on

Helps in validating capacity allocation at the expertise level? 

Yes, albeit in a dysfunctional way which keeps the team in their
comfort zone of expertise silos. ( As opposed to tasks being created,
estimated and assigned close to when the work-item is about to be worked on)

No

No

No

No

Estimates may hide imbalance in expertise needed

So, bottom line, which unit would I recommend?

  • For the single team, story points are a good option if people don’t get goosebumps when they hear about the concept. So far my success rate is about 50%. Even if you do use story points, perhaps start with just using 1, 2, 4, too-big, or 1, 2, 4, 8, too-big
  • For multiple teams, I would start with story-level ideal days (which is effectively what SAFe uses), and use a variable resolution scale (e.g. Fibonacci).
  • For both cases, you should collect data for some time (e.g. three months), and examine if, in hindsight, counting stories is a good enough approximation for velocity, and if so, consider moving to a no-estimates-based approach
  • If the story-based velocity is pretty stable for all teams, you can move to a “no-estimates” kind of setup, and liberate your teams from a bean-counting mentality

Links

Subscribe for Email Updates:

Categories:

Tags:

Jira
Agile Exercises
Agile Development
Nexus vs SAFe
Risk-aware Product Development
Manage Budget Creation
Spotify
PI Planning
User stories
System Integration Environments
SPC
Lean Agile
Value Streams
An Appreciative Retrospective
Lean-Agile Budgeting
Continuous Planning
Sprint Retrospectives
Releases Using Lean
Agile Project
Agile Release Planning
Principles of Lean-Agile Leadership
speed at scale
Change Management
Agile India
Continuous Integration
Introduction to ATDD
agileisrael
Agile Marketing
ScrumMaster Tales
Portfolio for Jira
Business Agility
Legacy Enterprise
Certification
ATDD vs. BDD
Professional Scrum Product Owner
Managing Risk on Agile Projects
Operational Value Stream
chatgpt
Agile
Built-In Quality
WIP
Frameworks
Scrum Master
Quality Assurance
LAB
Agile Product Ownership
Test Driven Development
Continuous Improvement
Covid19
ALM Tools
Product Management
Agile Project Management
RTE Role
Coaching Agile Teams
Agile Risk Management
Agile in the Enterprise
Atlassian
Scrum Guide
speed @ scale
Story Slicing
Nexus
LeSS
Product Ownership
Applying Agile Methodology
Agile Contracts Best Practices
Agile Games and Exercises
AI Artificial Intelligence
Professional Scrum Master
Scrum and XP
Continuous Deployment
Sprint Planning
Jira Cloud
Agile and DevOps Journey
Kanban Game
SAFe
Scrum Primer
Agile for Embedded Systems
Lean and Agile Techniques
Pomodoro Technique
Agile Games
DevOps
Agile Program
Lean Agile Management
Tools
Scrum
LPM
Legacy Code
TDD
Lean Startup
Rapid RTC
Games and Exercises
Program Increment
Webinar
Lean Software Development
Presentation
Tips
Scaled Agile Framework
Accelerate Value Delivery At Scale
Lean Budgeting
Kanban 101
POPM
RTE
Elastic Leadership
Enterprise DevOps
Lean and Agile Principles and Practices
lean agile change management
Code
BDD
Engineering Practices
Scrum Values
Agile Assembly Architecture
Agile Community
System Archetypes
Scrum With Kanban
Continuous Delivery
What Is Kanban
Perfection Game
AI
Kanban Basics
Atlaassian
Nexus Integration Team
System Team
Scrum Master Role
predictability
Agility
Agile Israel Events
Video
Achieve Business Agility
GanttBan
Jira admin
AgileSparks
Risk Management in Kanban
Agile Techniques
Daily Scrum
Introduction to Test Driven Development
Kanban
Effective Agile Retrospectives
Professional Scrum with Kanban
Software Development
Certified SAFe
QA
Agile Release Management
RSA
Advanced Roadmaps
Lean Risk Management
Iterative Incremental Development
Agile Basics
Implementing SAFe
Self-organization
The Agile Coach
EOS®
Nexus and Kanban
Process Improvement
Risk Management on Agile Projects
ATDD
Sprint Iteration
Acceptance Test-Driven Development
Lean Agile Leadership
Amdocs
Lean Agile Organization
A Kanban System for Software Engineering
Agile Delivery
Development Value Streams
ARTs
Artificial Intelligence
Agile Testing Practices
Agile Mindset
SAFe Release Planning
Hybrid Work
Agile Israel
Limiting Work in Progress
Kaizen Workshop
Slides
Managing Projects
Implementation of Lean and Agile
Software Development Estimation
Lean Agile Basics
Large Scale Scrum
Lean-Agile Software Development
The Kanban Method
Kanban Kickstart Example
Release Train Engineer
SAFe DevOps
Entrepreneurial Operating System®
Agile Outsourcing
Jira Plans
NIT
Kaizen
Systems Thinking
PI Objectives
Planning
Scrum.org
Nexus and SAFe
SA
Agile Product Development
ROI
IT Operations
Reading List
AgileSparks
Logo
Enable registration in settings - general

Contact Us

Request for additional information and prices

AgileSparks Newsletter

Subscribe to our newsletter, and stay updated on the latest Agile news and events

This website uses Cookies to provide a better experience
Shopping cart