I want project planning software which accepts lower and upper bounds, or an estimate + %age confidence, and then at the end gives me a project plan with error bars on each part of the timeline. Let me give stakeholders a diagram showing exactly what we mean by it being hard to say how long this will take, rather than what appears to be a rock solid plan timed to the day.
Also, let me fire any project manager who pushes back on giving confidence levels (yes, I’ve met several).
If you ever get around to that, let me know. JIRA really is best in class and I want to shove it into a very tiny, very lonely box before tossing it into the deepest part of the ocean I can find.
I myself am hoping someone will pick up X=spreadsheets at some point soon.
My favourite is the ones who not only will push back on confidence levels but will then drill down into any estimates they think are "conservative".
If you decide they are described by fat-tailed distributions¹, then the exact distribution and its parameters are much more important than your estimated intervals, so again collecting them adds nothing.
So, yeah, collecting confidence intervals never adds anything.
1 - Like they should, because nobody does a project on anything they know well enough to describe with classical statistics.
I'm trying to understand what you mean by this. Are you referring to the fact that for thin tails, the sum of expectations will grow significantly faster than the standard deviation, and thus the 90th percentile will, relatively speaking, tend toward the expectation with more tasks? (I.e. an appeal to the LLN.)
However, in task estimation[duration] aren't normally distributed - they're much more log-normally distributed, and you can't simply add log-normal distribution parameters in the same way[2]. Instead he log-normal distribution is (largely) closed under multiplication which is fairly useless when we want to add tasks to determine total time.
Moreover, the reductive conceptualization of normally distributed estimations leads people to the erroneous communication of "X plus or minus Y". The assumption is that the bounds are symmetric(they aren't). The chance that the estimate is higher than the mode is much greater than it is lower than the mode.
As a data scientist, I want to say that the model for task estimation is over-reduced. Even so, I feel like the push-back I'd get from suggesting that folks estimate in log-space would be that I'm being ridiculous complicated.
1. https://en.wikipedia.org/wiki/Sum_of_normally_distributed_ra...
2. See the Fenton-Wilkinson approximation.
Its a more theoretically grounded relative of the common agile “T-shirt sizes quantified by Fibonacci sequence, without the repeated ‘1’” estimation recommendation (which is pretty close to just doing “each complexity step is ×1.5”.)
You won't add meaningful information to the analysis.
https://jbconsulting.substack.com/p/task-estimation-conqueri...
That said, I don't think the findings on that article generalize well. There seem to exist some properties that makes many software projects more erratic than a log-normal would imply.
On more erratic than log-normal, well, I mean it. There are projects that can't be fit to a log-normal, they have fatter tails. I have no idea why. A log-normal is something tractable, many projects aren't even that.
(Unless you mean that the summary of the estimate, rather than the shape of the distribution, varies a lot depending on whether you're quoting a 95% or a 99% confidence interval. But that is just the nature of long-tailed distributions).
I'm also not sure what it means to "fit" a project to a log-normal. A single project doesn't have a distribution, unless you're measuring completion time of individual tickets within that project. The entire project's completion time might be very far off of the median estimate, but as long as the distribution assigns a nonzero probability to it, it's hard to say from just one sample whether it was right or wrong.
If you are measuring the distribution of the individual tickets and they aren't distributed along a log-normal, I'd be very surprised; it would be worth seeing what distribution they do fall into to learn what state of knowledge is being captured by those estimates.
"Yes, I think it's a minor change. Almost certainly done by the end of the week, or else we have bigger problems."
"I don't know what this entails. Looks like it could be done in a day, but I wouldn't bank on that unless I can spend an hour investigating."
"This is a large feature, but nothing seems terribly complex. I think it's one or two weeks, but we should be ready for it to get snagged on small defects. We should leave room after the release for a rapid followup patch that may be needed."
You may think something takes 9 days, but if you are only allowed to answer either 8 or 16, choosing 16 is the only safe option and automatically includes some lack of confidence.