Hacker News Reader

Comment by jacques_chester | Hacker News Reader

Suppose you're asked to estimate a single sudoku problem. That's basically impossible, because n=1. In small samples from a large population, variation is enormous.

But what if I ask you to estimate the time for 100 sudoku problems? For 1000? What if, after 2 years of solving sudoku problems full time I ask you for your estimate? What if I develop simple statistical models based on the number of squares that are already filled in?

Already it has become possible to improve your estimate, to make it more accurate. The value of an estimate is balanced on the line between the cost of its uncertainty and the cost of its development. Over time, for any class of problem, the cost of developing estimates falls.

View on HN

> What if I develop simple statistical models based on the number of squares that are already filled in?

What if you were unable to? NP-hard problems are weird like that. What if the underlying statistical distribution had random variance or some other strange features that make traditional statistical models useless?

Maybe the issue is that we're trying to use intuitions developed with everyday problems that are essentially linear ("mow the lawn", "drive to the store") in places where they're not appropriate. Unlike usual everyday tasks, NP-hard problems (such as Sudoku or coding) have no known "royal road" - sometimes you just have to try all possible solutions.

NP-hard problems often have "good enough" solutions that don't take heat-death time to solve. Consider the TSP. You can wait until the last proton decays and tie up all matter and energy in the universe.

Or you can use an ant colony optimiser and accept that you will probably only come to within a few percent of the best possible solution. Oh noes!

And so it is with estimation. We can build an ever-more-elaborate estimate that will consecutively shave fractions of a percentage point of precision and accuracy. Or we can accept that they are estimates, that they are meant to be imprecise and that accuracy is a range, not a number.

Definitely true that estimates can be improved from some hypothetical floor of quality..

Estimates still come with a cost in themselves though. They're not an end, merely a means. So we should consider alternatives.

An alternative approach I've heard from some teams, though haven't had the pleasure of trying myself, is to break your work into similarly sized small chunks and measure team speed over time in terms of these chunks. This sounds a lot like relative point based estimation often seen on agile teams. However, the difference is that once you establish this sizing rhythm during analysis (with the help of estimation early on to hone in on sizing) then you skip estimation altogether.

I suspect this needs to be calibrated from time to time ("our stories have gotten too big") and exceptions need to be made (e.g. "there's just no way to split this work up and keep it releseable") but those scenarios are exceptions, not the norm.

A world without estimates sounds great to me.

> However, the difference is that once you establish this sizing rhythm during analysis (with the help of estimation early on to hone in on sizing) then you skip estimation altogether.

I'd argue that really what you're doing now is estimating the size of any given chunk. Chunks that are estimated to be too big are then decomposed.

But you still make some judgement about how big each chunk is before you actually queue it up.

> A world without estimates sounds great to me.

Most people estimate, they just don't think of it as estimating. Similarly, most people can instinctively perform differential calculus; they just think of it as, for example, "catching a ball".

I think that you're right that estimates are never ends in themselves and that their net value needs to be considered. In a sense, you perform a meta-estimate of an estimate's cost and return before doing one. Is it worth developing parametric models of when you will be home for dinner? Probably not, dinner will be cold and moldy by the time you get to it.

"What if I develop simple statistical models based on the number of squares that are already filled in?"

Better to just have the experience to look at a Sudoku and have a good feeling for how difficult it is - it's not just the number of filled in squares but their locations and their values.

After solving 1000's of Sudoku problems, experience will allow one to see the relevant patterns or rather have a feel for them when looking at a particular Sudoku. The process is sped up if one starts off estimating Sudoku solution times from day one and then compares those estimates to actual solution times.

A longer way to developing meaningful estimates (and I prefer meaningful over accurate, since a good estimate can still be very wrong), is to avoid estimating one's time as an end in itself.

> A longer way to developing meaningful estimates (and I prefer meaningful over accurate, since a good estimate can still be very wrong), is to avoid estimating one's time as an end in itself.

Could you elaborate? What would one estimate on a book of sudoku problems?

Who is the audience for the estimate?

An estimate of how long it will take me to solve 1000 Sudoku for recreation will not have much meaning to a client who just wants the solution.

What are the resources that will be brought to the task?

An estimate for solving 1000 Sudoku with a staff of professional programmers will be different from one premised upon bribing a dining hall full of seniors with coffee, juice and donuts down at the retirement village.

What is the unit of estimation?

Calendar days until delivery is different from man-hours because I spend so much time on HN.

The value of an estimate is the integrity and relevance of the process by which it was generated not its accuracy. Accuracy is a fortunate side effect - and then only sometimes. The key is that accuracy correlates to experience at making estimates, and rationalizing the failure to go through the process because past estimates have been inaccurate is the long road I mentioned.

Reply on news.ycombinator.com