Hacker News Reader

Comment by psunavy03 | Hacker News Reader

> Even with Agile we can’t get stuff done in a satisfactory time frame because we always have a backlog sized for a team twice the size of the one we have, when responsiveness is maximized when the system is running at 50% of maximum throughput.

Agile falls down when people misapply it, same as anything else. It's not just the size of the backlog; it's being able to limit work in progress so that you have the ability to adjust. What's more, management needs to get on board with the idea of probabilistic forecasting that's continually revisited, as opposed to trying to stuff complex work into Gantt charts and deadlines. Sadly, most of modern management refuses to make these changes, and too many folks in the trenches don't want to take ownership of their work and just want to be told what to do.

View on HN

There’s a modification to Gantt charts that uses Monte Carlo simulation to come up with a more believable timeline, but nobody likes bad news so it’s a fringe Agile thing instead of mainstream.

Great companies are few and far between. Everyone else thrives on self deception.

But if you're doing Monte Carlo, you might as well just iterate and keep re-running the Monte Carlo as you burn through the backlog, because that's a better way of having up-to-date information that using any kind of Gantt chart.

But as I said, the problem isn’t really technical anymore, it’s emotional.

You don't have to pick one or the other. A Gantt chart is just a pretty and easy to read graph based on the topological sort of activities and their dependencies and durations laid out in time. They also aren't meant to be static, unless you're working for a badly managed org that uses Waterfall the Gantt chart gets updated as things progress and new information comes in.

If you have a backlog, and you don't mark what is dependent on what (in progress or also in the backlog) you're just hurting yourself. Once you add that information and some very basic estimation (even just scale of expected effort is enough) you can generate a Gantt chart and use Monte Carlo simulations to get an understanding of your time estimates.

I’ve only ever seen Gantt charts used by waterfall assholes who talk about commitments and promises to get free overtime.

Some techniques are good but ruined utterly by the company they keep.

In my experience, estimates are the root cause of quality problems and expectation mismatches. No one treats them as estimates but as actual calendar times.

I've been fortunate to steer my company towards simply prioritizing work and communicating the prioritization to the rest of the company and more importantly, our customers. We don't give time estimates or timelines to customers, but provide constant updates on where something is. No one has complained about this in general. Of course, there are always exceptions - we resist them all we can, and that too is reflected as reprioritization of backlog.

  > No one treats them as estimates but as actual calendar times.

its true, though i suspect this is partially because its what the management chain wants: a date to report when it will be done (e.g will my okr for this quarter be met or not?)

  > estimates are the root cause of quality problems and expectation mismatches

i have seen this over and over, most quality issues and incidents are caused by decent programmers rushing to meet the immovable deploy/release window... because 'your not gonna make your estimate'....

> its what the management chain wants: a date to report when it will be done (e.g will my okr for this quarter be met or not?)

Not just OKRs directly; a software project is usually itself just a node in a larger dependency graph. For example, the company wants the release to coincide with an industry event at some time T, which means that the project must be done by time T, and also must be mostly done (in some well-defined way) at time T-6 months, so the marketing people can be brought in to do their marketing things in time for the event. In many cases (video games come to mind), if there's not a reasonable certainty the project can hit those goals, the company may be better off scrapping it altogether, rather than wasting millions on development, and millions on marketing, only to miss the target entirely.

In general, management wants dates, because most activities that aren't software projects are coordinated through calendar dates, and software projects don't exist in isolation.

Yes. For us these are exceptions rather than drivers of software delivery. External events are certainly hard dependencies, but more often than not, the deliverables are more marketing than software. That is, a proof of concept, something behind a feature flag for the event - that will not be used in production. We do this routinely, by placing a PoC on top of the backlog. Certainly not for all organizations , but I have had the fortune to shape this model company-wide.

  > In general, management wants dates, because most activities that aren't software projects are coordinated through calendar dates, and software projects don't exist in isolation.

very true, though i wonder how agile (small a) fits in to all of this, can an organization that is working iteratively on all fronts (including management) avoid (or at least reduce) these dependencies?

> Sadly, most of modern management refuses to make these changes, and too many folks in the trenches don't want to take ownership of their work and just want to be told what to do.

Interestingly enough this is something I see in management books and operations research books going back to the 70s. It's a lesson that hasn't been learned.

As for the ownership - I think that makes a lot of sense. People in the trenches know very well that they don't really own the thing, but are just at best responsible for it. I think that is perfectly fine, and the whole "ownership" language tends to obscure very real power dynamics.

> management needs to get on board with the idea of probabilistic forecasting that's continually revisited

From the manager's pov, though, that just sounds like guesswork. "When will my house be built?" "Eh, not sure, but theres a 60% chance the framing will be up by July".

Development managers need to learn to communicate on the same wavelength as their customers, and vice versa. It rarely happens.

The thing is construction people do talk like that.

I think that’s why rich people often make terrible customers. They are just as grouchy at plumbers and general contractors as they are at us.

Which reminds me, one of my life goals is to get a full rundown of GC tricks to apply to software development. I’m running out of time for that to make a quality of life difference.

You're conflating the complicated with the complex. Construction workers don't need Agile methods, which is why "there's a 60% chance the framing will be up by July" sounds so dumb. The physical properties of wood framing, electrical wire, shingles, and drywall haven't changed in decades. You can make detailed plans around these known facts, and workers generally know predictably what it takes to build a house.

Software is not like that. Codebases are too big, especially counting third-party dependencies. Tech debt is lurking everywhere. Customers don't know what they want until they see it. So yes, in enterprise-sized software, you need probabilistic forecasting precisely because you're NOT building a building. It's impossible to know things in enough detail up front to make big up-front plans that don't largely change like you could if you were building a house.

I say this with love but, you've never worked in construction have you.

Architects drawings are little more than nicely descriptive hopes and sketches of an intended idea, which a good contractor has to turn into an actual plan of work.

You want to see chaos go talk to the person running the development of a high end property in New York.

I have a way of getting people to talk to me about their professions. At the tail end of a water damage repair, the GC confessed to me that they flood customers with trivial choices to distract them from the illusion of choice in other areas. Like the physics of plumbing dictating where sinks can go.

As I mentioned up thread, I want to buy a GC beers and get them to tell me more, before I ever take another contracting gig.

> You're conflating the complicated with the complex. Construction workers don't need Agile methods, which is why "there's a 60% chance the framing will be up by July" sounds so dumb. The physical properties of wood framing, electrical wire, shingles, and drywall haven't changed in decades.

When's the last time you saw a construction project that landed on time? Construction projects are a classic example of forecasting difficulty. Lots of things have to go smoothly at the right time, including supply chain, coordinating work from multiple organizations (including local governments), and the weather has to play nice.

You need probabilistic forecasting for everything. I used it with great success to forecast the costs of the general renovation of an apartment we were moving into. Seeing the shapes and ranges of distributions was very informative (and in particular informed the decision on whether and when to take an extra loan). Can't imagine doing it any other way now, even though I had to hack my way into doing it, because approximately none of the tools I know of support this out of the box.

(I ended up using Guesstimate for it - https://www.getguesstimate.com/ - pushing it to the limit of nearly hanging my browser.)

Problem is, most people seem to be overwhelmed by those ideas. It's not hard, but then again multiplication isn't hard either, and most people are afraid of that too. This is a problem because software tends to target the lowest common denominator, which is how we get a million Trello clones, but no tools that understand that work breaks into DAGs, not 2-level-deep trees, or that Gantt charts are good to have, or that probabilistic Gantt charts would be even better.

> too many folks in the trenches don't want to take ownership of their work and just want to be told what to do.

And do you blame them?

If pressed I’ll put this as, “once you substituted your judgement for mine this became your problem.”

I’m not going to enthusiastically absorb the consequences of bad decisions I already tried to route us around.

Reply on news.ycombinator.com