Hacker News Reader

I think it might have to do with the fact that the older you get the more knowledge and experience you have; thus you see problems you didn't see before although they did exist all along.

I like KISS. Usually there's three ways to solve a problem:

1) The quick workaround/fix. Almost always costs more in the long run but is fast.

2) The good enough fix. Costs a little bit more time and might not be the absolute best solution, but it's good enough.

3) The "real proper" fix. Costs a lot more time than the above with little value added over #2.

These days I'm almost always opting for #2 by default because it's good enough. It's hard at times to let things go that you know could be solved in a better way. There's seldom a real need to go down road #3 for other purposes than your own satisfaction though, and that sucks at times.

Amen. I've found that changing my conception of software from a concrete deliverable to a process has led to a more healthy relationship with my work. It's never done, and since I've made peace with that, I've rediscovered a different kind of delight in working.

These days, I regard change requests as fascinating plot twists, not blemishes on my completed work. And I find myself asking, "Did that fix your problem?" much more and "Am I done?" much less. Since I consciously stopped trying to mind-read and anticipate needs, I've had much more success. Keep it simple and assume nothing. After all, expectation is the root of disappointment.

My clients like it and I enjoy it more, too.

These days I'm almost always opting for #2 by default because it's good enough.

Me too, but I think it only works (and also doesn't introduce problems in the long run i.e. still keeps the software maintainable) because the software to which the fix gets applied is already properly designed to begin with. So in the end #2 only seems to work because the one applying it is good enough, and the software it's applied to is also good enough.

At least, that is my impression from a bunch of experiences: applying #2 to what is aleady a trainwreck will usually cause problems in the long run anyway, standard avalanche effect. For proper, elegant pieces of software though basically the difference bweteen #2 and #3 fixes seems to fade away: everything is so loosely coupled and has such well-defined responsabilities that most fixes which are #2 simply cannot become any better no matter how much time spent because they are so focused there is only one way to fix it. Or there might be an alternative way, but the outcome is exactly the same so it's also not really 'better'.

It's not waste:

- you learnt

- you had fun

- you felt proud

- it validated you

This is a path you need to follow toward mastering a field.

You are what you are today because you did it, and if you are any good right now, the world is a better place now.

Oh absolutely. The journey is the destination and all that. And I'm undoubtedly a better developer and coworker because of my experiences.

I'm happy there are people out there eager to wade through the shit just as I did when I was younger. Their enthusiasm is great. I'm just less willing to do that shit-wading these days. It takes all kinds.

Our society has a "genius syndrome" problem. Its culture promotes the idea of a master solving all issues. He is usually 20 something and does everything right.

But the fact is:

- most great people are not born great. They become it. If you want great, you need time to nurture it.

- most great people fucked up hard a lot on their way. Greatness implies training. Training on important things with consequences.

- most great people are not alone. Something great is rarely achieved by one person. Even if something appears to be done by one person, you usually have a lot of entities supporting the work in some direct or indirect ways. Without it, the so-called genius would have accomplished nothing.

Bottom line, we should remember:

- good persons, like good things, are rare and have a cost. Plus they take time.

- The lone genius is the exception, not the rule.

So you messed up on the way ? Good. This is life investing in your potential. Can't wait to work with someone like you.

Shucks. For what it's worth, your comments have made my day. I'm sure I'd have a blast working with you, too.

You wouldn't, perchance, have a masters degree in stochastic modeling, would you?

No I'm just a good old regular Python web dev and trainer. Also I worked in the porn industry in France and in NGOs in Africa for half of my life.

Very, very far from 'stochastic modeling', which I actually had to google before answering your comment.

People with resumes like yours were always my favorite co-workers. Diversity of experience leads to more life lessons per unit time.

Cheers, friend.

> I worked in the porn industry in France

As a web dev?..

Among things :)

All costs are opportunity costs.

The ultimate question is: how else might you have spent your time?

Understanding technology and returns to technology, by specific technological mechanism (an area in which the literature is oddly silent) as well as the long-term costs and consequences, and the efficiency-driven economic consequences (the Jevons paradox and related elements), may give pause.

Yes, there's something to be said for learning. But applying learning to domains in which it's not ultimately most productive is itself an opportunity cost.

There is almost always a benefit to finding and reaping low-hanging fruit, in raising up the least-served, in recognising who may not be serviceable at all (about 50% of all people have zero or absolutely rudimentary computer proficiency), or to recognise the value in providing advanced, expert-user interfaces which may serve a very small user base (a few percent, perhaps a few fractions of a percent), but for whom the added productivity is valuable to others.

The trouble is that the tinkerers learn to cope with the idiosyncrasies of the legacy systems and build their shiny new stuff on top of them. In the end, we get software that solves a relatively simple problem but depends on layers upon layers of other systems and abstractions.

IMO, we need to fundamentally rethink the design of our computing infrastructure so that it suits our current way of using computers. This would start from the CPU microarchitecture and instruction set, through the programming interface of the operating system, up to the middleware used by most programs.

Do you know if there's anyone seriously working on this? I feel like there are a lot of people on this website who want something like this and would love to help out.

Yes, there are people working on this.

VPRI[1], the group around Alan Kay had an NSF-funded project to reproduce "Personal Computing" in 20KLOC. The background was that way back at PARC, they had "Proto Personal Computing" in around 20KLOC, with text-processing, e-mail, laser-printers, programming. MS Office by itself is 400MLOC. Their approach was lots of DSLs and powerful ways of creating those DSLs.

This group then moved to SAP and now has found a home at Y-Combinator Research[2].

One of the big questions, of course, is what the actual problem is. I myself have taken my cue from "Architectural Mismatch"[3][4][5]. The idea there is that we are still having a hard time with reuse. That doesn't mean we haven't made great strides with reuse, as in we actually have some semblance of reuse, but the way we reuse is suboptimal, leading IMHO to excessive code growth both with increasing number of components and over time.

A large part of this is glue code, which I like to refer to as the "Dark Matter" of software engineering. It's huge, but largely invisible.

So with glue being the problem, why is it a problem? My contention is that the key problem is that we tend to have fixed/limited kinds of glue available (biggest example: almost everything we compose is composed via call/return, be it procedures, methods or functions). So my proposed solution is to make more kinds of glue available, and make the glue adaptable. In short: allow polymorphic connectors.[6][7][8]

So far, the results are very good, meaning code gets a whole lot simpler, but it's a lot of work, and very difficult to boot because you (well, first: I) have to unlearn almost everything learned so far, because the existing mechanisms are so incredibly entrenched.

[1] http://www.vpri.org

[2] https://blog.ycombinator.com/harc/

[3] http://www.cs.cmu.edu/afs/cs.cmu.edu/project/able/www/paper_...

[4] http://repository.upenn.edu/cgi/viewcontent.cgi?article=1074...

[5] https://www.semanticscholar.org/paper/Programs-Data-Algorith...

[6] http://objective.st

[7] https://www.semanticscholar.org/paper/Polymorphic-identifier...

[8] http://www.hpi.uni-potsdam.de/hirschfeld/publications/media/...

The problem is that tech is always built with hidden assumptions. ALWAYS. Anyone who tells you otherwise is a liar or naive.

Not everyone can work with the assumptions that this tech demands, and not all of the assumptions are apparent (behavioral assumptions especially), so we end up either solving the same problem with a different set of assumptions or using glue code to turn things into a hacky mess that works.

Right, it's definitely the hidden assumptions that are currently killing us.

That's why the process is so difficult, because questioning everything is not just hard, it's also very time-consuming and often doesn't lead to anything. Or, worse, doesn't seem to lead to anything, because you stopped just a little short.

I'm sympathetic to your point of view, but Big Design Up Front never works, in my experience. It's impossible to anticipate future uses for tech that we currently consider vestigial, unnecessary, or inefficient.

Mainframes and terminals were the future until PC displaced them, until the web came along, until mobile and cloud computing smashed it all and brought us full circle (sorta). Designing to optimize the full stack for one of these iterations would have made the transition to the next phase much harder. Fewer people than you'd think foresaw the IoT, FPGA, Arduino, processors-in-everything revolution currently underway back in the early 2000s, when Moore's Law was still gospel.

In the end, I take a sort of Buddhist view toward the fuckedupedness of software: It sucks, but it's all we've got and must be accepted on its own terms. Find a strategy that works for you and that minimizes global pain. Do no harm. Attempting to force your way into Nirvana and out of Samsara only wedges you deeper inside it.

"Big Design Up Front never works"

Seems to work for NASA and others doing way more complex projects than I am .

They have much larger budgets and tighter tolerances than all but a few domains. Nevertheless, their approach is more modular than you think. They do a lot of designing to interfaces, especially where the necessary tech doesn't currently exist.

EDIT: Forgot to mention how little engineering (relatively speaking) NASA does these days. They farm a lot of projects and sub-projects out to contractors, devoting most of their engineering expertise to the requirements phase. Now imagine if your clients spent that kind of time communicating upfront! Software would be a lot better across the board.

And yet people praise SpaceX for doing much better and cheaper with their iterative refinement.

BDUF can work if you have a single organisation with a single stable set of requirements that's reasonably compact. The Kennedy "man on the moon" speech was such an example.

Where it falls down is trying to meet the needs to the 7 billion distinct human individuals, which are inevitably vague and shifting and change in response to publication of software.

SpaceX also doesn't have the track record NASA does, and cannot be compared.

In 50 years when they've been working alongside each other, maybe, but not right now.

NASA has also used iterative refinement in the past.

But the cost is enormous and flexibility nonexistent.

In general, it works if you decide what you want before starting the development. I'm not talking about knowing what you want: I'm talking about deciding what you want. Much like buying a car: no way you can think to all the possible detailed preferences of yours before choosing a new car, but when you sign the check you are voluntarily giving up any further discussion.

And then we end up with industry that continuously rediscovers and rebrands shittier versions of solutions that were already created in the 70s.

The problem with BDUF is not that you can't predict things well in advance. Indeed, you can, as it has been done countless times in the past in our industry. The problem is, doing the Right Thing doesn't make you the first to market, doesn't make your solution the cheapest and the most virally spreading. Hence we end up building towers of shit instead.

> The problem is, doing the Right Thing doesn't make you the first to market, doesn't make your solution the cheapest and the most virally spreading.

There's definitely that, yes. But I'd still contend that we're also crappy at predicting trends and future uses of technology. Yes, some people have predicted things well in advance. Some have done so accurately. Some have done so consistently. But none have been both accurate and consistent. It's maddeningly difficult, even at short time scales, and second-order effects quickly take over. The same applies to even tiny projects.

BDUF just has too much in common with communism: They're both forms of well-intentioned central planning, and they're both symptoms of the hubris that makes each of us think (s)he is more of an expert than we really are. And they both break down when unforeseen forces slam into their base assumptions.

I hear you. I really do. I've BDUF'ed my fair share of systems, and witnessed countless other people do the same. It just never works out. There's a damn good reason that "Worse is Better" keeps winning: It's got serious evolutionary advantages over BDUF.

> This would start from the CPU microarchitecture and instruction set, through the programming interface of the operating system, up to the middleware used by most programs.

Perhaps better to start from the top, and work down to the CPU architecture?

> IMO, we need to fundamentally rethink the design of our computing infrastructure ...

Would you please care to provide an example of what changes you'd like to happen?

Do you find it beneficial to pursue a long term productivity, to write much less code, more reliable and more performant code, that requires much less maintenance, etc.? I feel like you don't. Obviously motivation to do that cannot come from a career perspective, as nothing like that is going to make work easier for you, but maybe there is something else? Just trying to understand.

I hope I didn't give that impression. I find that simpler code is inherently more reliable, more performant, and easier to maintain. I don't try to look too far down the road, because I've been on enough projects to know that they never turn out how anybody thought they would. So I just focus on writing simple code. And simple code is most easily written by sitting down with the client and really understanding their need, paring it down to the absolute minimum viable solution.

As for "career" considerations... well, I haven't had a very traditional career, so I don't put much stock in that. I just spend much less time writing "features" these days, and I've found it's been beneficial to my productivity, client satisfaction, and personal happiness.

Apologies if I came across as disgruntled.

On the one hand I basically agree with what you said on the other: 'I mean, how many human lifetimes have been consumed by the shitty intricacies of dynamic linking alone. '

I guess none, by any normal measure of human lifetimes and consumption thereof.

If you add it all up though? I'd say I've probably spent as many as several person-days dealing with dynamic linking nonsense over my career, let's round that up to a week. Fifty weeks a year, 80 years in a life, if there's just 4000 people like me in the world that's a human lifetime right there.

(No pedants please, if you don't like "fifty weeks a year" then change "80 years" to "76.9 years")

> my time was better spent just solving the problem and then moving on to the next problem.

What do you mean with "solving the problem"? A temporary solution for a temporary problem, a definitive solution for a problem that could be forgotten, or something that had to be maintained? In the last case, moving on and leaving others deal with our mess is not professional at all.

That's not what I meant at all. Instead of building lots of bells and whistles and "features" these days, I tend to build the minimum viable product. The last 10% takes 90% of the effort, and I've found it's usually unnecessary/unused anyway. This frees me up to solve 3-4 problems where I once solved 1 (and 3-4 that nobody actually had).

This is, of course, a gross generalization.

Regarding your last sentence: We all "move on and leave others to deal with our mess" to some extent. It's unhealthy for a developer to set out to fix all the problems in a stack, and it's unrealistic to expect them to leave a tidy solution when they're done.

Iterative, collaborative software development is like democracy: Messy and inefficient and frustrating, and also the only viable way to get things done.