The software industry is going through the “disposable plastic” crisis

I get that “done is better than perfect” but it feels like we’ve swung so hard in that direction that we seem to be largely neglecting stepping back and building the rough roll for the job in all but the largest places who have the luxury of having enough resources to assign a big enough team in the background to do it.

The problem is that this works only for large clients and conservative problem domains (such as accounting) where things change at a glacial pace. Back in the day, only such clients could afford enterprise/custom software, so using a waterfall methodology to develop that software worked well enough. Took five years to deliver the final product? No problem because the set of requirements you collected five years ago and based your software on probably still apply at the time of customer sign-off.

Things have changed. Software has gotten much cheaper to develop, which means it has become way more affordable, and the types of businesses who need that kind of software are in heavy competition with each other and their problems and needs change at breakneck speed. So your best best is to have your own software development practices to match that speed.

Which is in turn driven by a venture capital industry that cares only about massive growth and nothing else.

Safety stuff? Done then right way with the proper attention.

Rest of the production line stuff? Hack on hacks in order to get things online ASAP. At least that's my impression from the outside view I've seen.

If you have nothing to sell you’re done. If your legacy cruft becomes overwhelming you’re done, but likely via a slow death.

Good teams are always struggling with this balance and there is no way to be right, but hopefully you can find that balance and be good enough.

The job right now seems just stitching AWS services together, and spending the rest of your time debugging and putting out fires.

Disregard for technical debt is certainly a force.

All these MBAs and non MBAs and Safe and Agile managers, they are all talk but dont allot time for issues that helps in better code-quality.

Open source is the only place I regularly see high quality code. There the devs are allowed to love their code like pets not cattle.

It might not be your ideal environment to be making products but it is our job to square the circle otherwise we’re not going to have jobs. For a concrete example look at the way game engine development happens. By and large engine development is slow and steady. Whilst making a game is something that massively benefits from blazing iteration speeds. Having that split between technology and product means you can do both.

Of course we’re also dealing with messy reality so things are faster or slower than each team would like at times.

Rapid iteration is driven by customers not actually knowing what they need. There are places where "slow and steady" development would work, but very often you'd just find out what the customer needs isn't what they've asked for over a much longer period. You'd end up throwing out a lot of the perfectly crafted artisanal code anyway just as you would in the rapid dev version. The main difference is that the customer would have run out of money by then, and failed, and you'd need a new job.

As for "and then some" that's the part that's on senior management and HR, for setting up review/promotion structures that actively incentivize sloppy engineering. When all of the rewards are for New Shiny, some people will actively sabotage efforts to spend time any other way. Nobody wants to delay their new-feature effort to deal with new test failures or cleaned-up APIs. Even those who have enough of a conscience to to the right thing get kneecapped at every turn.

I'm not sure I would even know how to build any of my C/C++ projects on Windows.

Absolutely, but you still need to present an interface. Essentially, which interface you present (REST, command line, language-specific library, VM/container, etc.) has in most organizations become a political question, to which implementation details are largely held hostage.

"Pham Nuwen spent years learning to program/explore. Programming went back to the beginning of time. It was a little like the midden out back of his father’s castle. Where the creek had worn that away, ten meters down, there were the crumpled hulks of machines—flying machines, the peasants said — from the great days of Canberra’s original colonial era. But the castle midden was clean and fresh compared to what lay within the Reprise’s local net. There were programs here that had been written five thousand years ago, before Humankind ever left Earth. The wonder of it — the horror of it, Sura said — was that unlike the useless wrecks of Canberra’s past, these programs still worked! And via a million million circuitous threads of inheritance, many of the oldest programs still ran in the bowels of the Qeng Ho system.

Take the Traders’ method of timekeeping. The frame corrections were incredibly complex — and down at the very bottom of it was a little program that ran a counter. Second by second, the Qeng Ho counted from the instant that a human had first set foot on Old Earth’s moon. But if you looked at it still more closely… the starting instant was actually about fifteen million seconds later, the 0-second of one of Humankind’s first computer operating systems.

So behind all the top-level interfaces was layer under layer of support. Some of that software had been designed for wildly different situations. Every so often, the inconsistencies caused fatal accidents. Despite the romance of spaceflight, the most common accidents were simply caused by ancient, misused programs finally getting their revenge.

“We should rewrite it all,” said Pham.

“It’s been done,” said Sura, not looking up. She was preparing to go off-Watch, and had spent the last four days trying to root a problem out of the coldsleep automation.

“It’s been tried,” corrected Bret, just back from the freezers. “But even the top levels of fleet system code are enormous. You and a thousand of your friends would have to work for a century or so to reproduce it.” Trinli grinned evilly. “And guess what — even if you did, by the time you finished, you’d have your own set of inconsistencies. And you still wouldn’t be consistent with all the applications that might be needed now and then.”

Sura gave up on her debugging for the moment. “The word for all this is ‘mature programming environment.’ Basically, when hardware performance has been pushed to its final limit, and programmmers have had several centuries to code, you reach a point where there is far more significant code than can be rationalized. The best you can do is understand the overall layering, and know how to search for the oddball tool that may come in handy ...”"

To name just a few off the top of my head, where there is no choice in whether the userland changes or not:

- Cipher suites and crypto protocols changing often end up "cutting off" older applications because they used an embedded SSL library that can't do TLS 1.2. This is required because vulnerabilities are discovered in older protocols and they must be turned off.

- Changes in the CPU instruction sets. For example, it is not possible to run 16-bit Windows binaries in a 64-bit host OS, but there are no 32-bit supported versions of Windows. Even if the majority of the code is 32-bit, a tiny 16-bit component can be a showstopper. I had to support a large (20K concurrent user!) environment that ran a 16-bit DOS application. On Windows 2008 32-bit. In the year 2015. Really.

- I've seen apps that just can't handle high performance CPUs or multiple hardware cores, simply because they were written in an 100MHz single-core era. I've seen all sorts of timing issues, deadlocks, etc. For example, one particular release of World of Warcraft simply couldn't patch the game on 4-core CPUs, because one task would race ahead, time out, and then deadlock the downloader utility. That persisted for about a year, because Blizzard thought it was "okay" to support only 99% of their userbase that had 2-core processors that were common at the time.

- If you app has any external dependencies of any kind, its days are numbered. License servers. Update checkers. Content providers. Cloud services of any kind. Whatever. It's going to be turned off eventually, and the app will stop dead.

I think you're oversimplifing things here. There are many reasons why programs did run on the old OS, but no longer work on the new that are not "new OS broke userland". Some of these boil down to inaccurate or short-term assumptions of the developer. Not exactly userland, but this story [1] why old Windows versions crash on fast processors is an especially entertaining read.

The sibling comment by jiggawatts provides good examples.

[1] http://www.os2museum.com/wp/those-win9x-crashes-on-fast-mach...

Alfred Borden : I had a similar trick in my act, and, uh... I used a double.

Gerald Root: Mm, I see, very good.

Borden: Well, it was, and then it went bad. What I didn't count on was that, when I incorporated this bloke into my act, he had complete power over me.

Root : Complete power, you say?

------------

I think about this quote often when I think if dependencies.

For old software that's not games, it's actually more common than not, because the OS really didn't break the userland contract all that much - what it broke is various undocumented assumptions that software was relying on. Games tended to be the kind of software that did it the most, usually in the name of performance (and sometimes also DRM).

Although if you unwind to before Windows XP, you also have to consider the Win9x/NT divide. Most games of that era were written for Win9x, and often relied on the liberties offered by the very lax process and memory management model. The transition from 9x to NT - which for most consumers happened via XP - was definitely a big userland breaking change for anything that didn't care about NT compatibility before.

(That's why the sweet spot for back-compat is circa 2005 - XP was already well-established, so all new software was written with it in mind - but technologies used were still of the older variety that doesn't drastically change every two years.)

   > Developers wanting the last shinny programming
   > language instead of mastering one.

But those bugfixes made the old code difficult to work with, and rather than take the time to understand it, new code is developed instead with a whole new set of bugs (many having been solved previously). Working with old code is unexciting and unsexy anyway, and it is unlikely to advance your career. And so the cycle continues.

Electron apps contain a local web server. (That's part of the reason they're such resource hogs.) In principle at least they work offline. Github's desktop app, for example, works fine on a disconnected machine. You obviously cannot push or pull, but you can do local git operations with the GUI.

I think more so what made me stop pirating software altogether, and instead either paying for it, or using FOSS alternatives, was in part the realization that it was going to be a bit silly for me to want to get into software development and wanting to get paid for my own software, while at the same time continuing to pirate software. So that is the main reason I stopped pirating software I think. That and the fact that I learned about a lot of FOSS when I started at the university.

Still, I don't like the reality because it obscures the bad parts of where we are.

I've seen a slew of software licencing models in my travels. I hate where we have wound up.

Some of my more-appreciated ones:

- Not-Crappy-Hardware-Lock: - When I was in the RF/HFC Drafting industry, a very entrenched program was utilized for most signal level calculations. This company used a solution that involved a Parallel or USB Device that took a Vendor-provided Dallas one-wire device. You paid X$ for the software, you had updates up to X date with that key. They did have a way to 'upgrade' but I don't remember what that involved as we never did it. The system seemed fair, I'm guessing it was fairly PITA to hack (remember, in this industry you have access to a decent number of people with good signal analysis equipment) but also had a fair cost standpoint. - It was nice that "Who can use it" was just tossing a key around.

- Usage + Audit: - Same industry. Market forces led us to also have a product from a vendor that also competed with the biggest CAD vendor out there. This company essentially had a multi-stage setup; The systems running the software would check in with our locally-installed license server running on a box, license server would periodically check in with vendor's home-base server (uploading our usage data and downloading some response, I'd assume). If there were any issues with our usage the vendor would call us and inquire as to what was up. - If your machine was not going to be able to check in to the license server (i.e. Field use, this was also before mobile hotspots were cheap, and not too many corps had proper VPN setups) you could 'check-out' a license and generate a time-expiring key - They were WAY WAY more than fair. Our usage was balance out based on some algo on their end. IDK quite how it worked, but for example; if you had 3 licenses, didn't use any Monday or Tuesday, 2 on wednesday, but then had 4 people simultaneously using on Thursday/Friday, they didn't care. - When somebody who may have been me left the app up for a month and had a checked out license for a laptop, while we were in crunch mode and others were using the application, we eventually did get a phone call. But even during the conversation they discussed the amount of logged time and it didn't result in any penalties or fees. - We got updates as long as we paid a fairly low maintenance fee. If we stopped paying updates, we only could use up to a certain version. - Seriously, maintainers of the DGN format, you guys were amazing about licensing and just great to deal with.

- Per-Seat licenses: - Arguably the most fair - You know what you're getting - Can do either with a central reg or reporting server - Possibly more balanced than above towards vendor, definitely lower maintenance level.

- Jetbrains style licensing: - Similar to Usage + Audit, but only the usage part - Works well. I dig it. - But seriously, what is up with the awkward 2-factor setup?

- SAAS: - How would you like to be charged today? - Vendor Lock-in - Change Friction - Don't own the hardware - Offloading of liability

Unfortunately, SAAS wins out because it's the best way to make money, especially when one considers that last bullet point. But also, when you consider 'base' charges versus 'overage'; as an industry we might be diving head-first into the same malarkey that we have complained about in places like this, or forums, or even usenet groups; the forced minimum pricing/bundling of services.

What's ironic here (for me,anyway) is that Cloud computing used to be different; you used to just be able to buy a hosted server. Somehow we jumped from that straight to the current model instead of more consumer-friendly concept of burstable VMs.

Its perfectly understandable why this happens but its a shame there is no way at all to play this game other than finding an old ipad that still has it installed where as old windows games still work fine.

Example: You can still buy a copy of Visual Basic 6 on eBay. Does your laptop even have an optical drive to install it? Do you have the patches that were released for it?

https://snapshot.debian.org/ https://softwareheritage.org/

> One, you can preserve the binary and run it with emulation. For webapps, this means preserving a deployable image. Your faddish/brittle dependencies are 100% irrelevant, because your image includes them already, right?

And what if the software's build artifacts aren't binaries or deployable images? Many projects (most, I suspect) don't output either.

> Or two, you can preserve the source code. Which for any project is going to involve a mess of complicated tooling that dates to when the project used to be compiled.

Well sure, but some projects have more mess than others. That's kind of the point of the OP.

> But if you're intending to preserve something for the long term, you should preserve the dependencies too -- e.g. node modules or whatever.

And if you made the decision to preserve the software long after it was initially developed, what then? Most people explicitly don't put their node_modules folder in version control.

> I don't really see what's all that different.

What's arguably different is that many of the currently popular ecosystems don't really value long-term maintainability, stability, or preservation to the same extent as the popular ecosystems of yesteryear did. Projects in these ecosystems can end up broken on a much shorter timeline than in other ecosystems. Project breakage also has a way of causing cascading problems.

I'm not sure why you're confused, you seem to have a good grasp on the solutions to these problems. Perhaps you're unaware that it's common practice not to implement those solutions ahead of time, and that is exactly what the OP is complaining about?

I can't count the number of apps in these languages where I've had to fiddle with downloading updating, compiling libraries.

Java and C# are great about this. Download it, run some command, and you're good to go 99% of the time.

https://news.ycombinator.com/item?id=24027663

What did you use to visit this website? Check your email? Open a spreadsheet last? If your a software developer, what do you edit code in? Your operating system?

Hard to believe you're using apps less than a few years old for any of those much less all of them - or even those that are by name older but have been through a Big Rewrite in the past few years.

The problem is that working old code often has far more corner cases and bug fixes in it than you think. It's easy to "replace" it with something that is buzzword-compliant and that uses the new shiny, but that doesn't cover all the cases and doesn't have all the needed bug fixes. That's not a step forward.

And yes, I really want to be using 20 year old software. I have software dating back to some originally written in the 1960s, updated in the 1970s, ported to the PC during the 1980s-early 1990s, which still runs fine today. Some of the programs require an emulator or virtual machine, but they still run fine. With a fair amount of confidence, I can modify and recompile the source code and it will still run. (I did this recently to add JSON output to a program written in 1989-1990.)

The problem is dependencies. Originally they were compiled in or included. But then people got all wonky about code re-use and shared libraries led us into DLL hell. Then we had things like OCX and VBX components that were shared and then vanished. Much of that is patched-over now to avoid some of those problems, and for things from that era, a VM typically runs them.

But more recently dependencies are much greater, and changing much faster, and much more often rely on network connections to specific endpoints which may not be there tomorrow. You can fairly readily pack some DLLs into a VM and run stuff from there consistently, but you can't pack a working snapshot of the entire internet into it.

That's aside from how much more complex the build systems and their dependencies have become. I know for a fact that I could still easily build and run code I wrote >15 years ago today, but not code that I wrote 10 years ago. Not all of the things necessary to build it are still available.

Admittedly systems like unemployment that are in daily use for business purposes probably should be updated. But if I want to play the original Fallout, released in 1997, I should still be able to. And in fact I can. Newer games might not last 5 years. If I want to go back and look at/adjust some vector images I made in the 1990s-2000s, I can do that in a VM, using the source files in the software used at the time, and generate new output files. With today's software, I won't be able to in the future.

Changing code (or UI or UX or anything really) just for the sake of changing it is just nonsense busywork to me.

Yes. When a piece of software is working the way I want, I don't want to change it. I want to keep it indefinitely. Aren't there games from 20 years ago you'd like to play again?

When a developer makes improvements, of course I want those updates. Keep me up to date automatically. But let me opt-out. Let me revert. Here's my dream: I'd be able to rewind my whole operating system back 10, 20, 30 years and be able to run and use everything from back then, exactly how it was.

It’s getting harder now with 64bit windows not supporting 16bit apps — they are living on a “legacy” Linux VMs tenant in Azure now, and probably will continue until around 2030. Possibly longer, as they are on emulated DOS now!

https://archive.org/details/softwarelibrary

You literally click a button in your browser and emulate right there. Have fun.

The only one I can think of offhand is Harpoon II Admiral's Edition, which had its own built-in memory manager that doesn't play well even with virtual machines. But that is one where they did eventually re-release an updated version that runs on modern systems.

Edit: The original release was 1989, Admiral's Edition was 1996. I don't know if they did those memory tricks in the original, but it's doubtful.

Before that, given the relevant files on a disk, you could rebuild the software. And even on later systems, given a sandboxed VM or emulator, you could run it. After that, neither is guaranteed or even expected.

Unless you also have a fully-functioning snapshot of the entire internet from that point in time also on your disk/VM.