Man spends entire career mastering crappy codebase | Hacker News Reader

Man spends entire career mastering crappy codebase

taylor.town

866 pointssurprisetalk570 comments

I swear this guy exists at every company I've ever worked.

He's the guy you go to when you find some legacy code which you have no idea how the hell it works and end up getting a 2 hour history lesson into a decade of company politics and failed replatforming projects.

They're hard not to love.

"... And the AIX machines were a real beast. We had 20 of those things, and each had its own 20 amp circuit. Had a tendency for the grounding to the steel case to fail. If you wanted to upgrade it you'd first need to grab a pair of thick leather gloves...."

I for one do not mind that part of computing getting abstracted away by the cloud, whatever its other failings. :-)

Working on the hardware directly is more than half of the fun.

An old IBM tech told me that hard drives failed when power cycling equipment could be temporarily resuscitated by placing the drive on the floor, twisting it hard by hand, then as quickly as possible connecting it back into the machine and powering on. Apparently getting the platters to spin helped them get around a worn out motor that couldn't make the initial spin-up.

I would believe them. I have heard a similar story. A data center had to go through a planned power cycle event.

All systems were working when they shut down, but ~40% of the disks failed to spin up due to worn down motors.

That's a lot of disks.

What’s a procedure to avoid it? Reboot machine once a week so disks fail not all at once?

Gotta know which way to twist it.

Yes, stiction is a real issue in old hard drives. In fact something that used to work well if you were cold booting an old server even in the mid 90s, with the expectations of an immediate migration, was giving the drives a good sharp slap to help break stiction so they could spin up. The only issue is it could cause a head crash. Another fun trick from back in the day was sticking hard drives in the freezer to cause the platters to shrink enough to dislodge the head and restore the air gap if you had a head crash so you could try to recover the data.

As much as I love hardware, I much prefer our current solid state wonderland.

> to shrink enough to dislodge the head and restore the air gap

Huh?

What I heard is what a cold drive would be a bit more magnetically stable and IMMSMV that surely worked, because I never had a drive with a stuck heads but I did had drives what would just abort the read or return gibberish, but after an hour in the freezer they would read just fine, till they heat up again. That's for the drives mfg after 2000, if that matters.

My understanding may be incorrect, after all the freezing the drives trick is mostly something that was shared among sysadmins like an old wive's tale. Nonetheless, it does work (or did).

The way I had always understood it was that if you had a head crash (which was caused by the head physically contacting the platter, overcoming the resistance of the air gap between the head and platter, usually due to physical impact) that the magnetism of the head would prevent it from lifting back up on its own, and that freezing the drive would cause the metal in the platter to contract away from the head, which was restricted in movement by its armature, thereby restoring the air gap and lifting the head away. If you started the drive spinning before it heated, the head would stay out of contact and you could successfully read data (some of it, for awhile).

That said, my understanding may be incorrect.

I remember upgrading from a 4.3 GB HD to a 20 GB HD around 2001. I had the old hard drive sitting on my desk, and the new hard drive sitting in the case, both attached to the motherboard via IDE ribbon cables.

Most of the way through backing up the 4.3 GB HD to the 20 GB HD, I heard a screech and the old hard drive twisted slightly on the desk... conservation of linear momentum. The drive was visible to the BIOS, but refused to spin up after that.

I put the hard drive in a ziplock bag, put it in the freezer to give the parts slightly more clearance from thermal contraction, and slammed the hard drive nice and hard on the desktop to free the stiction. That revived the drive long enough to finish my copy.

I did have backups of all of the really important information, but restoring from a stack of floppies is more tedious, slower, and less fun than percussive maintenance on a hard drive.

Back when I had an internship working on MEMS gyros for GPS guided mortar rounds, we had to sometimes perform similar percussive maintenance if static electricity had caused the moving parts to contact the substrate. We took the gyro, and smacked it hard on the desk in an "eyeballs out" orientation to give it something like 10 to 100 Gs of acceleration in an attempt to un-stick the MEMS gyro.

Yeah, did same in 1999 or 2000, but we _did_ _not_ take the HDD from the freezer, just pulled the ribbon and power cable through the narrow slit between the door's rubber gasket and the fridge's body.

Clearly, I meant conservation of rotational momentum, not conservation of linear momentum.

> Working on the hardware directly is more than half of the fun.

It never really worked until you gave blood to the computing gods on a sharp edge of a metal case.

Looks to hands and fingers intently

Hmm, yes, true.

I have 6 stitches in my left hand from such an offering.

Unless it was a recent incident, you should probably see a doctor about getting those stitches removed.

"Hey, look, I have all five fingers! Three on one hand and two on the other!"

I feel like the general improvement in fit & finish of cases and chassis is overlooked.

I, for one, haven't flayed open a knuckle on a random razor edge or burr on the inside of a case in like 15 years.

Every Friday I had to take a bunch of magnetic tapes to the bank so they could look after our backups ( which was a good excuse to pop into the pub next door to the bank...) Oh, and we never ever tested that the backups actually worked

My first job out of school was engineering consulting, writing software models of pre-production military radio hardware. The client was a military contractor, and they paid a crazy sum in order to have 3 DAT tapes: one tape always in the drive used to make the daily backups of the shared working directory, one tape always stored in a bomb-proof bunker, and one tape potentially in-transit via courier. I think it was once a month that the client's sysadmins rotated in the "unused" tape, sent the latest tape to the bunker, and waited for the old tape from the bunker to return as the "unused" tape.

Due to absurd government requirements, we were not allowed to use any version control software other than PVCS, and PVCS could only be used for storing official releases of the models. So, the last person in the office on Friday needed to make a dated zipfile of the shared project working directory.

So, one day I convinced the project manager to just let me rename our working directory, create a new empty one in its place, and tell our sysadmins that I accidentally deleted the contents of our working directory. It turns out that the client's sysadmins hadn't set up anything to expunge the oldest backup and had dutifully ignored the alarms that the tape was full. So, the latest available backup was from about a week after they last changed tapes!

The senior sysadmin prominently had a sign on his cubicle reading "Programmers are the ditch diggers of the 21st century." Those same sysadmins were supposed to keep the server room door open while I reinstalled Solaris on a Sunfire V1280 via serial connection from my laptop, because there were cables carrying TSSCI (Top Secret - Secret Compartmentalized Information) traffic. However, the V1280 consumed so much power that the server-room was under-cooled, so the sysadmins illegally shut the door to keep their cubicles cooler. They had top-secret clearances. They knew their obligations to keep me away from those TSSCI cables. Those lazy lazy arrogant sysadmins.

Anyway, unless you have recently passed a restore-from-backup drill, you don't really have backups. Even if you pay absurd amounts of money for sysadmins with top-secret clearances and absurd amounts of money to store your backups in a bomb-proof bunker.

You kids sitting there. All you do is change your EC2 instance size and restart it. Back in my day I had to go and install the RAM by hand. And you know 128Mb of RAM then took up a whole suitcase and weighed more than your laptop

The I fall asleep at my desk, periodically waking up shouting "STOP-A!" and hallucinating about being bitten by thick ethernet vampires.

Going to make myself a coffin...

"I had to go and install the RAM by hand"

When the Sun 4/330 I used to have needed a RAM upgrade someone flew up from London to Edinburgh with the RAM and installed it. Fortunately we were in the Grassmarket about 10 minutes from the Castle so he had plenty of things for him to do to keep him amused until his return flight.

Edit: I checked and the max the 4/330 could take was 96MB so I suspect the upgrade was an awesome 64MB. That machine cost more than most cars I have owned - even without allowing for inflation.

Yeah crazy money. My father used to import RAM when it was really really expensive in the 1980s. He used to take it on planes from Taiwan as hand luggage handcuffed to himself because it was that valuable.

I worked in a shop in the early 90s with a bunch of Suns, and we had hired this new sysadmin. One day we noticed that half the RAM was missing from a workstation, and few minutes later noticed half the RAM was missing from all the workstations. Turned out the new sysadmin was a cokehead and had stolen it.

Oh man that's funny. Only because something similar happened to us. Hired new cleaners circa 2001. Got into the office one morning and all the PC cases were open. All the RAM and CPUs had been stolen.

In the mid-90s, when CPU heatsinks didn't need to be screwed in, thieves discovered that the new ZIF/zero-insertion-force sockets also make good zero-removal-force sockets.

When I was in Korea in the 90s you could get LG RAM for about half what it would cost in the US. I had a scheme to buy a bunch of it to sell when I came home, but in retrospect it would have been the nerdiest case of "guy goes door to door trying to sell rapidly expiring meat".

This brings back memories. I got my start in parts wholesale selling 30pin SIMMs in static baggies out of my leather jacket in a Taco Bell LMFAO. It took some convincing to convince the management I wasn’t selling drugs, but the getup helped me to not be noticed by people that might otherwise know that I was carrying thousands of dollars in cash and chips daily.

I would sit there and wait for RAM hungry customers that would call me on my 3 lb cellphone lol. Each one has to buy at least a $0.39 taco so I wouldn’t get kicked out by the manager. They had armed security, who I tipped 20 dollars a day. I hung fliers all over town with tear-off tabs.

Then I went to COMDEX, it changed my life. Within 2 years I was in my early 20s and moving 2-3 million in hardware a year. It was a heady time.

Then, the business dried up overnight with online sales.

Fortunately I saw that coming and had moved into technical services and contracting, and sold the parts wholesale business to one of my competitors lol. The local wholesale distribution biz was dead within 8 months.

Hahaha hooky RAM dealer. Much respect though for being there at the start. I used to do something similar with pirated Amiga games but that was a limited market!

At first, it was an opportunistic grift by a teenager looking for party money.

As prices fell rapidly at the supply level, local box stores would occasionally sell off stock at deep discounts (reflecting the real market price). Being a teenage sociopath, I would buy every last bit of it in town and then coast on the uninformed price perception for a week or two.

Prices were falling so fast at that time that the stores were always wrong footed, so I was able to keep up that racket long enough to get to the bottom of that mechanism and start ordering in bulk directly from city of industry California.

Place was called Ma labs, iirc. After that it was obvious to start moving disk drives, video cards, processors, and later everything else too. I was just exploiting basic market inefficiencies , since the only other sources were the big box stores that were constantly 2-4 months behind the price curve.

Then COMDEX happened. It got me tied directly in to motherboard and peripheral card manufacturers. I also got introduced to the cocaine and extremely friendly booth-girl shmoozefest, and NGL for naive Alaskan me it was heady stuff. The real deals and the points negotiations were made in the afterparties with piles of coke and very persuasive “account representatives”. Memorable times were had in the hedonistic environment of an exploding industry.

At that time there were a lot of small manufacturers in industrial parks. They were usually pretty small outfits usually just a shop with a few hot air stations for warranty work and a wave soldering table, 15-20 employees at jet labs, for example, which was one of our big suppliers of motherboards, video cards, modems, lan, and other peripheral cards.

My activity festered a plague of tiny pc builders selling systems, and launched the careers of at least 50 technicians and small businesses. We used to put on big public LAN parties and public PC repair clinics as promotions for our “dealers”. I worked with a training/testing outfit to get new dealers A+ certification to reduce the damage to society and actually grew into a semi responsible citizen during that time.

Fond memories but I’m glad to leave that hustle behind.

After that, I unwittingly became a software developer for a major criminal organisation. I thought they were a car dealership, but gradually it became apparent as I built out their backend that the dealership was just a small part of a very large laundromat.

It took me years to extricate myself from that and for years afterwards they would give me gentle reminders that they were keeping an eye on me. Free 500 dollar an hour lawyer’s though. They never let me get near a courtroom or a police interaction without “proper” representation showing up even for years afterwards (statute of limitations?)

The fact that they always knew when I brushed up against the legal system in any small way, even traffic tickets in rental cars, gave me great respect for the scope of their concern. That and their obvious love for mid range turbine helicopters.

I was probably lucky to leave that alive.

I’d watch that movie.

I went to the closing sale of a local computer store that had been in business since XT days.

Owner lamented finding 4MB sticks of RAM that had fallen behind the workstations that had cost hundreds of dollars back in the day, as he was selling (and this was a while ago) laptops with 1-2GB of memory).

I worked in some ex-USSR government facility, and they had an old hp-9000 system, that needed ram upgrade. HP wanted some ridiculous thousands of $$ for like 16MiB of RAM. After some inspection it was discovered that the RAM is nothing more than 72pin SIMMs. We took some ram from nearby old 486 and it worked.

> HP wanted some ridiculous thousands of $$ for like 16MiB of RAM. After some inspection it was discovered that the RAM is nothing more than 72pin SIMMs.

Nothing’s changed on that front.

Sorry, could you tell me what "STOP-A" means?

Stop-A is the key combo you hit on Sun workstations to drop into the ROM monitor. Usually when they are locked up solid with a crashed Xserver and need a reboot.

into bootprom forth interpreter though.

"Back in my day, we only got one compile a day with our punch cards. Now you can compile willy nilly without having to put in the forethought we did" - ex coworker

> Back in my day, we only got one compile a day with our punch cards

You were lucky! In my computer class at school in 1979 we got one compile a week. Punched into cards which were taken to the local insurance company who actually had a computer.

Continuing with the Monty Python vibe, the first code I wrote was in machine code (ie hex, not even assembler). Though I worked with a guy who wrote machine code on a magnetic drum and they used the delay of the rotation of the drum for their timing. The good ol' days.

My thumbs still have blisters from manually stuffing memory expansion boards with 1Mb of DIP RAM for our 80286 PCs. We were doing image capture and processing, long before it either usable or useful.

I did a real internship (as in it was for my diploma and lasted only a few months) in a medical facility operating Big Machines. I remember a guy soldering a new battery to some old DEC/Digital hardware.

"They were afraid to turn off or reboot the DEC Vax for fear that if the disks stopped spinning they wouldn't start again. I saw it once at the retirement ceremony and it didn't look like a server but more like a set of industrial washer and driers. The users transferred files using the Kermit protocol and printed locally by a telnet client that could understand the special characters that redirected character output from the terminal to a lineprinter."

Maybe we're thinking of two different archetypes, but I wonder if their "hard not to love" nature is part of the survivorship bias of why they're still there.

A former coworker of mine claims that he lasted over forty years in the company out of sheer stubbornness. He might still be there.

Easy to love?

"Hard not to X" is not the same as "Easy to X". It may be easy to do something but also easy not to do it. It would be easy for me to yell "Elephant!" right now, for example, but also easy not to do it.

They are hard not to love.

They also hate each and every one of you.

This has not been my experience at all

They've hidden it well

Or at the very least: incredibly suspicious of every one of you, and very unimpressed with the latest fad in software development.

Not necessarily hate...there's always the 1-2% of new hires that are highly impressive and gives us hope for the future. They tend not to last too long. The rest seems hell bent on never even cracking open documentation, and keep asking "where are the architecture videos?" (at least where I work).

I think they're called senior devs who've been there longer than you.

They also tend to be brilliant engineers. It’s a heck of a lot easier to always be writing 1.0 of something. It’s a hellaciously difficult engineering task to improve a system that’s been improved upon for decades. These folks should be admired and respected, they’re by far the most crucial people at any company older than 30 years old. Instead we get condescending garbage like the linked article.

I just wish hiring managers saw it the same way. It's hard to explain how your last major project was a web enhancement that would have taken a couple days on a virgin codebase, because for you it involved a month of excavating through multiple strata of code dating nearly two decades old, spread across three codebases, two backend languages, a code generator which no one working there understands anymore, and a library who's documentation now only exists on the wayback machine.

A secret I’ve learned is you only need one job, so you don’t need to take just any job or be hireable at every job. It might take longer, but finding a manager that “gets it” is very important. The most important thing is to find someone who recognizes further that a team is more than the sum of its parts, that each individual is unique and not a cog to be evaluated on a metric rubric and forced into compliance with some idealized engineer. They do exist - I was one for many many years. I’ve recently given up in disgust though and switched back to IC, but senior enough I can shape how managers manage engineers. Regardless, my advice is, if it’s helpful, find the right job with the right team and the right company. It’ll take longer, may pay less at first, but finding the right place that respects work at its value not at its marketability is key to any successful career.

Or a library who's code now only exists on the wayback machine... and of the modified version that you actually use only a single copy exists on an old machine that one of the devs in the company never bothered to hand in for recycling...

Never mind that it's part of a decade old contract stipulating a quarter century support...

Greenfield is easy. :)

I wish people like this would architectural decision records so people inheriting it later wouldn't be left in the dark by the decisions made in the system

Because sometimes on the outside systems look confusingly written, but it could be a reflection of the scope of confusing business rules to begin with that they had to scope out

What I propose wherever I go is writing a deprecation plan before production release. It may not be kept up to date over the lifetime but at least a plan for what it would look like to exit to another platform exists documenting assumptions and one way doors, why things were done, and what a replacement would look like and why. I’ve never had great success though, people are too eager to make the next 1.0.

It's funny, startups turn over so fast that I've managed to become that guy while staying at a company less than two years.

Hard to be anything else if the entire job is spinning up a yet another microservice and then spending most of your time debugging in production because shit keeps breaking all the time.

Oh good, we’re not the only ones. The rest of my time is spent saying I can’t help people fix problems with some old microservice my team “supports”

You'll have that. I've always said those places should keep an anthropologist on staff.

Damnit that's me at the moment. I need a new job.

...these OS/2 machines were once interconnected by a networking technology called 'token ring'.. no joke. It was basically an entire network where if one single node on the network misbehaved it would bring down the entire network - they didn't self recover either! you had to reboot them all. I remember once when I misconfigured a token ring card on a PC and it brought down the mainframe! It shut down the bank for 30 minutes"

To be fair, in those days you could sink Ethernet (broadcast hubs, no switches!) with a faulty or malicious PC spamming the network, too.

Didn’t have to reboot the others, though.

> in those days you could sink Ethernet with a faulty PC

That is still possible in the modern era.

I used to work in a place where every office on the floor was connected to the same ethernet hub. We also tested a server product that became a DHCP server if you checked the wrong box, so periodically everything would stop working until the team lead came out of his den and went to yell at the new guy.

step one: disable spanning tree, because the LAN is so big it takes half an hour for a port to come up after plugging something in

step two: plug a very cheap switch into the network

step three: have someone move their desk around, and plug a spare cable into the switch at both ends

step four: spend a day doing a binary search by unplugging banks of switches to figure out where the heck all these packets are coming from.

What do you mean hubs? Back in the day we manually pierced thick Ethernet coax to get access; of course you could ruin the whole segment running tens or hundreds of meters if you didn't take care.

I’ve seen a few of those. Some are worth their weight in gold. Most are set in their ways from the trauma of trying to make everything not fall apart for so long. Almost all have stagnated and are incapable of bringing in new ideas. Almost all are husks of human beings from putting up and personalizing so much of company dysfunction

> incapable of bringing in new ideas

But they are REALLY capable of detecting bad ideas.

They always say no, and are always correct.

I mean I’ve literally had people like this fight adding in a version control system.

How can we be sure we can trust this "git" thing? Besides, CVS already does everything we need.

The CVS receipts are too long though.

A real life application of Odysseus's Oar. I have mentored junior devs who have "heard" of SVN and also think CVS is just a pharmacy. It was a real gut-punch the first time.

This reminds me, in a sad way, of a friend of mine who stayed on when a startup we worked at was acquired by a huge corporation. He has an amazing mind for technical detail, and at the startup, he planned and executed some hugely ambitious technical initiatives. But management at MegaCorp treat him as an obstructionist. They want to announce an initiative, launch it, declare it done, make up a big dollar figure for the "savings," and gather a promotion. They don't need the initiative to make sense or work in a technical sense. But he understands the immensely complicated legacy context, all the work required to accomplish anything, and all the things that will break if they leave out parts of the work, and he won't keep his mouth shut. He would love the challenge of running a big ambitious project, but the only way to get anything big approved is to drastically underestimate the effort, so he's stuck rearranging deck chairs until they inevitably eliminate the product and lay him off.

It's hard to walk that line.

There are really good or valid ideas that people try to implement in terrible ways.

things like: for our app on windows, we just ship LIBRARY.DLL with it. We should do the same with linux. (ldd shows 50 inter-dependent libraries)

This is actually something I've identified as a major con of working at startups (there are a lot of pros too, and I'm at a startup right now that I love): It is impossible to have these people, because the company is just too young. And also I think this kind of person doesn't usually join a startup for its first, like, decade.

I recognize the parody about me. ;)

Do you know why I am giving you the two hour history?

First, I recognize that I need human interaction, but do not really like people.

Second, I am hoping that you will learn & understand instead of just memorize & regurgitate so in the future you can resolve it yourself instead of bothering me.

Now, go away; I had my week's fill of socializing.

/joking - or am I?

> Second, I am hoping that you will learn & understand instead of just memorize & regurgitate so in the future you can resolve it yourself instead of bothering me.

Where does all this hope come from? I’m barely 15 years into my career and I’ve nearly given up…

Hope is dangerous, for it's itself very hard to resist.

I've started to attempt to gently grind that into juniors. After a certain point, hope is a vice you cannot afford. It is simply something you outgrow, if only because of the unceasing retirement from the mortal coil of those that can bail you out.

Hope is for the young and inexperienced. Everytime I come to your rescue, you'd best pay attention. I have an expiration date.

Give them a break! It's an advanced lesson.

For me, it's usually trying to rapidly justify the existence of whatever Rube Goldberg machine I'm having to explain. There's always reasons behind the madness, even if they're not particularly good ones.

> justify the existence of whatever Rube Goldberg machine

I'm at least going to be waiting for questions to be asked.

"Why was it done this way?"

"Circumstance combined with 'path of least resistance'."

Ha ha but really.

I’m not trying to explain why you shouldn’t change it. I’m trying to help you understand what competing forces made it that way in the first place so you can decide if any of that’s still relevant.

> They're hard not to love.

I often search for a clearcut answer to a technical question and I'm met with a 2 hour history lesson into a decade of company politics and failed replatforming projects.

Yeah, thanks for telling me why John from accounting was a dick 10 years ago and you had to code this module in a certain way. I really don't care. I'm new to the codebase and I just want to know how it (the codebase) works.

I'm currently in this situation and a colleage never gives straight answers to anything. It's always some little rant about something and when it's done, I still haven't got my answer.

Easier said than done, but try telling them that. "I appreciate you sharing context, but it's too much to take in at once. It would work better for me to get a more clear-cut answer."

> I'm new to the codebase and I just want to know how it (the codebase) works.

You should want to know _why_ it works that way too if you want to do any meaningful work with it. Context matters. I’ve seen many cases where the way something works seems dumb, only to learn later they had already tried the “smart” way but ran into some obscure problem which the “dumb” way solves.

This is important. As soon as we come to the understanding that the coders that came before were not all idiots then we are forced to ask the "why". There is generally a decent reason why something is coded the way it is. Could be as simple as it was an emergency and meant to go back and fix it but never had the time or it could be a valid business edge case that absolutely had to be wedged in. There is almost always a why. Otherwise you tell everyone about a fix you made that 3x performance and you see their faces go gray as they explain you just shut down some archaic but mandatory process in Singapore.

The assumption that everyone that came before was an idiot is really frustrating...and yet I find myself doing it too.

Generally speaking to that person helps, although sometimes they do turn out to actually be an idiot.

When I realize I'm writing some code that seems dumb/overly complex on the surface, I leave a comment explaining why it was done that way.

It probably depends on the person, but I personally enjoy such tales. Gives me more context about why a piece of code was written the way it is now. But then again, I also enjoy scrolling through the commit history of a repo like it's an archaeological dig site, so I might be the weird one here ahaha.

It's full of "fix" or "fix typo" or "update blah.c"... tiny commits straight to the master branch without CI.

Anyway, I really don't care about the past in this way. A simple "accounting needs this for that" is enough for me. No need to explain that Adam was getting divorced at the time, so he was grumpy, and, and, and...

Sometimes, the 2 hour story isn't worth the time it's told. Sometimes, the 2 hour story gives you the insight necessary to satisfy Chesterton's Fence. It seems like in this instance, that's not the case, but I'd definitely encourage you not to dismiss stories in general because you don't think you need the history to achieve your goal.

That attitude gives me “I don’t know how my code works, I copied it from the Internet and it does compile!” creeps.

Because I don't want to hear about personal stories? How is that related? I do very much want to know how code works. It's the very core of my argument.

And this propagates the problem. Getting an answer without understanding. Sometimes the history is necessary in order to not repeat botched attempts at "fixing" something that people have already tried.

This is where another one of your Pied Piper boxes would go. Okay. Let me show you the next location where we would install one of your Pied Piper boxes. ... okay

It doesn't seem like a bad way to go. If the company is stable enough and you can cash in your engineers salary for 25 years and have crazy good job security.

Seems better than being a 50 year old trying to wow a 28 year old interviewer with your skills on whatever hot new framework just came out.

If I’m still building webshit SaaS products at 50 then something has gone wrong. I should be either retired or some kind of high level manager.

Just one?

They are worth their weight in gold, truly. I had the pleasure of interacting with one such man at my previous company weekly for an extended period of time while planning/designing a rewrite. Listening, thinking, and questioning his stories and perspectives allowed me to formulate problems and architecture in a more first-principles, fundamental way and I believe that experience really helped me mature as a young engineer (as well as the project).

I usually give the history lesson to forestall the endless series of "Oh god, why do we do this?" questions.

I am slowly turning into this person and I am not sure how I feel about that.

https://users.cs.utah.edu/~elb/folklore/mel-annotated/node1....

Man spends entire career convincing himself the reason he keeps job hopping is because everyone else is writing crappy code.

Karl Hackerman shows up to each meeting complaining. He seems sour that he hasn’t been given the go ahead to rewrite the system in Rust. He knows a lot about bleeding edge frameworks and best practices and always seems annoyed that he can’t use them at work. He thinks it’s because everyone else has settled in to becoming wage slaves instead of striving to be the best programmers they can be. Everyone puts up with him because they know he’ll quit in six months anyway.

Karl is 47 now. Hasn’t held a job for more than a couple of years. Has mostly been freelancing lately and trying to sell books on tech and frameworks nobody cares about anymore.

I did not expect to wake up and be attacked like this. Not even a warning. Savage.

Good thing I'm not in my 40s yet, still time to recover from my wayward path.

Yeah this is ruthless. On the other hand, I feel ok with my present course because the intermittent chance to use new shit is the only thing that keeps me interested anymore. That and the exorbitant pay, I guess.

FYI, Karl has been reincarnated as MyCurrentIntern

Author here!

I love this so much. I've totally been this guy :) Learning how not to be an insufferable doucheb*g was a surprisingly hard challenge for me

I've been writing a longer fiction piece about overzealous know-it-all engineers in the style of Confederacy of Dunces. Probably won't be done for a while, 'cause there's a lot more of my soul in it haha

Awesome! Enjoyed the post. Keep up the good work as long as it makes you happy. :)

Please do! That’s my favorite book! I love your other posts too

The only book i’ve read cover to cover in one day. I would buy a programmer-based version of it in a heartbeat!

This was great.

I felt this way earlier on, but quickly learned to get fulfillment in my day job purely from shipping and contemplate the mysteries of computer science largely on my own time. Sometimes they overlap, but good day jobs are often 80% boring in my experience.

I lost it at "best practice". I once worked with this guy. He would use that term whenever he didn't have any good argument, which was basically all of the time. He would have the strongest opinions, whenever it didn't matter...

It’s appeal to authority, a logical fallacy, the things people who can’t accept they’re either wrong or outnumbered fall back on.

For ten years, I worked on codebases written in C++ without using any third party libraries (no, not even STL -- we needed faster, thread safe versions of strings and maps which were not available at the time; Boost? What's that?), its own custom messaging middleware built directly on sockets, its on distributed process management system built with unix syscalls (obviously). There was not a single line of HTML, Java etc. anywhere. It was a comfortable, high-paid life and I solved cutting edge problems (HFT, distributed, concurrent computing). In 2013, I forced myself to quit, start with a small company that paid me half the salary so that I could learn the web tech stack from the ground up. It was all to avoid this fate. I'd like to think I've succeeded.

> so that I could learn the web tech stack from the ground up. It was all to avoid this fate

same fate

> I'd like to think I've succeeded.

possibly not

What about it would you like to avoid? I don't understand the fears that people have in this thread. To me this sounds like the Platonic ideal of a programming job.

EDIT: I'll reply to my replier here because /u/Dang rate limited me (again), though I must commend him for letting me have one more comment than usual before he pushes the button ;)

> I would say the ending -- the part where you are unemployed and unable to support yourself in your old age because you invested decades of your brainpower into a highly specialised, non-transferrable skill.

I would contend that such a person, unless very reckless with finances or extremely underpaid, should have enough money saved up to make what is mentioned there very unlikely. That said, I know some high-income earners struggle to not blow their entire bank account each month

I would say the ending -- the part where you are unemployed and unable to support yourself in your old age because you invested decades of your brainpower into a highly specialised, non-transferrable skill.

I should think that years of specialized C++ work in hft environments would have come with impressively high pay… in which case one could retire early and self study anything for fun, not hope to find a job for financial reasons.

Ideally, yes! Unfortunately, there is such a thing as being young and not knowing your own worth / not knowing how to negotiate / not having a manager who looks out for your career growth.

Understood.

In that case, if web development is your interest, then you have many possible paths.

The primary choices are full stack, backend, or frontend.

Obviously full stack is most flexible because you can theoretically do it all. But keeping up with so many technologies simultaneously is really tiring.

Take a look at Phoenix framework. It is a fantastic way to get started, and it will grow with you into just about any future project need. Plus Elixir is an excellent language built on top of the incredibly powerful Erlang ecosystem.

The web stack might give you more transferable skills but how did you calculate the trade off vs the 50% pay cut? Was the salary not actually very high and at a low ceiling compared to web work - i.e. you expect your salary to soon exceed what you made working in HFT? Or was there a serious risk of layoffs happening far in advance of your retirement? If the goal is to support yourself after retirement, are you saying in certain circumstances halving your salary 10 years into a tech career ultimately optimises your entire-career-earnings?

It wasn't such a purely material calculation. Less stress, more time to spend with family and friends, learning more things and doing something that you know actually adds value to the world -- those things matter.

That's mostly the guys fault and not the jobs fault.

Even in a bad code base you can start making things better, apply modern techniques and learn new things.

Also most of the skill you gain there is transferrable. E.g. just because you have to work with some legacy PHP project, it doesn't mean that you would suck at creating modern applications in JS. It would just take a couple of weeks to get use to it.

Making things better and applying modern techniques don’t map to a feature story. In the environments being discussed you’ll have to wait for a cadre of architects and program managers to convince the C levels to commit to a multi year, millions of dollar effort to have any hope of improving a code base.

You don't have to rewrite everything, to start fixing things.

Must be nice to live in your world of being treated as a professional.

'So your story ran over again help me understand where we can improve your work flow'.

> Even in a bad code base you can start making things better, apply modern techniques and learn new things

All th bad codebase I've seen were a result of fossils who are still with the company gatekeeping anyone from fixing their code.

A few years ago, I had one architect stubbornly refuse to use source control, rejected all PRs that didn't use his homegrown buggy C++ libs for strings, refused to let anyone do the http server bits unless they used his C++ server and so on.

If there isn't a gatekeeper, you wouldn't have the bad codebase in the first place.

I've got to disagree. not every contributor is making the code base better. I'm in a situation where we are such a small team that there really is no gatekeeper, and we only require one approve per pr...and I've come back from vacation to some monstrosities that I have to look at with no time to fix. After a pairing session with a a fellow developer, where I took the extra effort to refactor something along the way, probably an addition 15 minutes of effort...she quipped- no wonder it takes you so long to finish your stories sometimes. I mean, that's not why- I'm ADHD, so my lack of productivity sometimes is invariably due to a day where I forgot my meds or didn't get a full nights rest.

> "just because you have to work with some legacy PHP project, it doesn't mean that you would suck at creating modern applications in JS"

I disagree; working on a legacy PHP project isn't just opportunity cost (failing to sharpen your knife), it's actively harmful (leaving the knife in salt water, accelerating the rust).

Shouldn't you have ample savings, a private pension and social security?

? In the US? Maybe not. I'd bet a lot of hot shot young programmers think it will be gravy for ever.

Start saving now.

Being good at C++ is very much a transferable skill.

I'm sure he could learn STL (standard C++ library) in a fraction of time it would take someone else to become proficient in C++ in general.

It is a transferable skill. But not one many people want to pay for except in very niche areas these days. So if you are out of a job expect to grind for awhile trying to find someone to hire for C++. Unless you have network connections then you might be ok. Being good at C++ only goes so far, you have to have skills in techs people are willing to pay for. Not many companies are willing to wait while you skill up in their stack unless you are willing to work jr but they will not hire you anyway because 'you will jump ship'. Its not great.

that part is a complete fiction. People who have a lot of experience in the embedded world or performant but esoteric codebases are highly thought after. Even if the literal job isn't transferable, the skills for sure are.

My mother was a COBOL developer and didn't work for 20 years because she stayed home. She recently returned to her old bank she worked for because they are dragging people out of retirement as they can't find anyone else. She earns more than I do now.

People are crazy to give a job like that up for generic web dev work.

Most of my early programming work was with C and C++ (after I left academia, where it was mostly Fortran). I am, simply put, a better programmer for it, objectively so when compared to my peers who've never touched it, at basically every place I've worked at since (including a FAANG-level company, currently, where I'm mostly writing in go and python). At least, that's the feedback I get from them.

I think it's fair to say this skill difference is largely from that work in the C family, where I learned a number of different paradigms for design and development in that world, with all its footguns and low-level "gotchas". It has always been easier for me to parse and understand others' code, identify subtle bugs, to use debugging tools, and to identify some optimizations by examining the code that others might spend days using profilers to find.

Some of that would come with engineering experience in general, regardless of languages used, but not all of it, but I picked these skills up faster and earlier in my career, thanks to that early work. Skill with C++ is definitely transferrable, in at least some cases.

Are you aware that rate limiting is not something a moderator goes around doing to a single person, but instead a server managing its own workload?

Hacker News has per-user configurable posting rate limits. Moderators use them.

Was it something that ran in an embedded environment?

High frequency algorithmic trading. Often ran on colocated hardware within the exchange / trading system data centers. Started with stuff like Sun SPARC, Solaris and Oracle and we slowly made our way to Suse, Intel and... surprise, our own database optimised for fast writes.

Bloomberg? Comdb? If not, sounds very similar, anyway. Been there, done that!

Naive question: is this like creating fancy tech that is helping rich people to front-run the poor and middle-class?

Because that's what a lot of people think online. Maybe it's more nuanced. I'm curious how someone from that carreer sees it?

No, the poor and middle-class are not day trading stocks in any sort of meaningful volume.

It is to help the super-rich who own hedge funds to frontrun the rich upper-middle-class people with significant disposable income.

Unfortunately, yes. It never sat well with me. Colocated systems saw latencies no mom-and-pop investor could ever dream to achieve. That is one of the reasons why the second company I joined was a non-profit that built systems to help track human rights violations, enforce labor rights, monitor democratic elections etc. Third job was in the food industry and the last one, in housing / property management. So I'd like to think I've made sufficient amends.

I wonder why mom and pop are investing in individual stocks at all. Buying the index has been a passable "get rich slow" scheme for me.

The whole business model is misunderstood by your average internet commentator.

When you have a market, you need someone to be there to provide liquidity. Imagine if you're a farmer and you show up to the market with your wheat, but all the bakers have gone home that day. Or the baker shows up and there's no farmer. The market maker stands around all day offering to buy and sell so that you don't have to wait for the guy you're really trading with. Of course this middle man wants to get paid for it, but your cost as an average Joe is next to nothing. This is trading in time.

Now imagine you want to cook a meal and your ideal meat is beef, but actually you're ok with pork, so long as the pork is cheap enough to make it worth it. How much cheaper should it be? Well your fellow who knows all the pork and beef people will be able to gauge where the balancing spread is, given the amount of interest. In fact he will from time to time do the trade when the spread is out of line. This is trading in space.

So why all the fancy tech? After all market makers used to stand around in a pit in a colored jacket, and they didn't have degrees. My first boss in the market was one of these guys.

Well, things have gotten very tech heavy because as soon as prices are out of line, there is money to be made. Or rather, lost. As a market maker, you are constantly out there with your prices, offering to buy or sell at a very small spread. If some news happens that affects prices in a big way, you can be sure that you will buy when it's going down and sell when it's going up. In order to both have tight prices and avoid this "adverse selection", you really want to be able to react as quickly as you can when your system decides that something's up.

Ah, the ol' liquidity spiel. One wonders how markets were even possible before HFT...

I explained that. It doesn't matter whether it's a computer doing it or a guy in a jacket, but once one guy gets a computer all the market makers need it.

So take away the computers. Or more realistically, batch run all orders once an hour.

Retail consumers aren't exactly disadvantaged by the gradual narrowing of spreads, so what are you trying to fix? The latency arms-race is exclusively professionals fighting each other.

Getting back on topic, being a software developer for an HFT shop can be a lot of fun: serious technical challenges and a rapid feedback loop in a role where you may be the revenue stream rather than just another cost center. But posters highlighting NIH syndrome are spot on: you risk pigeonholing yourself.

Markets were possible. But what about the current offering of ETFs and similar products? Despite all the fancy talk, many HFT firms are just market makers for the products that enable average joes to invest well and cheaply.

You don't have to be that old to remember how terrible the experience used to be, and how easily even sophisticated individuals were ripped off at every point of the process.

Was that like Gomez Addams with the paper ticker coming out of that glass domed contraption, calling his guy to buy Consolidated Lint?

> You don't have to be that old to remember how terrible the experience used to be

I guess I'm not old enough. What was the system like? How did sophisticated traders get ripped off?

You could easily solve all of this without HFTs - you might introduce a few seconds of latency but virtually no real investors care about that level of latency and it would save them billions per year.

Would save money for the market makers too. Assuming someone couldn't get an advantage, and that's the rub.

Yes you could, but the point is that HFTs don't create any new problems that need solving. They don't make anything worse.

> and it would save them billions per year.

This is not true.

It absolutely would if you remove the cut HFTs take out of the market. I'm sure you'll make some argument about liquidity and spreads but, in the aggregate, the money from higher spreads goes to market participants so the market as a whole still saves by eliminating HFTs, even if the spreads go up a bit.

This is like saying if we eliminated millimeters from rulers, and only used centimeters, everything would get closer together.

HFTs do not take a cut out of the market, their cut comes 100% from other market makers. Those are the only folks who would benefit from eliminating HFTs, and unless you’re one of them, who cares?

I don’t care who buys when I am liquidating a position. That’s the whole point of a market. Therefore I don’t care about HFTs.

"Your average internet commentator" is ignorant by choice. No argument or description will convince them.

You're describing market makers, not all HFT participants.

True but trying to keep it simple.

Generally, it's not about front-running normal people. Instead its racing other HFTs. Either a race to capture arbitrage opportunities, or to update your orders on one market based on trades made in another market.

The aim of the game is to collect a small spread on a lot of transactions.

The scary thing for an HFT like that is being 'run over' by a big party selling or buying a lot over a day. If a pension fund is dumping 10٪ of their holdings of shell, it's hard to collect a spread because you need to find buyers for every stock you buy from them. Meanwhile the price is dropping as they sell off.

Hence you occasionally see 'front running' with price improvement. Instead of buying from the market at 1.02 a HFT gets to sell to you at 1.01, yielding you a better price. This is a good deal for the HFT, because a normal person isn't going to run the HFT over. And it's a good deal for the normal person because they get a better price.

No, it's a zero sum game. HFT firms fight to be the first to pick up a penny, but they can't force anyone to drop more pennies. They don't cause prices to go up more than they would have otherwise, or extract more value from the poor and middle class.

So you left a job that paid 2x and exposed you to hard problems that require reliable solutions for a job inside the horror clusterfuck that's the modern web where nothing really "works" and there's no real engineering to speak of besides monkeys throwing crap in a shared pile and shovelling it out the door ASAP.

Good for you I guess?

Most people I know, inc. myself, would call this a "bruh moment".

I can't fathom willingly entering the web mines if you've already sunk in the hours to master high perf native code, OSdev, dist systems etc. People usually enter web to avoid having to learn such things.

To discard that knowledge and instead compete with an endless stream of fresh grads, boot camp grinders and LLMs for less money? A madness if I've seen one.

Fun fact: I'm actually a bootcamp grad in web dev doing the opposite. I agree.

Speaking as someone who does both:

Web tech is more immediately rewarding. With C++ (and especially with modern web dev agile processes), you can spend a week building enough framework and infrastructure crap to stand up the barest inkling of a demo of the solution you plan to implement, only to have the whole thing pivot out from under you and become wasted work.

More complicated languages and frameworks that manage memory and do UI and everything for you let you get to a prototype much faster, so you can see whether it'll work or whether you want to toss out the idea much sooner.

Tell me you know nothing about web dev without telling me you know nothing about web dev

It's hyperbolic but it's a common sentiment that I tend to agree with. The web dev world changes too fast and too often, many stacks involve absurd complexity for dubious performance and maintainability and other metrics.

I've been doing web dev for 20+ years. Which part is wrong?

It's not hard to find feature support or implementation differences between browsers. But these days, that's not really the problem. I can go read the ES language specification, and generally find user agents/node/deno/bun have compliant implementations.

The problem is brittle "ecosystem" stuff. How are the create-react-apps going? How is module bundling going amidst the UMD/CJS/ESM war?

HFT is still CRUD, CRUD for the extremely wealthy.

If you want interesting, go to HPC large scale simulation.

The arrogance you are showing is disgusting.

Even if the decision he made was a bad one, you don't know anything about why it made sense for him at the time.

People aren't algorithms running in complete isolation and silence. They live in nuanced circumstances you can't possibly understand by his short history in a comment.

I hope you can find understanding and peace in the future.

Damn so it's not just me having that impression. Guess I better steer towards other things..

> so that I could learn the web tech stack from the ground up. It was all to avoid this fate. I'd like to think I've succeeded.

I'm not sure if that's intentional humour or not.

Out of the frying pan, into the cow manure.

There is a massive advantage in being able to see the web tech stack for what it is and wielding it without over-engineering or fanaticism, while your competitors are spending 2-3x on monstrosities that essentially does the same thing, but slower.

The day a professor made me implement a simple http server was the day I achieved code enlightenment.

At that level of abstraction it’s all just strings

That was the web, yeah. Absurdly simple. Now all new protocols are binary and require reading several academic papers instead of a single RFC.

> At that level of abstraction it’s all just strings

Exactly... I worry about the masses learning of this and gaining literacy over the landed gentry.

PHP/CGI basically had it solved. I emulate PHP in .NET6 today using verbatim and interpolated string operations.

Relevant: https://twitter.com/serbantanasa/status/1582019078533046272

I was originally concerned about your parent comment regarding your new foray into web dev (that C++ path sounds amazing), but this comment has put my soul at ease.

If you walk into web dev with aggressive principles and a willingness to be controversial, you can get a lot of shit done and it will feel great. Watching others struggle under the various JS framework circuses has almost an opulent/aristocratic vibe in 2023.

Fair point. The web stack is the one thing I actively avoid doing anything significant with for exactly the reasons you've just described.

And now you are jumping from one "stack" to another, none the wiser, your knowledge hopelessly outdated just a year in. The principles you learned in HFT? Those apply all the same.

I know it's not all about the money, but over the course of your entire career, do you think that move was financially positive or negative?

I'm very roughly estimating that you could probably have worked in the HFT job for around two-thirds the time you're going to have spent working on generic web tech, and still been better off.

Absolutely 100% positive. I was largely an individual contributor all that time, and only did some unstructured, untrained, unguided people & project management. Perhaps because it was such a niche, engineer-dominated environment. Several years after leaving that company, I quickly learned to effectively manage people and projects, architect systems using more mainstream/open source tech and eventually reached exec level (but that's a different story).

That's really some fields I'd love to get into -- less need to talk to people and more need to talk to the machine. What a dream! Alas I only scratched the surface of C++.

Ah, that's the missing punchline: you had ambitions to be something else other than an engineer, and you (quite correctly) recognised that you were never going to grow your wider skills at such a place.

For someone who had wanted to stay an IC, I don't think the move would have been quite as wise. Being in a niche, engineer-dominated environment, and being paid extremely well for it, is nirvana for many lifelong ICs. It certainly feels very different from the experience of Arthur from the linked article.

I don’t think anyone working with low-level C++ for HFT can face similar fate. Most of skills gained at such job seem very transferable. The article seems to be more about jobs where 95% of work consists in implementing CRUD in accordance with very specific business logic of the company.

The only reason to change such job I can imagine is to sit out your non-compete package. But I am not in this industry so maybe I’m missing something.

Yeah, I find it amazing someone would transfer to standard web CRUD development from that. Sure you can have enough of all the custom stuff, but there are plenty of companies that will hire you for your current salary for non-web work. (you can probably guess how little I like web work)

Same! The poster's pivot reads like horror to me. Like they lost their mind and joined a cult where they worship Memnon with sermons of JavaScript.

> It never sat well with me. Colocated systems saw latencies no mom-and-pop investor could ever dream to achieve. That is one of the reasons why the second company I joined was a non-profit that built systems to help track human rights violations, enforce labor rights, monitor democratic elections etc. Third job was in the food industry and the last one, in housing / property management. So I'd like to think I've made sufficient amends.

Ah, nevermind. Turns out they were worshipping Memnon with C++ and renounced that cult with web tech! :p

It's not that you won't have the skills, it's somehow demonstrating them.

I also worked at an HFT C++ shop that used no STL. Every data structure has a weird name and API that you get used to. The whole architecture has its own idioms, and hasn't needed much upgrading in terms of language features.

Imagine you learn the grammar of English. You learn how to pluralise nouns, when to put an s at the end of a verb, how to to use commas, and so on. You then go to work at a place where they have a different word for coffee, drink, and order. Along with every other word. You end up learning where everything is and how to talk about it, and you're productive.

How do you apply for another job? Chances are you will be asked how to order a coffee and drink it.

Very real issue btw, I still have friends there.

This kind of issue cuts both ways - how do you hire new people and get them productive quickly on a code base where they will almost certainly understand nothing?

Getting rid of weird data structures that now have STL analogs then is a productive exercise - existing devs get to learn more STL and new devs have more familiar ground. You get to delete code too.

I work at a HFT shop and put C++ code is modern (C++23, GCC 13) and uses stacks of third party libraries

Yeah I'm not entirely sold on the NIH story. Plenty of things written by other people will work just fine for HFT.

Unrelated question: do hfts make you move out to east coast, or are there some in California?

HFT firms are primarily in Chicago / NYC.

Firms are mostly in the financial centers but maybe there's a remote role here and there.

There's at least one in the bay area though.

I did the same, I thought C++ was heading for obsolescence. Now a master of Java, Javascript and can write great full stack apps. Meanwhile HFT C++ jobs are paying double, I'm thinking of going back, even if the systems are boring.

been working on java jobs and easy code bases since I graduated 10 years ago, nothing had been really challenging and I'm not complaining. Would be interested in a job that pays more even do you think working on C++ jobs is more challenging or just boring?

Depends on the job of course. Probably more technical problems and tuning. Often code is low level which is interesting if its your thing but not if you like business problems.

Sounds like your old place had a good idea what's good about c++. It's not the STL and certainly not boost.

Legit, is that place hiring? Sounds like an ideal stack free from the tragedy of the commons.

Those stacks can be full of 'not invented here'. They are unique and have their own quirks. Quirks that you more than likely can not google for any sort of guidance as to what is going on. I wrote one of those stacks a few years ago. No std libs at all. Why? must fit in 32k and most of that space is needed to hold data. That means throwing a lot of those helper bits out or spend weeks figuring out how to bend the compiler to your will and not include everything. When it is easier just to roll your on CS version of whatever. But it sucks because you probably have odd edge cases.

Yeh it's a rough but rewarding life; I spend most of my hours on problems that aren't on Google, or any LLM, so I'm kinda used to it now. (Stockholm)

Would love to know of more spaces which require this level of programming - usually I feel kinda underworked at most shops.

Though, true enlightenment is combining the urge to NIH with some, light, sensible borrowing from those smarter than us :) very much a key skill unto itself.

Game engines have a lot of low-level C++... Often not the best working conditions though from what I hear.

The space I was in was IoT type items. Usually hub controllers. The thing is that is getting less and less as you can get a fairly robust thing now with a couple hundred meg of flash and ram for a very reasonable price. So those 'must fit in 32k' items are few and far between these days. With that sort of space you may as well use the proper libs that are usually nicely documented and all the quirks have been seen by hundreds of other poor souls such as yourself.

Embedded programming is this 99% of the time. Abstracting hardware away is a silly notion. Docker? Cloud? Hard to do when your code only runs at all on one specific custom board. The protocol you wrote to talk to that one chip isn't working? Time to break out the oscilloscope and look at the actual data going to the pins.

Join us, the pay is worse and the work is harder, but also the prestige is lower.

I sometimes really miss embedded programming; it felt like real engineering in a way that higher-level software development often doesn't. Alas, the shop I worked for could not keep me busy enough, the projects began to feel repetitive, and I got really bored; it'd be hard to go back.

> Join us, the pay is worse and the work is harder, but also the prestige is lower.

I tip my hat to you!

I have a pretty similar story. I spent 7 years working as a subcontractor for a big telco with CORBA, pthreads and the ACE reactor library. First week, I halted a whole contact center service by adding a sleep call to the codebase (it was on UAT environment). Fast forward several years and I could effortlessly debug cores with thousands of threads, reason about semaphores, reentrancy and even tell the difference between the POA and the BOA Orbix adaptors. I began to become the mythical guy who knew what everything did not because I was technically good, it was because I understood the business and how it mapped to the different architecture elements.

Then I felt like I could spend my whole career there and decided there was a whole world out there in the web, began to learn PHP in 2004 until the mythical "blog in 5 minutes" Rails demo appeared. I switched jobs in 2007 and I have never looked back.

Interesting. I've used semaphores with pthreads (only once -- after that I realised plain old mutexes will do). Like your sleep call, I once brought an entire service to a grinding halt by adding a simple `fsync` call (to fix a recovery failure because once a sequenced message we thought we had written to disk before a crash had actually got lost in the buffer). Understanding things at that level makes any new tech fairly easy to grok.

> I understood the business and how it mapped to the different architecture elements

If you have access to key systems, you can definitely reverse-Conway to understand how an organization really works.

https://en.wikipedia.org/wiki/Conway's_law

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.

— Melvin E. Conway

That's so correct, this was Conway's Law in action!

> learn the web tech stack from the ground up

Ouch. Should have stayed in legacy C++.

C++ is such a damn chore. This is partly why, everyone always assumes they can do it better, they don't trust any other code, so you end up with monoliths of NIMBY code and esoteric build setups. Code reviews are incredibly painful for the same sort of C++ reasons, everything can effect everything, the whole idea of encapsulation tends to fall apart when memory, time, ordering behavioral effects get relied on after some amount of time usage. Every change is a footgun to be found in production under just the right circumstances.

Hard to be productive when you can't trust the language, the tools, or other developers to be correct.

I hope it worked for you, but I don't see any big difference. Web technology will become dated faster than the old C++ codebase. If you learn some web tech stack today I expect that it will be obsolete in 5 to 10 years. What are you gonna do next? Move to another small company for half the salary to learn everything again and compete with 20 year olds?

> in C++ without using any third party libraries (no, not even STL

As God intended it!

That first place sounds awesome

Don't live to work, work to live. I bet Arthur partied his ass off back in the day, stumbled into work without sleep, napped during retros and refinement. He might be the least useful dev but he's a king at Burning Man. Now with his flush 401k, he can finally retire to make little wooden figurines to sell at the farmers market, where he really just wants to regale people with stories of the biggest fish he ever caught, or his dream of opening up a ramen food cart if his in-laws will go in on it with him. Arthur's lived a life - just not at work.

Yes that would be great and I do hope for mr A the reality is some variant of this.

More often than not though, its not so rosy. Arthur would be overweight, riddled with medical issues, solitary guy with unhealthy attachment to XYZ (anime, gaming etc).

But if he had kid(s) and raised them well, well as a parent too I do have tons of respect for him for that alone, rest are details when looking back.

Hard to do if most of your life is work. I find it very hard to find space for an actual life after subtracting a full-time job, exercise, personal hygiene, nutrition, etc. from my waking hours. It's essentially ~2h a day + the weekend, which often also consists of getting stuff done you didn't manage during the week.

I hate the thought of my life starting in retirement. Especially if I might be 67 by then, which is the legal retirement age, where I live.

https://taylor.town/10-minutes

> Especially if I might be 67 by then, which is the legal retirement age, where I live.

When you say "legal retirement age", what do you mean? Is this to receive some sort of retirement benefit like Social Security in the US? Or is it something else?

That, plus you'll pay penalties (10%) if you draw on a retirement account like an IRA/401k before retirement age.

The retirement system is completely broken over here. As an employee you have to pay into it. I pay > 500€/mo, which a big part of my pay check and one of my largest expenses. My employer has to pay the same amount on top. You might or might not consider this essentially an additional cost for me as well. It is intended to be the primary pension plan for most people, though by the time I will go into retirement, the whole system will be unviable financially and I might get nothing. If I do get something, I get it starting after completing my 67th year of life. Of course I'd rather invest that money privately. If I choose to go into retirement early, I will have to pay hefty life-long penalties - 0.3% per month that I retire early. Obviously a couple of percent over 20 years of retirement or so can hurt quite a bit.

It might sound silly, but it's one of the main reasons I want to leave this country.

It sounds silly because Germany is a good place to retire in, not a good place to be young and work.

> by the time I will go into retirement, the whole system will be unviable financially and I might get nothing.

What do you mean when you say "financially unviable"? There have been similar concerns raised about the US retirement system, but AFAIK that only means less money coming in than promised going out. The income is predicted to stabilize at ~75% of expenses after the "Baby Boom" generation finishes retiring. At the worst, I'd expect a pension cut rather than a complete collapse to 0.

I don't know how the US retirement system works, but I think Germany is pretty unique in that the current working population is directly paying for the pensions of the retired population. There is no investment of those funds of any kind or something like that. The state is already taking up huge loans to finance this, as we don't have enough payers. We have about two payers to one pensioner at the moment, but when I will be 67 with the current population development it will be one payer to two pensioners. Even with more massive debt one payer will not be able to pay the pensions of two other people. It might just be a huge cut, but if it's not enough to live off of, large parts of the older generation might end up on the street. A large part of the younger generation is aware of this, but 23% of Germans are over 65 and the median age is 45 so older people generally have a lot of voting power and the issue is largely ignored. Very recently there were proposals to switch to a system in which the pension payments are invested into index funds, but it was decried as "gambling away our pensions" and the proposal was shot down.

The US retirement system (Social Security) is paid for the same way. The population pyramid for Germany does show a much larger hump of upcoming retirements compared to the US though. The ~30yo population cohort has grown in recent years though (through immigration), so maybe it won't be as bad as you expect.

Why do you think Index Funds would solve the upcoming issue?

https://en.wikipedia.org/wiki/File:Gemany_population_pyramid...

mate you think retirement age is 67, thats very cute of you. just wait for 20 years and see how many times this milestone will change with the way the world population is going :)

This. People don’t see what’s coming. The Era of Abundance is over.

I don’t know your circumstances so don’t take this personally as these are some general ideas:

- work 4 days a week only

- save 30–50% of your income and retire after 15 years

- cook your own meals to eat healthier and cheaper (bring lunch to work)

- buy used stuff from a couple years ago

- fix your own stuff and buy cheap “broken” things that are fixable

This way you might not need to die at work or wait 67 to retire.

I think this was a contrived example, but I really appreciated all the little details here! They struck close to home and felt very validating, even if accidental :)

There are talented people everywhere, the probability of somehow being ground breaking and world impactful is ridiculously small. Anyone pegging their self-worth on that goal is setting themselves up for a lot of mental health struggles.

My levels of ambition and relentlessness absolutely shifted after having kids, I used to be absolutely furious about being the best. Now I am very much "fuck you pay me" and my priority is having as much free time as possible for family and friends. I for one would be fine with Mr. Westbrook's career although thankfully I haven't had as crappy colleagues as him throughout my career.

There are people of all types who will want different things, no point in judging. As the great thinker Alicia Keys once said: 'you do you boo!'

Agree with you. I don't understand the point of this article other than some sort of tongue and cheek snobbery. Hope it maybe made the OP feel better about themselves.

Author here!

For context, I recently wrote a tongue-in-cheek guide on how to coast in large companies.

[0] https://taylor.town/corporate-camouflage

That got me thinking about what happens when people, especially engineers, coast too much, or fail to plan out their careers.

I know plenty of smart people that are enchanted by comfort, then chain themselves to companies that take advantage of them.

If you've developed transferrable skills, this parable probably isn't for you :)

I personally think it's okay to be ambitious, and it's also okay to chill.

Honestly, I just thought it was hilarious -- a sort of parodic counterpoint to the Story of Mel.

I thought I was reading at The Onion!!

You say tongue-in-cheek while, almost verbatim, summarizing my day-to-day strategy at $megacorp! Seriously great stuff.

I will not call it snobbery but a difference in generational culture. I am assuming you are below the age of 45 and came into the industry where jobs were aplenty , super low interest rates and the industry was rapidly innovating and risk taking is encouraged . For a lot of folks from the Office Space days of 80s and 90s that wasn’t necessarily the case . Tech jobs were limited and you probably had to post or submit a paper version of your resume. I would encourage you to take a look at JavaScript code from 2003 or a Java codebase from then and see how dreary and different the tech industry was back then . As for transferable skills there is a reason IT workers were stereotyped as autistic nerds before the brash,well rounded, rock climbing risk takers came in. Culture, risk, opportunities and interest rates had to do a lot .

Edit- I was probably like Arthur Westbrook when I was working on a visa and had a family to support . As soon as I got my greencard my appetite for risk taking increased and showing my transferable skills increased . Situational context .

This level of reaction tells me more about you than the author. Referring to it as 'snobbery' shows that you interpret satire as a personal attack on someone, probably yourself. I clicked to their home page and immediately figured out that they dabble in tech and like to joke around.

This is as much a tip for myself as for you: take a deep breath before reacting.

I think it was just a joke :)

You're just going to off load your lost potential and lack of ambition on to your kids, and so on and so forth. It's the main reason why people have them.

I mostly had my kids because I was drunk.

Boredom is another common reason. ;)

Well, if nothing else, it certainly solves that problem.

You're being negged into oblivion, but you have a point.

There is "I'm OK with my pedestrian job but I still try to do my best and actually like the people there" and "my job sucks all joy out of my life and provides me with nothing of value except money".

I have kids, but I somehow skipped the "I want to be the best"-phase and went straight to "I'll make do with whatever you give me" so nothing really changed. I actually work a bit harder now so my kids can see work is good - also to pay for their fancy shit.

Even stupid jobs can be sources of relative joy if you let them. Don't put up with abuse of any kind, though.

The opposite is no good. I had 1 or 2 opportunities which a lot of people of my age were looking at with envy. I was not super gifted but they came out of passion: by working on the things I loved relentlessy, I ended up being noticed. I was good because I worked a lot. It was not hard work since it was stuff I enjoyed a lot to do, but I spent countless hours on it (about 25-30 hours a week in addition to my studies or day time job).

Problem with the kids is that when I tell them about that, they feel they don't have enough passion to succeed. So you see, eventhough I was on the right side of success for a while, it scares the hell out of them...

> they don't have enough passion to succeed

Then they should find something they are passionate enough about to succeed.

Have kids really bring things into perspective. Every moment with them is unique.

Every moment without them is also unique.

Everything is unique, but some are more unique than others.

it is all the same to me

Some dichotomies are more dichotomous than others.

Having kids is such a traumatic, life-changing event. I mean this in the most positive way. I can see why so many people see this as the before/after watershed moment of their lives and ambitions. However, it's understandable that not everyone needs to have kids to come to this realization.

For me it was the profound perspective shift from being a leaf node of a tree so to speak to becoming just one link in a long chain. It tends to relax the do-or-die drive quite a bit.

Maybe the do-or-die drive is relaxed because you (or your body) realizes that you've fulfilled your biological purpose and passed on your genes?

It also makes it feel all the more exploitative that young developers are encouraged to work long hours and put their everything into their job. And that women are de-incentivized to have children by the market, if not capitalist society at large - don't do it, it'll cost you your career! We only give you two weeks of leave after you gave birth. You can't afford child care, or all your wages would be spent on it! etc.

That's a US only problem though.

You don't think it's an issue in Japan or south-east Asia?

It’s not, really. Japan is absurdly good if the only thing you want to do is have children, and don’t care too much about how or where they’re schooled.

1 year of leave per child, inability to get fired while on leave, free childcare.

The only negative is that all those make advancement options for working age women 25-35 whom are expected to have kids ’soon’ nonexistent.

Oh sorry, I was talking about the "young developers are encouraged to work long hours part" not the after birth care.

Japan has the highest fertility rate in Asia now. SK is doomed economically over the next 50 years.

In France, women get 10 weeks & men get 4 weeks paid leave.

This is surprisingly low for a country known for its long vacation! Especially the men.

Here FAANG it ranges from 18 to 24 weeks for women, 12 to 18 weeks for "non birthing parent" (men, or adopting parents).

There's State-by-state paid family leaves laws, which makes it usually around 12 weeks (men or women).

You cannot fully appreciate your moments without kids until you have kids though.

Why do people feel they can make sweeping assertions about other people's experiences like this? You don't know anything about the lives of the billions of childless people out there. Many of us have livestyles that wouldn't be possible if we had children, and our "moments without kids" are totally, incomparably different from your moments without kids.

The smugness of people with kids is just annoying.

People with kids have also been people without kids at some point in their life. So I'd guess they know both perspectives.

I didn't read their comment as smugness.

They know two perspectives. Both from their own point of view.

Maybe I should be smugger about myself having kids, but it was really meant as nothing else as a lightheaded remark. From my own perspective of course - there is clearly my (nick)name over it.

I can relate. Sometimes they are pure abomination.

Lighthearted. I meant to write "lighthearted". Anyway.

Sure, and if it was just your comment it wouldn't annoy me. But these sorts of comments about non-parents from parents are constant. I totally parents that they enjoy having kids--why is it so implausible to parents that I can equally enjoy not having kids?

> The smugness of people with kids is just annoying.

The ability of people without kids to have a firm belief on how they’d feel with or without them is also astonishing.

I doubt people with kids are smug about it. It’s just been life changing for them, and to some extend they must regret that other people will never get to experience that, especially if they were initially hesitant too.

Anyway, fully agree it makes you appreciate your time more. I enjoy adjusting the flow of time to school age again, even if it’s not my own.

> The ability of people without kids to have a firm belief on how they’d feel with or without them is also astonishing.

First he tells me how I feel, now you're telling me what I believe?

I don't know how I'd feel if I had kids. What I do know is that I "fully appreciate my moments without kids".

If a non-parent told you that you can't truly enjoy your free time because you have kids, wouldn't that be a bit rude?

> If a non-parent told you that you can't truly enjoy your free time because you have kids, wouldn't that be a bit rude?

Nah, I’d nod my head and agree, but I imagine that’s not the point you were going for xD

I feel it’s like growing up, it’s hard to appreciate childhood, or university, or any of that until it’s passed.

By the same token you start to appreciate the seas of time you had without children once you have them, but by then it’s too late of course, and you can only appreciate them in hindsight.

I’m sure the same thing will be true after my children grow up. I’ll appreciate all the time I got to spend with them more in hindsight.

We have drastically different life experiences.

In general I appreciate things a lot more in the moment than in retrospect. And I'm much happier now than I was as a child or a university student--the thing I appreciate most about those times is that they're over.

And look, it's fine we have different experiences--I'm not here to invalidate your experience. But when people say stuff like "You cannot fully appreciate your moments without kids until you have kids though" you're invalidating my experiences. That's total bullshit--you don't know anything about my life and whether or not I truly appreciate it.

If you want to talk about your experiences, that's cool, and I'm actually even interested to hear them. But don't pretend like your experience is some universal truth. It's not.

Maybe - but I don't have to. I'm happy enough as it is to not have children. I don't need to loose a leg to enjoy going for a run either.

It seems tragic to me that you had to procreate to appreciate your time on earth. I am grateful and thankful for every moment. Life is full of wonder to me without the burden of children.

It appears you're grossly misinterpreting his statement. When you have children, you treasure alone time _that much more_. That's his meaning.

It isn't some desperate plea like you're implying.

Appreciation has a range of values. Having kids for many people is like going from black and white to color TV.

Yes I can. That's why I don't have kids.

To me I enjoy every moments with my children. Not only enjoyment, I couldn't have imagined how much I'm learning from them and I'm experiencing life so much fuller now. For one example, empathy, all the books about how to be empathetic can't make me a bit more empathetic. My children do. Another example being learning to listen, it is humbling to have your children asking you to listen them more. Ok, one last example is gaining inner peace of people not "understand" you. It happens in work place a lot that sometimes seemingly good idea goes unheard. There are lots of things I can share with my young. It is okay if they don't understand now. And I'm learning from them too so what I think I know might not matter 10 years down the road. Live the moment

The rational side of me tell me it is all my selfish gene in action. And I'm loving it.

This is stated as if the only way for someone to experience kids is to have their own.

I didn't need to have my own kids to appreciate the peace of not having them (although at some point I do still intend to have kids, just not soon). I just had to frequently babysit my niece and nephew.

Nowadays I visit them only for a few days every few weeks and even then it's blatantly obvious that kids would mean a sharp drop in my ability to have as many different projects and hobbies as I do right now, for at least 5-10 years. Makes me really appreicate my moments without kids, as much as I also appreciate being able to see my niece and nephew grow and develop into their own selves.

> I used to be absolutely furious about being the best.

I found out that just being better than yourself takes you a long long way.

I have never competed, but just did my thing. Then found out that I've already know more and ahead of many people I'm inspired by.

Yet, this was not my aim, and still is not. I'm just one of the so-called dark developers. I just do my thing, try to do it better every time, and don't care about hall of fame of anything casual or serious.

I recently put a little modal on my phone that just says "Compete against yourself" every 30 minutes, and I'm finding it remarkably helpful at focusing my efforts back to what I want to see in the world.

I like the mindset of trying to be just a little better than you were, every day. It feels... correctly scaled, somehow.

What does it mean in this context to compete against yourself? Anything at immediately at hand or on the todo list? Or is it just a reminder to get something done you thought of doing but normally wouldn't otherwise?

It means "It doesn't matter what other people do, are doing, or have achieved". In other words, do not compare yourself to others.

Do what you're doing, a little better this time, then GOTO 10.

I've become very fond of the phrase "comparison is the thief of joy" lately. This seems like a version of that. Keep your scope local only.

I like it phrased this way: "There is nothing noble in being superior to your fellow man; true nobility is being superior to your former self."

Great for the upside of growth - how do you change it for when you start to age and skills erode a little bit. That would be deeply frustrating. Need to change the model at that point! What to though?

Your skills don't age like your physical body does.

My existing skills don’t, but learning new ones sure takes more effort.

It's not the effort of learning, it's the effort of tolerating all the know-it-all newbies that increases.

You are saying it like skills aren't contained in that physical body.

You can program using a blow stick if you have to. Its a mental ability not a physical one. Unless you get dementia you could do it forever

That is true, but the rapid change in the tech industry makes "skills" mean something different than it would in, say, woodworking.

Once you learn woodwoorking, you're going to be good at it (until you lose strength, flexibility in your fingers, etc).

In our industry, to 'stay at the same place,' you continually have to learn new technologies, new concepts, etc.

Computer programming isnt like woodworking though, its all in the mind

The AI helps a bit, I'm feeling a bit recharged with it, like I have a few more extra years of capable contributing in me. Easier to get stuff done, not getting worn out with technical minutia as much.

Move more into to people skills and relationships. It could be management, marketing, sales, HR, recruiting, mentoring, training. There's a plethora of side functions. Or take a lower paying less demanding role. Do harder technical work that isn't time constrained like open source or personal projects.

You willingly turned on a notification every 30 minutes that tells you to compete against yourself? I cannot tell if this is parody or just Peak HN

I'd go with Peak HN.

Everyone has their quirks. I've got my computer speaking the time every 15 minutes and everyone I know would be driven absolutely insane by that. It helps me keep track of time.

Some people put a postit on their mirror or laptop screen.

This is a comic I love (titled: Weird): https://viruscomix.com/page500.html

I store a copy in my "life changing comics" folder.

Ooooo... a peanut butter banana bacon sandwich sounds awesome.

Highly recommend.

I've done a PBBJ (peanut butter bacon jelly) but never a PBBB. Might need to try it.

This is a great comic. Understanding and appreciating other people's differences goes a long way. So much harm comes from being fearful or judgemental of people who are different.

Now I'm curious - what else is in your "life changing comics" folder?

Baki the Grappler.

oh hell yeah you're my kind of guy.

A couple notables from my collection:

   Hey, C'mon: https://www.viruscomix.com/page528.html
   The Human Race: https://viruscomix.com/page531.html
   So Mature: https://viruscomix.com/page532.html
   Sexier Th>n: https://viruscomix.com/page542.html
   Copper: https://www.boltcityproductions.com/copper

There are other another honorable mentions, but I was not able to find the original URLs, so I didn't add them here.

I'm intrigued by human nature and why are we the way we are, and search for a calmer, more peaceful self inside me. Apparently meditation helps a lot.

Mine is currently on a 15 minute loop as well, but it just plays prerecorded numbers from 15 to 0 telling me how many minutes left in my 15 minute time block

The time notification reminds me of a screenless "watch" 10 years ago. It would buzz every 5 minutes as a supposed aid in perceiving the passage of time for various tasks.

https://www.theverge.com/2014/1/30/5361210/skrekkogle-durr-t...

Yes, 30 minutes is a shockingly long time. I'd say 5 minutes makes a lot more sense.

Those are rookie numbers. You need to get that down to 5 seconds.

Amateur. I have a neon sign that strobes every millisecond in front of my office chair.

How embarassing for you.

I have a GPS disciplined rubidium clock with a 100MHz output hooked up directly to my sinuses. The slow sensation of waves washing ashore keeps me awake long enough to interact with the rest of you when I am taking my daily luxury 3ms break to read HN. Further, I have a script that, when my break time is over, automatically terminates any conn-

If you find yourself becoming desensitized to the notifications on your phone, I suggest training a parrot to stand on your shoulder to liven it up. Great for parties too.

i suggest a new push notification on your phone that says "be funnier" - all the top comedians do it

That is coincidentally similar to a pomodoro sprint: 25 minutes of work (compete with yourself) and 5 of something else.

Would you mind expanding on what you mean by ‘dark developer’? I could guess from context but Id appreciate your definition if you dont mind.

90% sure it's "dark matter developers" as in "they never write blogs, they don't go to user groups, they don't tweet or facebook, and you don't often see them at large conferences." - https://www.hanselman.com/blog/dark-matter-developers-the-un...

Yeah, that's true. I have a blog, and share the code & docs I write for myself, but my blog is not about programming, and the docs are more like cheat sheets for general public.

Well, the code is just GPLv3 licensed small tools which I develop according to my own needs. Nothing impressive.

I don't use Twitter, Facebook and Reddit anymore. I just converse here, and with a Discord server I really like.

> I found out that just being better than yourself takes you a long long way.

That resonates with me.

There is a saying where I'm from that says "most people are just kicking dirt" that basically means people are doing the bare minimum or don't care at all, and if you just do your thing and really care about your stuff, you're already ahead of the majority.

Being anxious all the time, I use that saying in the opposite way though: just relax and do your thing... you don't need to worry about not having a job, all those other people are barely doing anything and you can't be the best at everytime. Just do your thing are relax.

> I found out that just being better than yourself takes you a long long way.

I usually hold vast disdain for the "self-help" genre of books, but after a recommendation from someone whose opinion I value greatly, I read Atomic Habits by James Clear. I've never found myself agreeing with and finding revelations from such seemingly mundane and obvious statements. Sometimes seeing a statement or opinion you take for granted can make you cognizant of that point.

I've since tried to adopt such "agile" methodologies in personal improvement. Clear says 1% improvement each day always sounds attainable, and such iterative improvements add up.

> There are talented people everywhere, the probability of somehow being ground breaking and world impactful is ridiculously small.

No there aren’t. For every Jonathan Blow, Salvatore Sanfilippo (redis) or Mike Pall (LuaJIT) there are probably thousands of run of the mill developers working at feature factories. You don’t build amazing software by being lucky. Your boss at GenericCo will never go out of their way to ask you to build that thing you’ve always been dreaming of making.

You choose to work on software you find interesting. Or, for any of 1000 totally valid reasons, you choose to be small.

If you think you’re all that, don’t blame your company for “not getting the opportunity”. Please. You already have all the tools you need to code. Make something cool!

I don’t even know where to begin with this comment. It’s a gross mischaracterization of what was being said for starters.

Its not a mischaracterization, its a projection: if you don't do something that parent commenter believes is important then you are not important.

Moral : before starting new job always ask parent commenter are this new job is important for him?

That’s not an accurate summary of my beliefs. Maybe this would be clearer:

If you want to be interesting, choose to do something interesting. Luck is rarely the limiting factor in your creative output.

Thanks for explanation.

> If you want to be interesting, choose to do something interesting.

Interesting for who? For you or for someone else?

I just finished 1 month at clown school doing an intensive performance class. In clowning we talk a lot about coming on stage with a dream. A dream to do something great. To impress the audience. To move mountains with your words and presence. If you come on stage with no dream, the show is dead.

I think work is the same. If you go into work each day with a dream, it has life. You are there for a reason. You love your work. You want to change the world. Whatever. But if you’re going in to work each day with no dream of your own, you are a walking corpse.

The truth is almost certainly between those extremes. People who never put themselves in a place where they can do something amazing are obviously much less likely to do so.

Equally, there are many, many talented people out there who never achieve recognition for what they do or were capable of.

Redis will bit rot. LuaJIT will bit rot. Jonathan Blow did make some fantastic games, but even then it's in a sea of games that are pretty great. Everyone is pretty small in the long run.

> No there aren’t.

You underestimate the odds of talent in a population of 8 billion humans.

> If you think you’re all that, don’t blame your company for “not getting the opportunity”. Please. You already have all the tools you need to code. Make something cool!

There are neurosurgery robots operating with my code running in the background for several years now. I am now working on ophthalmic surgery software for sub-retinal operations. I would think that is actually kind of pretty "cool!".

However, I still know full well that I am a cog in a big machine. I still know full well that my impact is not world shattering. And I still very much think "fuck you pay me!".

My self worth is pinned by family and friends. Work is fun but it's a side quest. If you think differently that is ok, and so am I.

Surgery bots are cool. My software just keeps track of drugs

There are a lot more drug infusions than there are surgeries so you win!

As someone that has relied on such devices to survive cancer - thank you.

You must think I'm cooler than I really am. I don't even do any devices. Just boring ass inventory control.

Stephen Jay Gould in "The Panda's Thumb": "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops"

(whilst double-checking my memory of where that came from, I found the obligatory LinkedIn post using it as an advert for consultancy services: https://www.linkedin.com/pulse/einstein-cotton-fields-sweats...)

> There are neurosurgery robots operating with my code running in the background for several years now. I am now working on ophthalmic surgery software for sub-retinal operations. I would think that is actually kind of pretty "cool!".

That’s super cool dude! Surgery robots are pretty world shattering for the people whose lives are saved by them. Contributing to that sounds like something well worth feeling proud of too.

You may not want to hear this (I certainly didn't when I felt similarly)...but perspective does change with age/personal life...with all due respect do you have kids? A partner? Any major commitments outside work? That's an enormous part of OP's statement.

People don't stop being talented/smart when they have kids/families/personal lives...but they will have less time and their perspectives do change.

It's a balance/spectrum where most of us end up somewhere in between two cliches: do what you love and you never work a day in your life on one end & work to live not live to work on the other.

Your response is completely out of line with what you were responding to.

Yea this is as classic fallacy.

This famous talk outlines it: https://www.cs.virginia.edu/~robins/YouAndYourResearch.html but theres a huge amount of factors that go into becoming one of those guys.

Its not just talent, or desire, or luck.

Its:

  - Talent - obviously have to be really good to even do anything in the first place
  - Vision - Need to have a goal of some sort to accomplish something 
  - Decision making - The goal needs to actually be something the world finds useful. 
  -  Drive - Be able to take constant set backs and negative feedback and believe in yourself to an almost delusional level. 
  - Consistency - This means doing  and thinking about your "thing" for a REALLY REALLY long time. Like weekends and holidays and ignore family and etc...

The talk above goes into it pretty in depth, but theres no mystery about it. You need ALL those things and no less. People who achieve at this level aren't accidents. They are simply not only talented in their fields, but also in areas of being able to consume volumes of information and handle failures. They're the ones getting up at 6AM every day working and thinking about their "problem" and going to bed dreaming about it, for years.

You can be a 10x developer, but you also need to be 10x in several other areas to actually accomplish valuable things. Theres a grand canyon of difference between them and everyone else.

Been in software dev for 43 years. Done the 6am and 96 hour weeks. Then realised I'd missed a big part of my kids growing up, and once they were dead, spending little time with my parents. Too busy you see.

I'd recommend people really think about the costs.

Engineering is one of a few special disciplines where peoples' hobbies align easily with ability to earn money at most levels of professional performance; being mediocre is still a recipe for a generally acceptable standard of life. Compare it to things like sports or acting or music. It's possible to make a living at the lower echelons (read: everything below elite level), but it's hard work or mundane and often not well compensated.

While I love programming and computer science that is one of the reasons I chose programming over other fields. I figure if you want to be a journalist you’re basically broke unless you’re one of the top few that are also making money writing books or doing podcasts. If you’re a historian you are basically not doing well unless you are one of the top and get a tenure track professor position. If you are a programmer, and aren’t even that good, you have a comfortable life in the us. The bottom 20% of software jobs are still in like the top 75% pay range.

Maintenance doesn't get enough respect / glory too. "Arthur" should be highly thought of.

Folks writing new code in the wonderfully free world without a ton of odd dependencies and corner cases ... IMO often aren't doing as much "work".

Writing new code with no existing constraints or requirements isn’t some masterful feat of engineering, usually.

Working in an existing code base, on a team with multiple features going in, gathering requirements from stakeholders and negotiating deliverables on a timeline while refactoring is happening, libraries being upgraded and CVEs being fixed - navigating all of that while being a useful, compassionate team member - that’s priceless on a team.

I largely agree. My goals have shifted from having an impact moving the technology world forward to not being frustrated and depressed from work and getting enjoyment and satisfaction from personal small projects with no deadlines.

Since having kids, most of these have shifted from a few months in time to a few years, but they are unambitious, poorly defined and unimportant so who cares if they take a little longer?

My main goal at work is not getting annoyed or frustrated and getting paid well for my time.

I'm in a constant state of pendulum swing with this. (Not with the goal of being "ground breaking" or "world impactful", but just doing "great" work within the range of my own self-perceived capabilities.)

When I first had children, my pendulum swung way over to disinterest in seeking anything in my work beyond financial support for my family and time to spend with them (while still supporting and not breaking trust with my professional colleagues).

But my kids are growing and becoming more independent all the time, and I'm currently in a phase where my mental cycles and energy are on an upward trajectory over time, and now I'm back to being excited to use those cycles to do impactful work. But critically, I think, with significantly more wisdom about how to strike a healthy and happy balance for me and my family. I mean, I'm not naive, I don't think it will be easy to strike a good balance, I think it require be constant work and effort at it. But as I've emerged from the baby-brained haze, I have renewed clarity that neither focusing my energy entirely on family or entirely on work is the right path for me.

So true, I wish I had this perspective before having kids as well, I was so focused on the rat race and on "being the best", to the point of being hospitalized with intestinal problems. Now I have a great job, that gives me joy but the two things i care the most in life are my two little devils.

It was once said that the national anthem of Hell is "My Way", and that the motto over the main gate reads "Do what thou wilt".

Pride, radical individualism, self-promotion. These are extremely common today, especially in the US (just compare an American resume with a European CV). Both the US and Europe have their problems, but this particular obsession tends to be more conspicuous in the US. Paradoxically, it doesn't bear the desired fruit because it misunderstands the nature of personal excellence.

This isn't a vote for collectivism or mediocrity. Only dead fish float downstream. It's a vote for the notion that the riches of society are the result of countless incremental contributions up and down the hierarchy of value for the sake of the common good. But liberalism embraces an inverted view of reality. All our relationships are viewed instrumentally and transactionally. We place desire above reason. We've become atomized, because nothing sensible unites us (perfect for those in power who view people as "human resources"). The "social contract" view of society is a disaster.

I absolutely LOVE this comment. Thank you for this prose.

> the probability of somehow being ground breaking and world impactful is ridiculously small.

See I have the opposite take. There are so many damn facets of modern life/society, and new avenues/facets/domains coming out every year, that it is actually pretty easy to find some niche where something hasn't been done yet that is world-shattering for some specific community or niche.

World-shattering for everyone is pretty overrated.

I also went from dreaming of being the best at 1 thing to "fuck you pay me" in a hilariously short time.

10 years of being underpaid and yet being the most productive guy at work will do that to you.

Funny, but there's nothing wrong with that if Arthur was/is happy (which isn't mentioned).

Exactly this. Guy had a comfortable stable job for years. Probably knew codebase and technology enough to have lots of time for family/friends/hobbies. More than this story scares me how HN reacted

I can only assume it's because HN is full of young, inexperienced people whose main wish is to stay on the cutting edge of technology rather than sinking into a stable profession

You may be right, but I don't think experience is what makes people prioritize stability over anything else. I'd be so depressed at a real job like the one described, it would be irrelevant that it was stable and gave me extra time. Some people are ambitious, some just like variety.

Variety for the sake of variety is foolish though. If variety is not required, injecting it where it's not needed is just called "breaking things"

Most jobs are like that though... Unless maybe if all you do is develop very small applications, that your company just sells and forgets.

The scope of ambition is larger than your job. Very ambitious people use a job as just the means to an end.

Having a comfortable, stable job for many years sounds like the worst version of hell one could possibly invent for people who live like I do.

There are better things to aspire to than predictability whilst making someone else rich.

Perhaps his aspirations had nothing to do with his day job?

Wait you mean life isn't a video game where my position in a company completely defines me as a person, and if I don't destroy all my colleagues (read: competitors)with cruelty and cunning schemes I still have value?

Only if you don't have other aspirations in life. My day job is solely to fund my personal projects that I actually enjoy. Not everyone is out to be insanely rich.

If you fairly profit from your direct efforts in a lifelong career as a productive programmer, you will merely become rich, not insanely so. You appear to have constructed a strawman.

Making someone else rich is the objection.

No, perhaps I was too subtle. My point is I don't care about the fact I might be making someone else rich. Life isn't about coming out ahead of everyone else you interact with. Some people simultaneously understand your point, and also don't care about it. Only within your own subjective opinion do you think they must.

This view may change as you get older and "unpredictability" starts to mean more negative experiences.

Your imagination is lacking.

Having no job or income and having to worry about how to afford food or rent is surely much worse than "just" having a stable job.

Or how about a super stressful job, like the Japanese salarymen who are literally killing themselves?

Arthur was also able to purchase his large family home in the 1990s for $90,000 which is now worth $2.3 million.

Nothing in the article said Arthur lived in the Bay Area.

It’s not just the Bay Area experiencing this.

Maybe Arthur lived in Springfield, MO and worked at Bass Pro or O'Reilly Auto Corporate offices and lives here.

https://www.zillow.com/homedetails/8057-W-Farm-Road-100-Will...

It’s not, but from the outside the Bay Area’s cost of living looks impressively bad. A typical house in my neighborhood in the Northeast goes for less than 1/4 of that.

Some people like to either suffer or think they are so great that even their if statements will survive into the next millennia

>More than this story scares me how HN reacted

Projections, my dear Watson.

Or whether the employer was happy (it was - he stayed there for 35 years).

I mean… he’s eating cat food. Do you know many happy people who eat cat food?

Cat food is relatively expensive though... I don't undertand why people do (or claim to) eat that.

It was from when there was a time when cat/dog food was wildy cheap. So people would eat it. It is just a leftover from the 60s/70s.

I met a thru-hiker on the Appalachian Trail that carried a few pouches of cat food. You need to carry emergency rations and they need to have decent nutritional content. But how do you stop yourself from eating it when mildly hungry? His theory was, it should be so gross that you wouldn't possibly eat it except in a true emergency. He made in from GA to CT without dipping into the forbidden slop.

So really he just carried unnecessary weight. He could’ve carried an extra pouch of tuna fish meant for humans and it’d have been the same.

There is though.

It's 2023 and today companies will quite happily lay you off the moment you aren't profitable to them.

This fictional person made it all the way to retirement and good for him. But the guy who followed this strategy for 15 years and now is job hunting because the company needed to cut 18% of its staff? He's got a huge gap in useful skills and decades left before he can retire.

That guy is in real trouble.

This is the real reason I put moderate effort into staying current: I've interviewed too many guys with 15 years in the field who can't write actual code at all because they only have written a few dozen lines ever. They are a sad lot and it's embarrassing to even interview them.

If I could be guaranteed I'd have the same job forever with the same tech then who cares, but I've never met anyone for whom that was true. The people that don't learn either retire early or are forced to switch careers.

They worked the same year 15 times, or maybe the same two months 90 times.

Post hoc ergo propter hoc. They aren’t incompetent because they worked the same job for 15 years. And they probably weren’t even laid off because they were deemed incompetent.

You just got unlucky enough to interview them instead of someone who got laid off after 15 years and were amazing at their job and could pick up a new tech they’d never heard of and immediately be off to the races.

35 years in a single easy role with good job security, plus an early retirement.

Arthur was happy.

> The struggle itself towards the lack of bugs is enough to fill a man's heart. One must imagine Arthur happy

My favourite passage from Camus' little known book Software Development is Existentialism

In our company we do medical devices. It is a hard SDLC business. Once a colleague of us left and shifted over to another (much bigger) medical device company. I met him some years later and he told me, retroperspectively, our company is a high-performing, super agile group. His new employer needs a year for a two line code change without really changing but only adding something. And that despite him pushing and pushing.

To be fair: to modify blood dialysis systems is no joke business.

AT&T 5ESS switch had a similar atmosphere back in the day (no idea about now). The rule was no function deletion so you delete the code in the function instead. I remember reading about a very talented guy from Bell Labs being on the 5ESS team and writing some ridiculously small number of lines of code in one year due to all the rules in place to prevent major telephone system issues.

There is a lot of analysis that goes with it. I worked in both medical devices and also transportation.(Rolling stock signalling systems). It is definitely not a joke when you are moving such huge mass of metal with people in it.

This is like the programmer equivalent of WH Auden’s poem, The Unknown Citizen [0].

When I first read this poem I thought it was sad and about a person who wasted their life on mundane, average things. But now I think it’s a positive for a person who lived an average life on the outside but likely had a rich internal life that they didn’t share with the world and that we shouldn’t expect people to have metrics that we can externally measurable as to whether they were great or not.

I think it’s fantastic they this man’s passion is Soylent cat food hybrids for human consumption. He worked all those years and finally broke free for his passion.

[0] https://poets.org/poem/unknown-citizen

Also : Sisyphus.

The concept of life as meaningless, to be happy with the drudgery, or repetition, instead to be happy with the 'journey', to be in each moment.

Very common throughout literature/history.

I'd like to think we are somehow different, that modern man with technology is reaching some new level, to find some purpose. But now, not so sure, it's still just day-in-day-out, then you die.

https://en.wikipedia.org/wiki/Sisyphus https://en.wikipedia.org/wiki/The_Myth_of_Sisyphus

I'd argue that modern technology takes us out of the moment more than anything.

That sounds like a well above average life. No problems and a functional member of society in every respect? Good relationships? Five children?!

It seems obvious this guy was in fact happy, despite the last lines of the poem, or else he would have acted out at work, become an alcoholic, divorced, or whatever. That or a supreme stoic.

some old verses that rhythm with the current zeitgeist

> Our researchers into Public Opinion are content

> That he held the proper opinions for the time of year;

> When there was peace, he was for peace: when there was war, he went.

If this is a jab to supposedly mediocre programmers, it fails to be funny.

I have a friend, let's call him AP, who is a programmer in a local java shop. He's been working in the same place for 12 years since he graduated. He kind of fits the description on this article, but he is one of the happiest people I know.

He gets paid above average wage, and his work is easy and not very demanding. He knows the codebase inside out, and gets along well with his colleagues. Outside work he goes out cycling or for a beer with his friends or travels the world with his wife.

Different people have different priorities, and some are not obsessed with excelling at work.

Author here!

I agree with you :) here's some more context

[0] https://news.ycombinator.com/item?id=36985316

It's making a jab at "minimum effort" employees who manage to coast for years or decades. Note the "hundreds of lines of code".

There are many fantastic engineers who work on otherwise boring, rotten codebases. No shame in that.

Everyone can build something new and shiny on the green field. If it doesn't work out you can always find someone or something to blame, because what you set out build was an awesome, highly innovative[1] endeavor anyway. Moving on to something else new and shiny after a while is not only ok, it is expected.

Sticking to something, through good and bad times, on the other hand, that's not something many people can do, and while it is by no means sufficient in most cases it seems to be necessary for sustainable success.

Examples: Guido van Rossum[2], Linus Torvalds, Daniel Stenberg

[1] You wouldn't even have to put the "high risk" nature of it in the fine print, because it's readily understood.

[2] I could not find the source but I remember him saying in an interview when asked about Pythons's breakthrough, that it never had one but that it constantly and steadily grew.

You're examples have very little to do with an average org's IT projects. Not in scope, definition of 'success', or definition of 'finished'.

If you're company is building a programming language or operating system you need to make sure that is what you love doing.

My point is that grit is undervalued. It exists within and outside organizations but if I'd tell you my examples from inside you wouldn't know them.

Also perseverance is not the same as being stubborn and pivoting is not giving up.

starts promising but ends unnecessarily snarky and condescending.

why dumpster diving? and those guys cannot be replaced by any number of junior devs - that is exactly the problem with them retiring. they know the legacy stuff nobody wants to touch but everything depends on.

so, from that angle this writing reflects very poorly on the author and his work experience.

having said that - the title could fit just about any lawyer. law is the crappiest codebase that you have to spend a lifetime learning. especially the german tax law - but it guarantees a very comfortable income.

also in defense of people like that man i'd even argue that mastering a legacy codebase is mentally and motivationally not that different form mastering a game like chess.

Maybe dumpster diving is extreme, to drive home a point.

But, there are a lot of old people that are hungry, and work as Wal-Mart greeters.

If you didn't save/plan well enough, and 30 years of cool software development ends as being a Wal-Mart greeter, that can seem equivalent to dumpster diving.

guys from that generation (with its past financial opportunities and real estate prices) and of a character required to even work like that - humbly and diligently maintaining a complex legacy code base, very rarely fall victim to financial issues when they retire. those men (and women) are usually very zen and collected. loneliness and suffering from a feeling like no longer being needed is probably the biggest (mental) threat. but the aforementioned personality traits will provide for developing new opportunities like voluntary work or maintaining some open source repo.

don't worry about them ... worry about yourself (if you are young and in IT) and especially about those poor junior devs put in place for this guy.

If you are the kind of person who thinks the future of IT looks bleak right now, imagine what it must have looked like in 2002 when NASDAQ was in a three year death spiral and down 80%, which is an experience this guy lived... and then in 2009 when the stock market was down at levels previously seen in 1997...

If anything, the lesson should be not to worry about yourself because the only way this man could be forced to resort to dumpster diving is if he was given too much to worry about his retirement accounts.

Parent and great grandparent comments are correct. I'm sure the author of the blog had fun with that, but it's fiction. These kinds of pragmatic guys are not the ones you have to worry about, they will be ok. I think they included it to take the steam out of the obvious counter to their snark: Arthur got paid well for his work and did alright, so who cares about the sniping and denigration?

>It’ll take two junior devs and a Keurig to replace him.

Also out of touch. You can throw N juniors at this kind of codebase and without the mentorship of an Arthur type, they will get nowhere. They'll do a "rewrite" with 1/10 of the features.

I'm in medical device software. I'm mid-way through my career (15+ years). I've been on my current codebase (which is 20+ years old) for 5 years. I work with a coworker who's been with this code base for 20 years, but is 45 years into his career. I push back hard when it's suggested that we bring on a fresh college grad (who wasn't even born when this codebase was initiated) and it's suggested to give him/her one of the harder tasks with no supervision. Management: "They have a degree in computer science, why wouldn't that be enough qualification?" Me:"For the same reason you don't name a recent MBA graduate CEO of our fortune 500 company".

Note: I'm all for hiring recent college grads, but they need to be paired with a senior dev, and management needs to understand that you're not speeding up the project this month. Ideally, you're investing in the robustness of your team. The challenge I face is that so many junior engineers are job hopping the first ~5 years. That's not a smear on them. It's good for them. But it's not great for me...

I would dispute that the person that plugs away at a legacy code base, does not necessarily translate to someone that is also good at personal finance.

They can actually be quite opposite, and finances are easy to let slide, ignore as the days go by, while you are zen'd out on some cool technical problem.

There is something to be said for stability. Since our protagonist retired early, the pay must not have been bad. The glass is half full, or so I tell myself.

I thought so at first too, but the planned "street performance and dumpster diving" tells a different story.

I know people who enjoy street performance and dumpster diving. Bringing your juggling balls (or guitar) to a street corner and trying to make money gives some pressure that makes it fun - and that you don't need the money means if people don't like you enough to pay it doesn't matter. Likewise people throw away a lot of treasures that you can fix up and sell/give away.

If he needs to be a street performer and dumpster diver to make ends meet that isn't good. However people do those things as a hobby all the time.

I worked with a guy who made easy into the 200k range in the early 2000s two separate pension plans. Total dumpster diver. He loved the thrill of finding stuff. Wore complete rags at work. Drove a car from the early 70s. Ate simple homemade leftovers ever day for lunch. He loved the idea of racking up a high score.

True, but people usually don't start with this on retirement. And the end of the story is about him planning to eat cat food(?). Not sure how to take this, as it also talks about Soylent and Premium Mix, which are not the cheapest.

Retiring at 58 would be awesome. Some people live to work and others work to live. I like what I do but I also have hobbies which I really enjoy.

I get that the post is satire but at some point in your career there are diminishing returns on learning new things. If you are somewhat specialized say in backend systems involving scaling, message queues, clustering, etc. it probably isn't a good time investment to learn whatever is new in the frontend world. And by learn I mean to become proficient professionally and get a job doing it, not just a "hello world" learning. Doing "hello world" learning on a technology is like learning how the yolk and rudders work in an airplane, you're not going to be flying professionally anytime soon.

Learning new things is how you keep your mind working. People who sit around doing nothing in retirement tend do die sooner than those who keep busy. What you keep busy with doesn't matter, just do something challenging.

Agreed. I feel the need to learn new things just to keep my learning muscle strong. Sometimes I actually need to learn a new thing, and if I haven't done it in a while, it's scarier.

Strong agree. There is enough to learn in one's professional career that keeping up to date on tangential technologies may not be worth it until you have a need.

Having hobbies where learning and applying things is not just fun but can also scratch that itch to keep your mind fresh.

Howdy, that's almost me! AMA!

I've spent 10 years as the lone dev at a small medical facility, writing database software and dealing with lab hardware etc.

I guess I've touched 100% of the codebase because I wrote it all, but I always wonder if my skillset is so niche that I'll always be at small companies.

Here's the deal though, work isn't everything and I'm at a job that recognises that.

You wrote it all? By yourself? That sounds ideal to me. The horrible part of this to me is that he inherited a mess and left without having done much to it, evidently being unable to do so for whatever reason. The headline says he "mastered" it, so clearly other forces were at work than his skill. That's the depressing part to me, pulling my hair out to try and understand someone else's antediluvian code base which was built under duress.

I think devs imagine someone else out there is having a better time at their computer than they are. I definitely do.

It also said he only wrote hundreds of lines in 35 years. So maybe one line a month. Either he's a wizard that spins gold from straw or "mastered" is being used ironically. Given that he could be replaced by two juniors I'll assume the latter.

Aah so Arthur is the guy after me!

I envy you. How did you get I to this gig?

One of the scariest short stories i've read all year.

For $666/day the author also sells infinite exponential growth. [1]

[1] https://taylor.town/hire-me

The site says $1000 for me. Regional pricing perhaps?

> Estimates

> 3m $60k double your daily active users

3 months is ~60 work days assuming a 5 day work week.

I think you are correct. It seems like all of the other prices align with the weekly rate as 5 days. It also makes the daily rate seem more reasonable, at a yearly rate of $240,000. In a better interpretation of this, I could have maybe also assumed that this is just a fun way of stating something like "hiring to work on doubling users costs this much, but there isn't actually a guarantee that the userbase will be doubled."

Page says 1000 though. Anyways this is crazy, who he thinks he is lol

That's a varied set of stacks they're comfortable taking clients for.

This is where I am heading. The article is motivating me to change that.

I don't get these threads, it's impossible for everyone to do a job that they love and that is meaningful ... it's the ultimate luxury. Try to find an alternative way to give meaning to your life. (I have not yet succeeded in doing that by the way ...)

It's just their minds screaming "oh no that's me in six months!"

For decades, Arthur raised flags to upper management about the high complexity of the codebase, and begged them to hire a couple of junior devs. Only then could he delegate enough work for him to take on the refactoring project. The software's uptime and stability - he reminded them - was critical to the survival rates of children stricken with spinal meningitis. But management churn and budgetary chaos at the org resulted in the project fluctuating between "investment" and "divestment" states for decades, leaving him to fend for himself.

He told his manager that the data access layer - consisting of 5k LOC batch scripts calling Microsoft Access - needed to be brought up to speed. Maybe they'd consider PostgreSQL this year. Containerization, he protested, isn't going to fix the problems inherent to the system's labyrinthian integrations with countless software vendors and proprietary components. He also reminded them that it's 2023 and Windows containers still don't support RDP.

They still needed a replacement for "Bob". Bob left the org back in 2008. Under duress, and impossibly tight sprint deadlines, Bob was asked to make surgical modifications to the WingdingsAbstractFactory.java class - to be deployed directly into production no-later-than-end-of-sprint. Bob complained that there were no conceivable DR plans, or QA engineers assigned to the project. After a ten hour deployment - and a highly blameful postmortem - Bob exited on record-short-notice.

Arthur was forced into an early retirement under the recommendation of his physician. It was said that his stomach issues were brought on by the stress at work. Only Soylent and tinned meat were within tolerable levels of discomfort.

Arthur will be remembered by his loved ones for his patience, steadfastness, and creativity.

I've done something like this for a couple of years, albeit not full-time. The project was written in Symfony 1.x and got stuck there, as Symfony moved on to higher versions and wasn't backwards compatible with 1.x. The codebase consisted of thousands of lines of poorly documented and commented code, intermingled with custom JS modules that nobody understood. When management wanted to implement changes, it would take days to comprehend the potential effects on different parts before I could initiate any modifications. The project would have required a complete rewrite, but the management was staunchly against it. This has been the most frustrating and unsatisfying tech work I've ever undertaken, alongside past tasks of migrating MS Access apps.

I sometimes find myself pondering why I didn't cease working on it, especially given the compensation wasn't particularly impressive. It's crucial to recognize and act when you sense that a project is no longer benefiting you, and simply move on.

I love that your response to this article is to try to prove you're that guy.

We get it. You're that guy. :-)

That is one extreme. The other one is developers hopping from new project to new project without ever even seeing the product going live or getting user feedback/bug reports from the live product.

This is quite funny in a painful way. I identify on a few different vectors. I started my "professional" career in telecom. There I really did become an expert on some very small percentage of code. It was exhilarating to do patches in 286 assembler for an obscure language (CHILL). Now many years later, I work in NPD at one of the small divisions of a giant Medical device company. I wrote 90% of the legacy code. It is embedded and I have not needed to update any of it in 10 years. With changing management styles, the new push is to buy new products from smaller companies, and just to the required paperwork for regulatory approval.

It is satire but I think people like this are heros. Not doing the flashy fancy stuff but keeping the world ticking over nicely.

I don't think it is satire. Or reality has become the satire. This seemed entirely real to me.

Poe's law strikes again!

Hadn't heard of Poe's law. I'd say the best sarcasm does contain a lot of truth. Think it used to be that the best sarcasm did leave you wondering a little if it was meant to be sarcasm.

Interesting idea. It can be that everyone has different knowledge base, so everyone has individual interpretations.

For the post in question. I've seen it in real life, so it didn't seem 'extreme' enough to be obviously 'sarcasm'. Though, I could tell it was sarcasm, it was perhaps the good type of sarcasm in that it contained a lot of truth.

There will be a lot of these people maintaining the currently fashionable microservice/multiple languages/use as many AWS services as possible/scalable systems in a few years.

This is my retirement plan.

I have a friend who just started a job working with AS400's, learning RPG. He's not much of a programmer, but I am, and this made me think, hmm... maybe I should learn it too. There's going to be some serious job security in that field, once AI is writing all the CRUD web apps from scratch in Nodejs.

I 100% believe crud will be automated

I’ve tried to look for the COBOL and other legacy languages jobs where there is a shortage for. I came short. And the few I found didn’t pay like there was any kind of shortage. There was a lot of willing offshore sweatshops offering their services though.

There is something admirable about someone who commits to maintaining a codebase, no matter how crappy it is. We don't see this often in tech and often being a person who is more reliable and stable is more desirable than two junior devs and a Keurig machine. Speaking from my first hand experience when reverse engineering a poorly documented product for a small company these people were the real heroes. Thanks to all the "Arthurs" that keep the world spinning.

I found this on his site https://taylor.town/-10x, I like it even better

My boss straight up told me a couple of juniors could do my job the other day

Find another job

Most coding is pretty mechanical... and most juniors can pick that up easily. In spite of the mysticism surrounding software development, higher level languages are actually intended to be easy to pick up. If you have good problem solving skills, those are harder to replace with a couple of juniors (or ChatGPT).

If coding is mechanical, then the code will often turn out to be overheady, uninspired suboptimal solutions á la everything is a class, not well extensible and maintainable, or the task at hand truly not fit for the skill of a good developer.

Coding is a craft. There is an art aspect to it and there is a lot of room for inspiration and ideas to improve things.

> Coding is a craft. There is an art aspect to it and there is a lot of room for inspiration and ideas to improve things.

True but often enough great code is not needed just working code will do just fine.

I honestly never found a private codebase in any of my jobs that was great and clean and arty.

I don't disagree, but in my opinion code organization (which is what most of these things are) is mostly subjective, in other words, beauty is in the eye of the beholder. Good organization certainly helps with maintainability and extensibility, but it's not a prerequisite, and it won't on its own solve tricky or vague problems.

Maybe he tells those juniors that GPT4 can do their job.

Danger, Will Robinson! Time for a new boss, possibly new company.

It only took me 2 years in this "new" old company to become irreplacable! I really have to hop.

Leave and make him prove it.

Question: Wouldn't a couple of juniors be more expensive? I don't think I ever work at a company where junior developer was paid less than half of a senior developer.

That's me, got fired 4 years before retirement. Made them rich. Young guys came in, it all had to be rebuild in microservices. Last press release, they have tripled their staff, financial turnover went down, the new system is working but is lacking the functionality of the old systems. All the pitfalls i warned them fore and got me fired, became reality.

How did Arthur avoid promotion to management?

They couldn't hire any replacement VB6 programmers.

He was sufficiently good at programming:

> Westbrook once touched legacy code without ending the company.

This strategy doesn't work, I did this twice and then got "promoted" to Architect so now I sit in meetings >24 hours a week.

As long as it's not >24 hours a day, there is still room to grow.

usually "architects" are just people there to keep the communication rolling and make sure devs or teams don't work in silos all their life.

probably your communication skills were too good for management to passt up the opportunity of promoting you.

Programming is a job, it is not a mission to save the universe. A few programmers become famous for a combination of talents and luck, but nobody can control either of these things. In the end, 99.99% of programmers do the same things that this guy is making fun of.

Life is too short for working with crap code. Unfortunately there is more crap code out there, than there are opportunities of working on beautiful code. At least in most jobs there will at minimum be compromise and part of the job will be dealing with ugly code.

Most developers see and work with crap code because good code runs well and rarely needs to be touched. At least that’s what I tell myself.

Makes me wonder how many of us work on things, that are actually relevant and not just reinventing things or adding to future dystopia.

Now that I'm older this sort of comment actually makes me laugh. Thanks for that. Good code.. oh, sweet sweet summer child.

Nothing like working on crap code that past you wrote.

Life sucks and then you die, I don't need constant reminders of it from the internet though.

Recently on a call my client introduced me to his $BigCorp whale client as "my IT guy" and his client said "oh yeah I have 12 of those here on my team".

I'm cool with that. I keep doing work valuable enough to pay well for, and the rest is semantics.

This is me.

30 years grinding it out on a very small esoteric system.

Hard to explain to people "no, really, my skills translate".

Paid well enough, but can't move, locked in.

But, the small esoteric system is being replaced, so I'm running out of runway and might not make it to retirement.

This is the dream imo. I love this kind of work

His manager noted, “Arthur had a knack for working hard without learning transferrable skills."

^^^This!

Arthur was lucky to retire at 58. He probably made fat bank which turned out to be golden handcuffs. If he ever got downsized, he had been f*d.

> He probably made fat bank which turned out to be golden handcuffs.

This is a big assumption, not everyone in this field is a high earner - plenty of people make okay-ish salaries without being anywhere near highly paid.

He retired at 58 so I'd assumed. Maybe he's got a side hustle or embraced the super frugal lifestyle.

That's what Dan did at my old job. Retired at 50. We would go on per diem travel and he would bring food from the hotel. We called them 'Dan-wiches'. He was smart and frugal. Now living his best life.

That's not how it works. Those who've been there 20 years and who would have difficulties finding another job are those who get low salary increases. On the other hand they've been there for so long without looking elsewhere that they probably have no idea that their pay is low.

Wasn't this a bit of a self deprecating summary of his own career?

Many of the comments seem to assume that it was written by his colleagues and that the description of his career is condescending or snarky.

Personally, I find it really funny, and that it probably describes many of us, despite the fact that we think we are way too cool to ever be like the character described in the piece.

Most of today's new tech will become tomorrow's 'legacy'. Imagine how your awsum multi lambda hairball deployment will look in 30 years. (If it's even around in three decades.)

"Most of today's new tech will become tomorrow's 'legacy'" Only the successful ones

This is how it should be on salary. If you're good enough then you can get time back from the job and work less per week. It starts to look crazy good when you are doing like 5-10 hours per week because you're so familiar with a code base and have a decent understanding of the business. If they aren't going to give meaningful raises for doing maintenance, then time is the only thing you can get more of.

If companies want people to work harder, they'll have to give more compensation that scales better with performance.

The idea that two interns are going to be able to take over for this guy was the funniest part of the article. Maintaining a large old legacy code base is a hard job because there is so much to learn before you can even start. Codebases like that live that long are often used in important places where a YOLO approach of “test it in production” can cost tremendous amounts of money or even lives. The article makes fun of the guy for only committing hundreds of lines of code, but they worked perfectly the first time and every time.

> Westbrook believes he grapsed over 4% of the entire codebase

He's doing well

Einstein once said "Once you stop learning, you start dying". I feel like in IT it's the case for most what people are thinking. In my case I have been switching tech because I felt I wasn't learning anything new or simply because it's was dying. I feel like these guys like him are probably too confortable in their confort zone or too lazy to learn anything new (the less you do the less you've got motivation + you can't teach old dog new tricks).

To be fair you can learn outside your job. I wouldn't call him lazy just because he stuck with a company for a long time. Most jobs will only require a small percentage of a programmers' knowledge, so people supply their urges with outside endeavors. Hence reading books, doing Open Source, side projects, etc.

Lots of significant inventions in computer science began as a side project only the creator cared for initially and had to borrow his own time to do it.

This is quality satire :) it also works somewhat for larger codebases, even non-crappy ones.

> Arthur had a knack for working hard without learning transferrable skills. He will be dearly missed

When I was at Google during the pandemic, I noticed more and more people saying they needed to learn open source tooling so they can have skills that transfer to the outside world.

That company is full of brilliant engineers who have mastered (part of) a great code base but have to start at the beginning with popular non-Google tools.

Nothing against Arthur Westbook and his career choice [;)] but I think the proper way of turning mastery of a crappy codebase into a career is joining a bug bounty program. Some are big enough (think Yahoo's scope) that you can earn a living mastering them and finding bugs. But won't those bugs dry up eventually you ask...well thankfully new features, API endpoints, (sub)domains etc. are intruduced constantly.

I'm unfamiliar with bug bounties but how often do you get to peruse codebases? I thought most of the time you're working in a blackbox setting with just an outside interface that you peek through?

Honest question: how many HN'ers have worked on a codebase they considered non-crappy?

Or, put a better way, for all the codebases you have worked with, how many were crappy and how many were not?

I'm increasingly convinced that there does not exist a large codebase in actual use that isn't crappy in some way or another.

I mean, a lot of people work on things that, despite being billed as a Great New Thing, are throwaway projects that will disappear without a trace in a few years (once management changes, the market turns, or the founders/VCs cash out, or the SEC comes knocking, etc.)

At least a 35yo codebase probably actually does something more useful than sell a narrative whose purpose is to enrich a few random jackasses.

Often times great ambition requires moral concessions that are untenable even in the face of the end result.

Politics is a helpful lens. Either you compromise and muddy the purity of the result, or you take the high risk path of being uncompromising and greatly reduce the odds of achieving any result at all.

When facing this, many individuals optimize for lower-risk self-fulfillment over great ambition.

This type of cynicism/snark used to be a lot more funny before it dominated all internet discussion and interaction. Maybe the people who spend all their time online are the bitter/angry ones and it just has taken over all discussion?

In any case it's just a job, and anyone harboring illusions otherwise will learn the truth soon enough.

I got bad news for some of the folks in this thread: just about _everyone_ can be replaced with two junior devs and a Keurig.

From 1997 to 2012 I was the sole IT guy at a trade show company. We had a database called Smiley.... I wrote 3 different replacements for it, they were always afraid to convert over, even though I was fully prepared to run them both in parallel.

I've been this guy, sorta.

I've been afraid to even try personal projects since then... that job broke me.

I left my previous job because it was obvious this was my future (or would be my future if the company could stay in business, which was unlikely). My mandate was to fix all the problems without changing any of the founder's work that caused the problems. My new job is terrific.

"Two junior devs and a keurig" - that will totally wreck the system by refactoring it to "modern patterns" without bothering to understand what it actually did. But it's OK, they're younger and therefore better. Thus, the cycle begins anew.

This article may and should be the trigger for thousands of life-affirming resignations.

I think that was the point, but see all the sympathising comments here. Being full-on driven by your day job is only one way of having a good professional life, and I wouldn't say it's the best or the most common by far. The way how software development is democratised and slowly becomes the next blue collar job, the engineering hero ethos will fade.

Unrelated: this link opened almost instantly, what’s this magic? Even HN takes longer!

If you serve plain static HTML from S3 or Cloudflare Pages (or even a vanilla nginx on a vps) and put a decent CDN in front of it (and don't load 3p resources), it will astound people how fast your page loads. I regularly get this feedback on my own webpage.

It's not rocket science - it's how we made webpages in 2001. Browsers are really good at that.

I'm worried this will be me someday. I suppose that's the point.

Does somebody work 35 years in same company, it is usually 2-5 years -> new company grind rather than working at same company.

Yes, I know many people in software/tech who have been 20+ years with the same company. Hopping every 2 years is not common.

That said, it is a red flag for me if most people in the team have been there for very long: it means that as a new comer I will always be bottom of the pile and have no career growth opportunity.

That’s a common story on HN but not at all a reflection of real life. Be careful letting the internet convince you of something without doing your research. There are lots of us with deep tenure at a single employer. It would not surprise me to find it’s still the majority of professionals.

I've seen it. Some people are weird and just want a stable job that pays them alright so they can focus on their family and other things they care about.

A few times in my career I've seen people with a decade-plus experience leave companies they would have stayed at for another two decades if it were within their control. It's sad. They struggle a bit because they often realise that they've over fit their skillsets to that single company.

One guy I knew was hired as a "web developer" in ~2005 because of his HTML/CSS knowledge. He continued to do basically just HTML/CSS until he lost his job in 2020. Lovely guy, but he saw his job as just a job and couldn't give a damn about learning new technologies. I often wonder what he's doing now... I doubt just knowing HTML/CSS is a hireable skillset anymore.

Why are people weird for wanting to "focus on their family and other things they care about."?

In the rest of the world they aren't, this is HN though.

Context is always important.

> Some people are weird and just want a stable job that pays them alright so they can focus on their family and other things they care about.

What's weird about that? It's how things work in every other sector, why would our sector be any different?

I wouldn't want to be stuck in the same position for decades, but that's different than being at the same company for decades. If I found a company that treated me right, paid me enough not to worry about money, and allowed me to progress internally (at a reasonable place), I wouldn't complain.

I had about 15 jobs over 40 years, not including 9 years at the two companies I started. But some were short term contracts, and several (in a row, sadly) ended in my employer going out of business, mass layoffs, etc. I always found doing something new was revitalizing. Also found that sometimes something new was awful. Being comfortable changing what technologies you are expected to use helps, taking advantage of new tech being new, and not many people knowing it, is good too. At my last job before retiring I stayed for 5.5 years, and could still be working there today, except it was enough of a career.

If I don't have easily-transferrable technical skills, how much additional value can I offer to a new potential employer in any engineering (non-management) role above junior or maybe middle-ish? I obviously recognize that there are other areas of skills, and that learning new tech gets easier with experience, but isn't there a limit to that?

I'm that guy at work. I solved the problem by completely rewriting our huge and aged out internal application. The plot twist is that I also completely wrote the original crappy codebase.

actually terrifying. feels like the onion. let me laugh it off hahahaha

If he already mastered the crappy codebase, then "two junior devs and a Keurig to replace him" or even a senior dev might not be able to replace him in a short amount of time.

The things people do for "job security".

This was a funny title but not a nice post. Ick.

Is there anyone who is accessing this page via a command-line text-based browser?

Spend time digging deep into something, or spend time skimming the surface of a bunch of things, I guess.

Why does Taylor have to play Arthur like this, poor man will need burn cream after this blog post.

I thought it was the conclusion for every single developer in the world when they retire, no?

It will be interesting to see how the average HN'er is doing when they're at 58.

Alternately, Man Earns Enough to Comfortably Support His Family and Retire Years Early

I feel personally attacked

mirror: https://archive.is/12eOI

The headline reads as if it was from The Onion.

And for this, I spent six years of college...

Finally something funny for once! thank you.

That THAT is the internet as she is spoke.

Would be a dream to retire at 58

You gotta do what you gotta do.

I just learned ASP Classic 3.0.

Is this humor LLM generated ?

Bit of a threadbare post

I am Mr. Westbrook. AMA.

Guaranteed to be C++

This checks out.

Ctrl+F the comments for `dailywtf` ... no results. Sad.

I am this guy

Harsh.

Reply on news.ycombinator.com