Simultaneous Shortage and Oversupply (2019) | Hacker News Reader

Simultaneous Shortage and Oversupply (2019)

jefftk.com

32 pointsluu22 comments

Having had applied to and even interviewed at some of the companies/jobs listed in the blog post, I will provide an alternative explanation:

These roles are just like many others in the field, companies are supposedly desperate to fill them, but set unrealistically high expectations for the hiring bar so the role is never filled.

Context: 6 years professional experience, open to moving anywhere in US, experience at two SV unicorns, fine taking a reasonable pay cut to work on these things, still rejected or never heard back from every job of this type I've applied to/interviewed for.

I came to say something along these lines. Why would ea be any different to the rest of the tech sector? They want to hire someone with 10 years experience for the (sector adjusted) price of a recent graduate, and the interview process is unreliable as ever so when the perfect candidate walks in the door they probably fail to recognise them anyway.

Having been in a form of ea myself for around 10 years I'd also add that it takes a long time to start being effective. Lots of skills besides tech to learn, which will be very specific to whichever form of altruism you're involved with. So although I haven't looked at those job postings, the simultaneous over and under supply could possibly be explained by skills mismatch.

Not sure if you've advertised it as such to these potential employers, but you should not reveal your hand (that you're willing to accept a lower comp than what you currently earn)

As a similar but "dual" experience: I am a very desirable candidate in my specialty. My resume is really really good looking, to the point where jobs have commented that they were worried I was fake. I've never been turned down for an interview, and only turned down for a job or internship once.

Having been offered positions at a number of companies who "really need" someone with my skillset, a lot of these places are shockingly inflexible on pay. I make 3-4x what (I infer) they normally pay people in nominally similar positions, and they seem absolutely unable to match that for procedural reasons, even though based on their description that seems like a reasonable investment for a specialist they really really need. The companies I end up working for are typically outsize successful, and I attribute a lot of that to their willingness to fork over some extra cash when it makes rational sense, and shockingly few companies in SV fit this description.

There are reasons to be inflexible on pay. I imagine it works enough of the time, and a candidate will accept the lower pay. I hypothesize also that inflexible pay policies make it more difficult to be accused of discriminatory pay practices.

The problem is that expectations and accepted norms in software engineering are out of reality.

A few factors involved here:

1- The 'Full Stack' Developer

You'll often see companies looking for 'full stack' developers, and in their mind they are expecting someone who is a master of both the backend and frontend.

But, that is, in fact, exceedingly rare. Most people will gravitate towards a part of development that they are most comfortable in. That's not to say that a developer can't be good at both, but the number of people who are great at both is a small group indeed.

Node.js is a symptom of this problem IMO: Whatever you say about JS as a backend language, the drive to have web developers to not have to 'learn another language' to do backend work is fairly obvious.

So, rather than looking for a mix of team members that are good at either backend or frontend, they look for this godlike 'full stack' developer. What they often get instead is whomever is best at convincing them they can handle it all.

2 - RDD Driven development

Oftentimes, rather than picking the existing tools/paradigms that are right for the job, the tools marketed as the solution to the problem are instead chosen. And of course, the expectations for those marketed tools wind up being out of whack (i.e. the recent job posting looking for 12 years of Kubernetes experience.)

3 - It's easier/better to get a better job somewhere else than an internal promotion

Many orgs make it difficult to advance, or advancement doesn't have the same gains as taking a similar role elsewhere. As a result, people often leave, and because of corporate/cultural norms (screw offboarding, we need to finish the sprint commitments!) knowledge loss is a result.

4 - The Second order effects of the above

Basically, the points above feed into a vicious cycle: The developers better at bullshitting (that also realize the above) will produce RDD driven projects and hop to the next space. But because they have spent most of their time studying 'persuasion' rather than good development practices, they leave a huge mess, which only makes the hiring manager's expectations that much more unreasonable.

What is EA??? The answer is really simple: oversupply of mediocre talent, and shortage of top talent.

Given the mention of MIRI, I'm pretty sure EA stands for Effective Altruism.

There's the problem. That niche is so tiny that few outside it know their acronym. And it talks to itself so much that the people inside it think people outside it know their acronym.

Agreed - this article would have been a lot better if the author defined what they meant by "EA movement".

It broke my reading flow to have to go and look the term up.

Effective altruism. I had no idea that was a thing.

Also, I don't see what's particularly altruistic about working at Deep Mind or OpenAI - they pay pretty well from what I know.

There probably is a genuine shortage of programmers willing to work full time at a socially or environmentally oriented non-profit in the Bay Area.

> they pay pretty well from what I know

The core point of effective altruism is to maximize the positive impact you have on the world. That doesn't really have any correlation how much you're getting paid.

> There probably is a genuine shortage of programmers willing to work full time at a socially or environmentally oriented non-profit in the Bay Area.

Would that be more impactful than working at high paying job and then donation that money somewhere? I doubt these non-profits are constrained by their lack of programmers.

Leaving everything else aside, the author seems to ignore a glaring point that in part solves this apparent paradox: everything is a huge game of musical chairs.

Imagine that everyone he knows who want to change jobs go and filling those openings. Brilliant, now the other companies new to open new positions to fill those who left. In big companies, particularly in places like SV, the attrition rate is so high that they need to constantly hire just to keep their numbers even over time.

People are less interested in doing the thing they said they're going to do than they are in saying they're going to do it. Examples:

- "I'm going to start working out"

- "I'm going to 'learn to code'"

- "I'm going to follow through on that thing from Craigslist"

- "I'm going to meet up for coffee"

- "I'm going to change jobs"

- "I'm going to find somebody to fill this job"

Aziz on flakes: https://www.youtube.com/watch?v=_RbMv7HUiO4

A lot of the time, life intervenes. And a lot of the time, they say that just to shut their friends/family up about an issue.

Aziz talks about commitment-phobia, but on the other hand, one of the most frequently given personal growth advice I've seen in the last 20 years was: learn to say no. Stop trying to agree to every single thing someone proposes to you. I suppose they're flip sides of the same coin - say "yes" only when you mean it, and say "no" otherwise.

Not exactly relevant to this context, where the question is "[Why is there] simultaneous shortage and oversupply [of programmers for the EA movement?]".

People don't post job ads or tweet about wanting to work in EA because their friends and family are nagging them about it.

My answer to the question is that people have an idealized version of the person they'd like to be, and this differs from how much work they're actually willing to put in to follow through. This goes for both the people claiming to want to find work and the company (i.e. people) claiming that they're looking for someone to hire. In other words, there is a simultaneous shortage and oversupply because people are flakes.

"So, why don't these openings get filled quickly? " What made him think that ? Not sure I agree with this premise. OpenAI, Deepmind, etc have no difficulty hiring.

Isn't the standard answer that most programmers on the job market are bad, and those who are good are already employed and thus rarely on the job market?

The complementary scenario is also plausible - the employers that are decent have low attrition, write reasonable job ads, and fill positions quickly, so most of the ads (on a number x unit time basis) are from clueless employers with self-defeating criteria.

What is EA?

"The EA movement has a ton of programmers ..."

effective altruism

What the hell is an EA?

> the good jobs are all in AI safety,

Today's problems - like "AI" systems that replicate existing biases; decisions too opaque to check whether they're based on protected characteristics; and systems with higher error rates for some minority group - are about 75% politics/advocacy.

After all, no matter how good your solutions to those problems, you've got to persuade companies to adopt them voluntarily - or legislators to force them to.

Tough to find that skill set.

Reply on news.ycombinator.com