Why isn’t there a glut of good software engineers?
blogs.law.harvard.edu
blogs.law.harvard.edu
So the pool of great programmers is generally limited to those who love the job _so much_ that they’re willing to forgo income to do it. Someone with a “125 IQ” and strong math abilities could get a job on Wall Street, or as an actuary, etc and make more money for less frustration.
At Amazon, they target 75th percentile comp – but I really doubt they also target hiring 75th percentile developers on average. So they’re basically trying to hire 90th percentile employees for 75th percentile comp. I’d argue the entire industry does that in comparison to non-engineering industries.
The next time a CEO complains they can’t hire enough engineers, ask them how much over market they’re offering.
That's something that I would never give up for an extra 50k a year.
I think a lot of programmers don't realize how much some other programmers get paid once you factor in other incentives outside of salary.
Also: it's sad that beyond the comp issue, they're often sacrificing prestige, at least in the US. It's just as hard to become a qualified engineer at top firm, but the general public just doesn't find that as impressive as being a lawyer, doctor or banker. I know lots of lawyers and bankers for whom a big reason they're in that job was the prestige and ego-boost associated with it.
I think a lot of it has to do with people overusing the term "engineer". I know people with that work for the phone company that have lower skill jobs but still get the title "network engineer" or some such. Normal people can't distinguish between this and the hard profession.
Uh, the reasons government IT projects are typically huge wasteful boondoggles are not because programmer salaries are too high or programmer talent is mediocre.
Additionally, gov't is traditionally entirely unwilling to agree to strict requirements, so require vast overbidding
this number is just as absurd as the $327 million.
Think of all the things that can go wrong! Security, PII, etc.
I have to wonder if it would be come a more regular market if companies weren't required to drug-test all of their employees in order to do more than $100,000 of Federal contract work [1]. I'm sure their are companies out there that would prefer not to compromise their employees' privacy for the sake of work they might not need.
[1]: http://www.nfib.com/business-resources/business-resources-it...
For certain values of "problem" as defined by the builder? Sure.
For what a buyer defines as a "problem"? Highly debatable.
Even in house construction, an activity that has been performed for literally THOUSANDS OF YEARS, there is no commoditization. If nothing else accounts for this, the bewildering array of local building codes assures that commodity construction as presented by a builder (like a national prefab builder) is really a business-level API that presents a uniform interface to buyers that hides a team that handles all the local idiosyncrasies underneath.
I don't expect a programmer that's been steeped in the world of UNIX daemons to write a good customer facing website on their first attempt, and I wouldn't expect a front-end Javascript programmer to know how to deal with multiprocess programming in C/C++. Asking them to do so may not result in something "with no problem". At least not in an acceptable time frame.
Sorry, but that analogy is completely wrong. I would recommend reading Fred Brook's "The Mythical Man-Month" (among many other books on software estimation) to understand why.
He argues that unlike activities involving physical mediums like house building, computer programming creates with an intractable medium. You would be surprised, in the field of programming, that many so-called "programmers" cannot even build a simple house that is stable on its own foundations.
So in short, that's not a valid comparison.
2) Programming is as much about communication as it is about writing code. People have this idea that you can reduce a program to a precise set of specifications that can be formulated by non-programmers and shipped off to China to be coded up. I find the idea ridiculous. People don't come with to lawyers with the precise contours of the case they want to build and simply ask for the lawyer to write up the briefs. They don't come up to doctors with a precise treatment plan and simply ask them to administer the medications. Programmers are problem-solvers, and that's a two-way street, and that makes shipping critical programming work over to China and India untenable because of communications barriers.
Yup... If this was actually possible, we could just write software to do it and not need programmers at all.
Rentacoder is readily available, its just that word on the street was most jobs paid about $5/hr.
Amazon's Mechanical Turk has some extremely fuzzy stats and not much more than anecdotes but earning around $7/hr has been reported. Higher pay, more fun, less effort?
Waiter/bartender at a tourist restaurant is probably the most fun and could pay even more.
Why is the market so fuzzy and illiquid? Why aren't there ten places I can go to find "write perl to import this old fashioned CSV file into that database" and see exactly what thats worth? Fix that and then wonder where the people are.(edited to rephrase there's no market, so no one plays the market)
You need a small amount of specific knowledge to start working (at a specific place), and leaving the job takes almost no knowledge away.
Software development is a very slow start, depending on the project.
Just try and see how long until a developer has the development environment. Yes, it should be a 'turn key' solution but that's often not the case.
1 - All projects can be run in a VM 2 - Setting the development environment is all a new developer needs
Let me try to instil some expanded vision into things:
- I can't have a VM for our specific embedded hardware with its specific devices
- I haven't seen a VM for AIX on Power (maybe except running on Power HW)
But most importantly, Vagrant can't put on the developer's head everything specific (s)he needs to know to do the job. Like how to use the proprietary utils that we need to integrate in a project. Or what the project is all about.
I hope you'll forgive me for misunderstanding "Just try and see how long until a developer has the development environment." -- The language is ambiguous, but "until a developer has the development environment" means software to me. That is, that the developer has the tools he needs to make changes to a code base. You're right that I may have taken too narrow of a view, but my interpretation also seems reasonable.
This would get rid of a ton of the risk of software projects. On the other hand, providing a really complete specification is nearly what programming is; by the time you have nailed one down you have done most of the real work.
Most of the market not only can't provide such specs, it isn't what they need. They need to pay for time in some kind of iteration process. Those who have pretty precise requirements and know what they need, are best served going to highly specialized vendors who have done the same kind of thing a million times and can therefore schedule and price precisely. But many of those already are selling that product for $29.95 or $300 or $3,995, with an option to buy extra services.
You likely already know where you can find code to import a CSV into a database to specialized requirements. Or at least you know who you could pay to tell you exactly what to buy. You just don't want to pay extra for these things.
More seriously, I suspect its a consequence of the 10000 hour rule applied to a discipline which builds some of the most complicated structures known to man. It takes a lot of training, formal or informal and a lot of good feedback to make a good software engineer. Plus, CS is not the easiest of disciplines to study.
That, along with the fact that good practitioners, in almost any discipline are by definition the top few percentage.
The bigger question would be, as the profession matures and builds up an institutional body of knowledge like civil engineering, would the number of 'good enough' engineers rise.
There are far less high IQ individuals in the world than the world needs right now.
We can teach coding to young people, but as of now we can't reliably increase IQ.
We have studies that prove agents such as piracetam administered to pregnant rats increase the intelligence of their offspring.
If the Tech titans that want cheaper labor really want to solve the problem of lack of quality engineers they could think long term and throw massive amounts of funding at neuroscience and nootropic research.
In the information age we all live and die by our wits. Policy and R&D money should be directed at raising the mean intelligence of the population through novel scientific breakthroughs.
Far easier though to just lobby for government policies that make it easier to import the rare kind of people Google, Microsoft and Facebook need.
So we have a situation where no one is focused on solving the root of the problem (a need to increase the cognitive abilities of humans) and no one is lobbying to protect the financial interests of working coders in the United States.
We need far better answers. The situation is only going to intensify as more low end jobs are automated.
I also find the suggestion that there aren't enough smart people to be able to do programming highly questionable. There are far more people who are capable but not able to pursue programming than there are people are who are not "fit" mentally.
Or hack whatever is broken with the educational systems around the world. I'm still in school (technical highschool, programming focus), and none of my friends that is worth his salt in programming is good in school, and that puzzles me because they keep telling us that they're desperate for good coders. D is one of the best js coders I personally know, and he raves about it all the time. He dropped out of the part of the school that I go to (which would lead up to college), and likely won't stay in cs due to that. B is one of the best php coders I know. He's also a D student. A's good too, but had a 2.0 GPA as well.
There's a fair share of good potential programmers around (even though my observation might be somewhat biased), but many of them won't ever get into college. And how do you face HR without a degree? Go to hell kid, we want credentials.
I'm pretty sure you don't need more leads, you need better conversion, or else we're all going into marketing.
Now, your friends may be more than capable at succeeding in university, and if a school only took the time to get to know them, they might see that, but, on paper, they look the same as someone who likely couldn't succeed in school (because they haven't, really, up to that point).
That all sounds all horrible about universities, disregarding these capable students because of an inability to really get to know prospects, but that's not on them. If you want to get into a good school, try harder.
As far as it defeating the point of getting an education, I'd wholly agree that college is not a worthwhile pursuit for a future-SE if your only reason for going is to get a job. You'll spend a lot of time in college learning skills that are not directly applicable to your future employment. You could arguably get to a point where you're good enough to hire more efficiently by skipping school. Whether or not it's worth passing on the things you really won't experience outside of school in exchange for a more efficient route to money is really up to you.
Can you code and are you at the level we need for the position? There are too many programmers that can't pass FizzBuzz but have CS degrees.
Our educational systems are broken yes, but what is even more broken are our memories.
I think that dramatically enhancing human memory is possible, and I think it's critically important.
IQ is relative. If everybody's intelligence went up 10 points, they would all still have the same IQ
The author is Phil Greenspun (http://philip.greenspun.com/) -- I'm pretty sure he gets what's required.
The competition is because the person hitting .400 is worth at least an order of magnitude more than a person hitting .200. The same discrepancy is not found in salaries, so you obviously prefer to hire from the top.
Accuracy is a better indicator of success for a baseball pitcher than velocity.
Perhaps Greenspun has "suffered" from working with too many great coders, or finds programming easy, so he can't understand why there aren't more great coders around. Maybe if he studied more sociology, or IQ statistics, or hell even economics (in the form of market demands) he'd be better equipped to understand why there aren't more great coders.
I know who Greenspun is, and I have a lot of respect for his programming chops, but this comes off as someone either disconnected from reality, or someone thinking that because they are expert at one thing, they are expert at everything.
Suppose that a programmer needs to spend 25 hours per week keeping current with new technology, getting coordinated with other programmers, contributing to documentation and thought leadership pieces, and comprehending the structures of the systems being extended. Under this assumption, a programmer who works 55 hours per week will produce twice as much code as one who works 40 hours per week
http://philip.greenspun.com/ancient-history/managing-softwar...
I think the problem seems to be (at least) four-fold. 1. They don't have the prerequisite cultural background. 2. They are self-motivated to pursue their own creativity but less interested in others'. 3. They don't have a fondness for excess money. 4. They have a hard time being obedient and subordinate.
Maybe controversial, but the way I see it.
The ability to "disagree and commit" is crucial for members of any team that want to both express creativity, but deliver results.
I think that there are enough developers. But everybody is searching for the geniuses that can create a competitive advantage.
Edit: disagree? Downvote if you want, but please counterpoint.
I think the answer is that the demand isn't actually very high. Programming is still considered a cost center for most companies, the quality of the product is determined by how much they are willing to spend on salary, not vice versa.
Sure, crappy companies in dying industries might not be able to compete, or might not even realize that they're failing to compete, but why care about them? They don't make the market for people who are actually good at software.
http://money.usnews.com/careers/best-jobs/dentist/salary
There are a lot of cities listed with low cost of living and $250k median salaries.
A friend noted "as engineers, we would do this stuff for free - it's just all the ancillary BS we get paid to put up with."
Here in NYC someone who understands modern web backends can easily be into the high $100Ks to low $200Ks.
There have been a number of articles pointing out the "myth" of the tech talent shortage. Perhaps the shortage really only exists for "top talent"
As the CEO of Contour it wasn't about pay for great engineers. It was about having an interesting problem to work on and a company culture they wanted to be a part of. Yes you compete on pay, but for a start-up, everything else you offer is as important.
Growing up I didn't even know what you could do with engineering. I loved math, but had no idea what you could do by being an engineer. I think a lot of kids don't realize the creativity you can harness with engineering and how you can "build" real things. It was always presented to me in terms of math and being nerdy. Just like writing, I hated it because it was about "grammar" and not about expressing yourself. Now I love to write.
For kids I think it is framed up wrong and until more engineers have kids and teach their kids just what you can do, we will be short software engineers.
I'm now trying to teach myself with codeacademy and it's hard.
I'd say this is exactly what my friends and I have done. We had been nerds growing up but the majority of us went into liberal arts majors in college. After graduating and finding out that our slips of paper were meaningless, a few of us taught ourselves how to program and a few of us built upon college IT jobs to go into system administration.
When we were younger, I think we generally had some pretty bad misconceptions about the software development as a whole, especially regarding relative difficulty compared to other careers. While the money and job security were the things that drove (most of) us in that direction originally, we've all become completely obsessed with a tech world that we generally misunderstood when we were younger. I've spent a lot of time trying to convince other out-of-work nerds to do what we did but I think most either don't believe you can get work without a CS degree (by assuming that we just got lucky) or think that it is well beyond their mental capacities. I'm not surprised a lot of unemployed americans are scared of trying something intimidating, but I think you'll see more and more young un(der)employed americans start learning code, especially as the learning resources become more and more available and more and more accessible.
I stopped reading the article at this point. Seriously. I've been programming for nearly 30 years and in that time no one ever thought there was a risk of our having a glut of good programmers. Ever.
Hell, finding good talent back then was much more difficult than it is now. So stop your whining.
Because a career in computer programming (still) sucks: http://www.halfsigma.com/2007/03/why_a_career_in.html
> Programming is easier to learn than a foreign language
Maybe 'coding' but not programming and definitely not software engineering.
Serious applications seem so complex with their millions of lines of code that people are simply scared off.
I know that the industry is now trying to entice youth and professionals to begin learning code with things like http://code.org, but they need to focus more on the rewards (i.e. like how much money you can make, and job stability even during economic crises).
Learning to code is hard, and you need to commit your brain to 3 years of serious edumacation, and then 5 years of industry experience if you want it to succeed.
That's a tough bill for anyone to swallow.