Google: 90% of our engineers use the software you wrote (Homebrew), but...
twitter.com
twitter.com
Succeeding at these whiteboard questions requires weeks of preparation. You need to practice, practice, practice. After enough practice, you are pretty likely to pass. So ultimately, it is more of a test of "how much do you want to work here." If you care enough, you can pass. That said, these types of interviews do favor fresh grads over experienced programmers (you have algorithms and data structures fresh in your mind), which means that they are flawed in IMHO.
Nothing says Google must accept sub-par technical people, but Google fails to realize you don't have to be the top 0.0001% in absolute algorithms intelligence to do great work.
Google's basic theme seems to be "you must be low latency ultra smart in our narrow interest areas" to pass an interview (or well-connected in other ways to lower scrutiny), but Google's main problem these days isn't lack of smart people — it's lack of projects with meaning and lack of how to organize smart people into anything resembling a coherent world-moving force.
In The Plex is great read for insight into their entire process. Basically, Google is run by people who have never been told "no" to anything in their lives, so they continue to think they are the best at everything until reality forces them to reconsider their delusional assumptions ("montessori naivety").
I think you made an excellent point. It seems that outside a tiny minority of world-class experts, Google is more interested in hiring interchangeable workers.
I know some people get frustrated by those kinds of questions, but I'd just see it as a reverse weeding out process. If you still want to work at (insert company here), just build something valuable and have them acquire you. At that point they'll acqui-hire you and you'll leap frog all the people who got in because they had Knuth books memorized.
You know, there is something to be said for someone who has a good product sense, but not good developer skills. They sound very valuable, so why are they applying to a developer position? Just because you are unqualified for one job, doesn't mean you deserved that job, or that you're worthless.
Go do what you're good at.
As such, the best approach to these interviews might be to say, "Look, I could take a stab at that algo question, but I haven't touched that stuff since school, and if I needed it I'd google it. It might be more productive if we talked about what I've been doing day-to-day for the last ten years to ship successful products, because that's the main value I can bring to your company."
Any mature company should focus on finding those specific candidates - the ones who fit the company and specific team, on several dimensions beyond standard skillsrequired for the job. That's someone who will be called the best candidate in that specific company. And it's absolutely natural that s/he won't be considered "best" insome other company.
It was several hours, with many interviewers one at a time, of academic exercises. Some I could see were relevant to some of Google's projects, but that wasn't the emphasis- it was very much a "how much do you remember from CS courses?" (with a few exceptions- one had me explain why a snippet of C would crash- something more what I would expect a senior engineer to know that was more practical-oriented).
I _think_ I did pretty well (again, it was part of a bigger due-diligence-- another story altogether, so I never heard), but in the end I was struck by the fact that (a) not a single interviewer had looked at or asked about any of my publicly available work, and (b) not a single one even asked something along the lines of "what are your strengths / what do you think your contribution would be" (or weaknesses, for that matter, but that seems less important).
Of course, one of the other guys I worked with lucked out and got interviewed by one of the paypal-mafia, and ended up with a reportedly very stimulating hour long discussion about actual work, strengths, and actual situations that one would hypothetically need to handle at google.
And ye olde "what's your greatest strength/weakness" question sounds like a terrible way to judge an engineer.
It could be dangerous. Let's say you explain some interesting techniques you used on your GPL'd software and a Google product starts doing something suspiciously like what you showed them. You may even sue and end up making their source public after some litigation.
Only to bad interviewees.
You'd be shocked to hear the number of impressive résumés I have seen who turned out to be absolute no-hires after 20mn on a white board. By "absolute no hire", I mean the kind where some of the interviewers tell me in the debriefing "I will quit if we hire this guy".
Not all excellent engineers have impressive résumés but they all share a common characteristic in my experience: they all really enjoyed working on the problems I asked them to solve on the white board. I even had some of them be disappointed when we ran out of time because we were having interesting conversations about the problem.
Hmm, at least in my experience it's really quick and easy to tell if someone is bullshitting about their resume or not by simply talking about the different projects they worked on. I know a few developers who didn't go to a 4 year school so they don't know all of the correct terms or algorithms (one even interviewed for Google and failed) but I would hire them in a heart beat because they are really awesome.
I still feel some sort of pair-programming exercise or something similar is better than simply trying to solve problems on a white board. Unless you're in a SCIF you're going to have access to the internet to refresh your memory, etc so I've never found utility in asking academic questions. In fact I've had a few people who could pass the academic questions but couldn't build an application to save their life so I don't even bother anymore.
That's just a self-reinforcing selection bias - you hire people that are good at these kinds of problems/enjoy them - you filter out the rest by default and it becomes ingrained in the culture - it doesn't say anything about the suitability of the candidates you filtered out. If you said "we hired a guy who had a great resume but he couldn't do algorithmic problems, turned out to be a bad fit for the company" - that would still be a single data point but at least it would be relevant.
One of the challenges with Google's interview process [0] is they are very worried about hiring mistakes, and also know that the process is very noisy. As such, they interview a tremendous amount of people for hiring. In addition, they've found it more efficient to just keep a high bar (missing many good hires to keep out 1 bad hire) than to conduct more than 5 interviews.
The one thing I've wonder is - if they want the top 100 computer scientists in the world to work for them, will this process produce that? Or is there another process for them?
I think the most effective way to get a job there is to build a small company and get acquihired.
[0] I don't work at Google, but I've researched the process in anticipation of an interview.
Many of the interviews could actually work if they broke the process down into 8 questions instead of one (how would you represent a tree? a binary tree? how would you find node X? how would you move node X to the other side of the tree?). But, the ability to accommodate understanding takes empathy and human connection, not being a jerk programmer on interview duty who is browsing HN while listing to the interviewee mumble on your speakerphone.
Never underestimate the lack of ability in the interviewer and not the interviewee. Except, the company always believes itself to be more competent than outsiders, so you're ultimately left with being condescended to unnecessarily.
Whatever! They asked me back but it sure is a frustrating feeling even if I tend not to appear flustered until I walk out, introspect, and realize what a doofus I was.
They don't just stop at binary trees. Think of it more like memorizing hundreds of magic tricks that could be done using a deck of cards. The tree is more analogous to those deck of cards. The data structure remains constant but the tricks you can do keep changing.
I know of situations where candidates have been grilled whole full hours on tree traversal methods, and then asked to code up perfectly working solutions for them.
Do these people seriously think people who can invent 50 years of CS research by sheer analysis over an hour on the whiteboard, will be ready to work for them to write shell scripts? If not you are only testing memory skills.
One thing that I'd recommend to anyone seeking any kind of job is taking some improv courses--they are pretty common in major centers and even some smaller towns, and can really improve your ability to deal with this kind of nonsensical interview situation, which tests a bunch of skills that have nothing to do with your ability to actually do the job.
Of course, the irony is that for vast majority of the people they hire, most of the work they do will be building and not much algorithms work. So then what will happen is either you have people who are great at building things and force them to suffer through the interviews or you have people who are great at algorithms but not that interested in programming.
Of course, you might also have people who are great at both, but I wonder how many of them there are..
Correct me if I'm wrong, but a binary search tree is a very simple data structure. It has rules, and just by it's definition, and the definition of "inverted", you should be able to come up with an iterative solution, even if it's just walking the tree and re-creating it.
It may not be the best, but it's a solution, it's a starting point.
But if you're interviewing for a job solving problems, shouldn't you be able to solve problems? Even if they aren't the best solution? Genuinely asking here, as this approach seems to have a 85-90% success rate for me.
If these problems were so reliant on individual problem solving ability, there wouldn't be dozens of books with reams of answers to these questions on the market. These questions test memorization and nothing else.
Google said early on they want to hire the best and brightest and that was the most important. We got the mantra that passing over a good person is better than hiring a bad one. And they were right. But they're a huge company now. They, and other gigantic tech companies are always complaining they can't find 'good people'.
But how do you make this philosophy scale? Do you say, "This person is smart but lacking some skills so we'll invest in them"?
I have no idea. :(
Google has decided that the only way to tell if someone is smart is by asking academic questions.
Even though anyone with half a brain knows that there are other ways to gauge how smart a person is... such as experience.
Does anyone really think Max can't look up how to invert a binary tree if he needs to do that? He obviously knows how to read and learn. So why is that a reason to reject someone?
Google is filled with smart people.. but their interview process is pretty stupid.
The "can't find good people" argument can usually be solved either 1) raising the pay, or 2) increasing the labor pool (i.e. H1B, etc). Raising the pay is what is done in finance because it's harder to make the argument that the required "skills" are lacking in the labor pool. To enter investment banking, and then private equity, it often only requires a B.A. in French literature and willingness to work 90 hours.
Google (and other interviewers) might as well be asking me to name the last 1000 digits of PI. The Kafkaesquen ability to recite how to do the task correctly reflects nothing about the candidate, except the strength of their memory.
It boggles the mind that the field of computer science does not have any standard way to detect the competency of a person. It is extremely discongruous for a company with as many smart people as Google to have such an absurd hiring policy.
"I see your resume says you've been writing software for 30 years, and wrote this really well known tool. Wellllll....That's good enough for me. You're hired!"
I don't support skipping interviews even for good candidates, but if you were going to it would work the other way around -- you'd value recent accomplishments over more distant ones.
People don't understand that cutting out folks who had their name attached to some big successes but can't solve a toy problem in an interview is usually a really good thing. It's a strength of the system more often than a weakness. You wouldn't believe the number of senior candidates I interviewed who had really impressive resumes but were clearly checked out from doing real technical work -- folks who obviously no longer lived close to the code and either were very rusty or just no longer had the temperament to spend time thinking through a slightly tricky piece of code.
One of the reasons not to skip an interview for a candidate you're confident is good is that a good interview can be a great sales pitch to come work for your company.
IIRC, a Portuguese proverb states that paper accepts whatever is written on it. One should always take a self-assessment with a grain of salt.
Investment banks recruit bright young graduates, putting them through a tough selection process (c.f. all those hard-working interns we keep reading about), and select the ones who perform/conform the best. The rationale for bringing in young (as opposed to experienced) people is that they'll work their asses off, can be moulded to fit the culture (i.e. independent thinkers aren't necessarily sought), and (perhaps most importantly) are too junior to present a threat to the incumbents.
If Google's going down the same path, it may be a sign that their corporate culture is solidifying, which would be a bad sign, IMHO.
For the top elite of experienced engineers that may not be the case, as they will come with some overall good analytical skills that Google wants. But for most in our industry, what we have learned over the years is how to be good in various sets of tech around our platforms of choice... and that is mostly irrelevant to Google, as they have their own platforms.
But then it occurs to me that they won't have seen them in -this- role acting -this- way for -this- film/play/whatnot and that's what the director needs. And given the amount of money and risk involved, it's probably wise to check first...
No computers. Pen only.
For me, rehearsing an answer works out. Then, I try to see how to deconstruct it and apply it to a similar problem I don't know.
I can deliver an answer sounding confident and competent because I practiced an answer. It's a bit of a crap shoot.
What irks me the most is that people are doing the interviews. They can be fooled and persuaded with tones and intonations without them realizing it.
Have a friend do a mock interview with you.
You spot patterns but there's so many different ways they can ask things and things that you just remember from studying it, that I imagine I'd do worse now unless it was close to what I've done since. On the spot I couldn't invert a BST right now.
I suspect you could probably pass a whiteboard interview a few weeks after taking a mooc on data structures/algorithms with enough depth for example.
For big companies that want to scale hiring process, there's really not much you can do.
You put difficult questions and you hope that bad engineers will not be able to solve them. Then you absolutely have to commit to not hiring the guys who can't solve the assignment, otherwise it's all for naught.
I should have turned it down. I've written on this thread already what a career-churning waste of time my brief stint at Google turned out to be because I was blind allocated.
Fortunately, the skills that got me noticed by Google were in demand elsewhere then (and, ironically, in demand at Google now) so I just exited the building when it became clear HR had labelled me a troublemaker for trying to find a position that made use of my talents rather than opt for the career reboot they seemed to expect of me. Great perks, but the 2nd worst gig I've had my entire career.
1) Most companies (for legal reasons) don't tell candidates why they weren't offered a job. Maybe it was because of the binary tree question, but maybe it was for some other reason.
2) Homebrew is a Mac-only product, so the likelihood that 90% of Googlers use homebrew is very low. Moreover, Google does not track the software its employees download onto their laptops, so there's no way they would even know the percentage.
One of the reasons why it's so powerful is because it gives lazy journalists a stream of easy reaction quotes. When else in the history of journalism has there ever been a private company that was so heavily promoted, regardless of the media source? And even then, the react quotes are poor quality, single-sentence shit.
This rant on twitter is absolutely meaningless unless you have a heap of context to go with it. And because twitter is popular, it's dumbing down public discourse along with it.
/rant, that I couldn't have fit in 140 characters.
The OP has written some great apps, sure, but there is a huge difference between writing a package manager for Mac (among other Mac/iOS apps and utilities) and writing incredibly complex, highly performant algorithms for say search indexing, machine learning, ai, etc...
In that context, knowing CompSci basics like Binary Trees (usually taught to pre-major CompSci students) has a lot of relevance. It's clear Google was looking for a Computer Scientist here, not just another developer.
I am also put off by the extreme display of non-professionalism here. It's like the OP took this personally as an insult. "I've written popular things, how dare you not hire me if I want to work for you!". That's not a professional I'd like to work on a team with. Rather, that's a throw-back to the "Rockstar" programmer that nobody wants to work with.
This boils down to a person who was personally offended a company did not hire them at a drop of a hat, as-if he was "blessing" Google with his presence.
I'd say, Google (and other companies) are better off without this type of ego.
They made the proper choice to weed this individual out.
Additionally, I think people have a mis-perception of the skills required to work at Google:
> writing incredibly complex, highly performant algorithms for say search indexing, machine learning, ai, etc...
...
> Google was looking for a Computer Scientist here, not just another developer
This is not really the case. The large majority of positions are for basically "just another developer" to maintain an internal Java tool, or some web UI, or whatever. They are complex in that they have many dependencies and often lots of legacy code, but not all to different from Microsoft, Apple, Facebook, or any company with a large code base. In my several years at Google, 0-5% of my time was spent coming up with complex algorithms.
That said, I don't think Homebrew is one of these. Always seemed like a quick fix for an OS X deficiency to me, and its competitors MacPorts and Fink being mediocre themselves certainly helped with its popularity. Not to rain on the achievements of the developers, but still.
Blazing the trail in machine learning, ai, etc... that's new territory where there are no knowledge bases or places you can go to source information about how to do it.
That's a different ballgame, and requires a scientist, not an engineer/developer.
We cannot just assume that since he applied for iOS something at Google that he's the best fit and what they were looking for.
Maybe the tools Google needs built are very CS heavy (hence the "Build us a Binary Tree" question)? Maybe during the interview his arrogant attitude was on full display? Maybe the interview team dug up past episodes of explosive arrogance like this very one we're discussing right now?
Even if they were looking for someone that matched his profile exactly, his post-interview display of non-professionalism is sure to hurt his chances of a re-interview anytime in the future (and quite possibly at many more companies than just Google).
He conveyed several things with this display, none good;
* he's incapable of handling rejection
* he feels entitled
* he feels he's better than everyone else
* he's unwilling to admit his own shortcomings
None of these are good qualities.
- The guy made Homebrew
- The guy interviewed at Google
- The guy was not offered a job at Google after his interview.
Anything else is unconfirmed and, speaking as someone who's thrown quite a few frustrated hyperboles onto the internet, sounds like frustrated hyperbole.
Frustrated hyperbole should not be taken at face value as fact; sometimes there's truth in a smaller version of what's said, but not always.
For example, there is no fact established that Google hired him because he can't "invert a binary tree". In actual fact, we don't know that Google even asked him to invert a binary tree (at least not specifically). It could be that they asked him a question that he thought was as irrelevant as academic datastructure exercises.
And we don't know that his answer was the reason he got turned down either. This is the part of the hiring process (and really any human interaction) that takes the maturity of recognizing that people and their motivations/reasoning are more complex than we reflexively flatten them out to be.
What facts are you talking about? The "facts" coming from half the parties involved? The "facts" coming from an upset and irrationally thinking person who's caught up in the moment?
Nobody wants rockstars in teams, because rockstars are bad team players. That's the reason why they're rockstars. Nonconformity (sometimes) leads to innovation. Like it or not, a lot of the tools and libraries which we use every day and on which the internet runs, was written by lone-wolf 'rockstars' too weird and too eccentric to work in large corps.
You don't want your whole army to go over those mountains, you send scouts. Scouts are terrible soldiers, but they are quick to find a path between the trees and rocks and they come back with new information, new ideas and concepts of how to optimally win the battle. Then and only then you bring the whole army and win.
A company, imho, should always have a small number of rockstars - impossible characters, crazies and lunatics with universe-sized egos, who have a track record of successful projects in the wild, working on insane things, alone or in very small groups, carefully managed by someone who can earn their respect - usually a lone wolf turned general.
They might be terrible programmers, they may have no education, but they have this unusual ability to see into the future and come back with interesting ideas and prototypes, which the 'soldiers' - people with solid fundamentals - can polish and transform into profitable products.
If a company innovates 'from the top', then it's trend is going to slowly be downwards towards irrelevance, because true innovation comes from the trenches and is done by the rockstars nobody wants to work with.
If that's really what they intended for him to develop, then why not ask questions about machine learning or AI?
The OP is the type of person that, six months from now would be demanding special treatment because "he wrote Homebrew".
Writing a package manager (been done before, a lot of times) is far different from writing machine learning algorithms (new field, blazing the trail for the industry). One requires an engineer, one requires a scientist.
The OP flamed out because Google was looking for something he was not. Instead of taking it like a professional, he decided to retaliate.
The tone he took in his post is appalling, and I cannot believe HNers are defending him. "I'm special because I wrote something popular (and make absurd claims about 90% of people using it), so you better hire me or else" is what he just laid down.
That's not the type of person I'd want to hire or work with... and I'd wager most would agree.
Getting rejected from one interview at Google isn't the end of the world. The right and mature thing would have been to go home and figure out the problem that you got stuck on.. and then try again. We all go through this process, why should we expect anyone else to get a free pass?
I recently watched an online game of chess between a grandmaster and an anonymous individual.* The grandmaster got caught off guard and got lured into a trap. Instead of getting defensive, he sincerely thanked the other individual for a great game and spent the next few minutes after discussing the intricacies of the trap.
Shouldn't someone as accomplished as the creator of Homebrew feel the same way about a problem he got stumped on?
* https://www.youtube.com/watch?v=Voa9QwiBJwE&feature=youtu.be...
That being said if I asked this as a warm-up and we'd spend the whole interview trying to get that done then of course my verdict would be No Hire. As an interviewer it is not my job to look at your GitHub profile - instead I am assigned an area to check and I try to come up with the best understanding of the candidate in that area. While failing to reverse a binary tree is a total failure in algo/ds you can still be hired since there are several interviewers (if you make it to onsites, that is), each probing a different area.
My biggest problem with Google style interview process is that it's easily passed by folks who already passed it in a different company. After having interviewed hundreds of Google's candidates I moved to another big company with the same interview type and the experience was really surreal. On my algo/ds interview I got asked slight variations of the same questions I was asking myself - and over time I've seen some totally brilliant, unexpected solutions. Must have been a strange experience for the interviewer who got his questions each answered in 3 minutes tops. I also made it a sport to solve each one in a different language because why not. Not sure though about validity of the signal the company got from this interview.
It is a terrible question.
First, you can't invert a binary tree (as in flip upside down). If you did, you'd end up with multiple roots and since all binary trees are rooted, you'd no longer have a binary tree. It'd be a tree, just not a binary tree.
If the questioner meant mirror a binary tree (swap left & right at each node), then it is a no-op. You do not need to modify memory to do it. Left & right are just labels for two pointers. You can just swap the order your variables in whatever struct defines your node and cast it against your existing memory (or change what your accessors return in a derived class or create a reverse iterator or however you want to implement it) and there you go, a "reversed" tree with zero overhead.
Either way, it is a terrible question unless you wanted to see if someone understood the difference between how a data structure is drawn on a whiteboard for humans vs. how it actually works. Maybe they were actually asking that question, but that seems highly unlikely.
And if they actually meant for you to recurse down and swap left and right on everything, it would dramatically lower my opinion of them because it would make me wonder if they knew the difference between how a binary tree is drawn on a whiteboard vs. how it is laid out in memory.
Inversion is a transformation that maps directed graphs to directed graphs. Binary trees are a subset of directed graphs, so applying inversion to them is not unreasonable. That subset is not closed under inversion, so you can get results that are not binary trees, but I see nothing in the question that implies that the interviewer was asking for the output to be a binary tree.
That may even be one of the points the interviewer wants to see the candidate address. Since the output no longer has a single node from which all other nodes can be reached, in addition to just inverting it they may want the candidate to mention the need to have some new auxiliary data structure to keep track of the multiple root nodes (and perhaps note that in the inverted tree each node only has one outgoing link, so if we are inverting in place each has room for two, so we can use that now available second link space to make a linked list of the roots, so we don't need any extra storage for the new root list data structure).
While only the person asking this question knows for sure I'd assume that was the intention.
It's not a _terrible_ question - it's just a recursive equivalent of FizzBuzz. You would treat this as a way of making a (good) candidate gain some confidence. And that alone is extremely important since the more at ease the candidate is the better the signal that you're getting. What I was trying to say there was that if I asked that as a warm-up and didn't get a satisfactory response very very soon I'd realise that your man has no clue about recursion and would move to the main question, choosing one that does not involve it. It would be something easier than I'd ask otherwise and would be closer to real-life scenarios - like first solving a simple problem, then parallelising the solution and correctly managing concurrent access.
Just because someone wrote on Twitter with limited number of characters that the problem was to "invert a binary tree", does not mean there were no clarification from the recruiter.
> And if they actually meant for you to recurse down and swap left and right on everything, it would dramatically lower my opinion of them because it would make me wonder if they knew the difference between how a binary tree is drawn on a whiteboard vs. how it is laid out in memory.
If you're thinking about making a real structure, I believe you should recurse down and swap. If you don't do this, rotating, joining etc, could be a real pain.
Sorry for the elementary questions. I'm bad at algorithms and just trying to get a grasp here.
I've interviewed hires before, and if I ask for a github, I'm going to take the time to review some of the code, otherwise why would I ask?
Be that as it may, if the interviewers hand you a reasonably detailed specification of what "invert a binary tree" means and you can't whiteboard it, I don't see how you can expect the job.
If you're expected to know what it means, and get no hint, then that is just stupid. "Whiteboard up an implementation for something we refuse to specify, beyond citing its name."
Note that in the original tree each node needs storage for two pointers. After inversion only one is needed. You can use that now unused storage to link together the multiple roots to solve the leak problem.
But note that only consumes the second pointer storage in the roots. Interior nodes end up with some free storage after inversion.
Since inversion is reversible, the binary tree and its inverse contain the same information. Does this mean we should only need one pointer's worth of storage in the binary tree nodes?
Is there something for binary trees similar to Knuth's xor trick for doubly linked lists?
The binary tree inverse specifications/implementations I have looked at preserve the information by selectively using the left or right links. For instance, given:
P
/ \
L R
L R
\ /
P
[1]: http://stackoverflow.com/questions/9460255/reverse-a-binary-...
FWIW, I find Facebook turning down Jan Koum in 2009 and then spending $19B to acquire his company even more ironic.
Objective - the process avoids human bias (this guy was in my frat, therefore is awesome).
Inclusive - someone doesn't need extensive past knowledge in a particular area of coding to do well in the interview.
Risk Averse - Avoids false positives.
Relevant - it properly tests the abilities needed for a coding job, and not irrelevant ones (like the ability to write an impressive resume).
Scales - this will be used for tens of thousands of interviews.
Easy-to-do - this will need to be done by hundreds/thousands of engineers, who would probably rather be coding.
It's easy to poke fun at what is perceived to be a flawed process. It's much harder to propose a solution that satisfies the above requirements. Google has done extensive research on this topic and has done remarkably well with it compared to other companies of similar size.
How can you make the interview relevant without knowing what the position is? I was asked the typical CS-type questions in my interview, but the team I ended up on required no theory.
How do you define false positives before you know what the candidate will work on? A superstar in one team will be a dud in others.
And let me add another bullet point to your process wish-list: gives the candidate a sense of whether they want the job. This is impossible when the interviewer is a random engineer from an unrelated team, unable to speak to what the candidate's work life will be like. A Google style process gives candidate very little information.
I would instead propose something very old-fashioned: teams hire for themselves. The usual reply is that this results in an "inconsistent hiring bar", but so what? Teams have different requirements and need engineers with different skills, so why shouldn't the hiring process reflect that? We are not fungible engineering units.
edit: really good english skills
Inverting a binary tree is pretty easy. It is not quite as trivial as FizzBuzz, but it is something any programmer should be able to do. If you can't do it, you probably don't understand recursion, which is a very basic programming concept.
This isn't one of those much-maligned trick interview questions. This is exactly the kind of problem one may have to solve when writing real software, and though you may never have to do this specific thing, it is very related to a lot of other things you might do.
I run a small software company and I very likely would not hire a programmer who was not able to step through this problem and express a pseudocode solution on a whiteboard.
Pretending you understand or blindly assuming what the interviewer meant is more of a red flag to me than requiring a few rounds of clarification.
In my opinion, an ideal interviewee would communicate that they don't fully understand the requirements and possibly offer a brief set of possible interpretations. Once the interviewer has clarified, my ideal interviewee would restate the clarification to demonstrate competence and verify their new interpretation of the requirements before proceeding.
If this was really about "reversing" the tree, as you mention, the question seems more likely to address how the candidate approaches the situation. Like, he should start by making sure they both agree on what the question actually means.
Once that's out of the way, it seems relatively easy to come up with a naive solution, without having memorised any algorithms. It seems more like a case of brainfreeze to me, which can be sort-of fixed with practice (which in turn many candidates refuse to do: the dreaded "If I have to cram for the interview, I don't want the job" statement.)
So maybe he really wasn't a good fit for Google, despite apparently being a rockstar developer. Hey, startups need rockstar devs too.
They are seemingly less interested in seeing how you solve problems and work through the process of software.
I would be more than happy to see this process change, I am just not sure what it entails.
I think "CS Fundamentals" are really just "abstract concepts used to teach programming", and calling them fundamentals is disingenuous.
Some companies give take-home assignments (small projects that can gauge how well you learn). I tend to find that these are smaller companies. You can always try there.
Also, the most important thing that they don't teach you in school is that getting a job is all about connections. Email some people that you've interacted with to see if they know of anything that might be suited for you.
I reached into my way-back machine and tried to derive some approaches where I simply didn't remember the answer (and I was very open about it). I made it to call backs both times, but they declined to move forward, probably because they wanted a younger person who remembered their big-Os off the top of their head, but I was okay with it. I told all the interviewers I had fun and I did.
Even if I had made it in, I'm not sure I would have taken the job at those times. So my lack of motivation ended up turning what could have been stressful into a fun look at their hiring practices.
However, I can see for people who are really dead set on working there, it can be a harrowing experience.
Serious answer: Companies that pull this crap deserve to starve and die.
How would you interview software engineering candidates?
If you're not at Google's scale, a take-home test that involves them producing a working solution to a problem in their own time, that you then expect them to be able to discuss, reason about, justify, and so on is a much better tool; even at Google's scale, where you might be worried people who copy answers off the net, have them bring the laptop they used in, and then get them to expand upon their solution.
All that said, I'd be surprised if that was the only factor at play. The Twitter comment itself seems to indicate a - uh - cultural fit issue. Additionally, and with no actual knowledge of the insides of HomeBrew, that a piece of software is widely used isn't evidence that the author is an excellent (or even good) developer.
Doing a blind and "surprise me with a clever (or traditional) algorithm" whiteboard interview is a teaching skill, not a coding competency. You have to illustrate, explain, make your self vulnerable while understanding the problem, and work in front of someone while being judged confrontationally. It's not sane. (and glob help you if you don't wrap your entire brain around the problem in the first 5 seconds, then you're clearly subpar and are worth less than dirt even with your 15 years of experience shipping products to millions of users.)
Google is happy to have 1,000 nerd projects and only one thing that makes money. As long as they continue to interview smart people on technicalities instead of well-roundness and ability to actually contribute or move the company forward, they have no need to fix their insultingly irrelevant interview process.
The main problem with interviews [imo] is it grabs stuff the interviewer is familiar with and throws it at the person in the more stressful position of being the interviewee. Its also [often] a very non-standard way of communicating. I don't know about you but all of my communication is via text or graphic, not whiteboard.
And, honestly, do you care more about the fact they can invert a binary tree from memory? Or that they can learn to in the next hour and implement it?
My guess is for 90% of jobs, its the second.
Well, if you wanted to pull that card, it would have been nice to mention the sorts of problems you've worked on in those 25 years.
>What I would do if I had to invert a binary tree: look it up.
Unless you're already good at algorithms, it would net you a mediocre solution.
For e.g. You would only know that there are multiple ways to implement an algorithm when you've actually done the work dozens of times and noticed that some implementations won't be appropriate for you needs. Like with many things, its about having experience, knowing whats a good fit, and knowing why something similar isn't a good fit.
Certainly - every single time you choose an algorithm, and then decide on a particular way to implement it - you could in theory, implement each algorithm in 10 different ways and then choose the best one after benchmarking. But that would be a huge drag on productivity. And if you had to choose 5 algorithms for 5 different tasks then it quickly becomes a quadratic complexity time sink. It would be far better to just know via experience. Its kind of like in chess - the better players just KNOW that certain paths lead to less favourable winning odds. Well, novices do take those paths and eventually find out the hard way !
Quite a range of stuff, unsurprisingly. I started on an HP3000 mainframe in 1976, if you want to go back to the very beginning, writing BASIC programs on a teletype or one of the two early CRTs, and storing my programs on paper tape.
Since then I've worked in DOS, 16-bit Windows, 32-bit Windows, 64-bit Windows, Ubuntu, iOS, and Android, using Pascal, 8086 assembler, C, C++, C#, javascript, Python, and Java. I've worked on applications in multimedia, telephony, banking, insurance, pharmaceuticals, cosmetics, health care, and I'm sure a few other things I've forgotten.
All of which will mean jack squat if, tomorrow morning, the most important thing I have to do is invert a binary tree. But I'm fairly certain I would be able to figure out what I needed to do, and I am fairly certain I could manage to implement it well. It's what we're supposed to be able to do, and if you think that having studied up on it so that you could pass a Google interview means that, a few years down the road when you actually need it you'll just whip it off the top of your head, then I think life may hold some surprises for you.
He also would've nailed it had he been given 10 minutes to do some research to refresh.
Silicon valley (and the numerous companies that mock the interview style) are testing for the wrong thing when they hire, then complaining about not being able to find good engineers.
Anecdotally, at a previous company, we tried an "open book" (i.e. you can use the web) interview policy for a few interviews. It was a train wreck.
Begs the question, doesn't it?
>> Binary trees are MOST ASSUREDLY on that list
Sure they are... and my intro to CS textbook was ~500 pages. I'll bet I could find something to trip up just about anybody, especially when coupled with the pressure of doing it in front of the class ^H^H^H people that have $$$$/power over you.
So I gave him one of the spare logic boards (open design anyway). And wrapped my hand around his. And squeezed. And asked him, if it doesn't exist, why is it making you bleed?
He watched impotently as the people who had been invited for the presentation played with my robots and ignored his as two of his guys tried to get it to work.
I finished the two projects I was doing with Google and did not call them again.
(Before you downvote: Yes, there is some video, and I consider the small amount of pain I inflicted that day a kindness compared to the much greater amount of pain that an engineer is in danger of enduring if he says things like "This thing that is in front of me, it does not exist", especially if he works with big machines).
Edit: typo
I think everyone knows that this is not a problem exclusive to Google.
From his perspective, I bet he's frustrated for the reasons he point out. If he never talks about it out loud, how would any one know? Keeping people silent about this issue is in the company's interest because they'll not have to really change, and they'll hire someone eventually.
I mean, I know I sometimes choke on problems I'm trying to solve in the privacy, quiet, and comfort of my own home on projects I really care about.
I don't know what went on in the interview but given what he said, I think I would be upset, too.
Too hard to say, tbh.
On the other hand, if it was a complex and reasoned decision that he's summarizing badly, then yeah, good call on their part.
Context is king here, and we have little.
And I'm pretty sure this is EXACTLY the type of guy you want to hire, I would strongly suspect writing mild mannered complaints about not getting hired on twitter has very little correlation with job success.
Interview with Lazlo Bock on Google's hiring practices:
http://youarenotsosmart.com/2015/06/08/yanss-051-how-google-...
Some of the claims Lazlo makes:
As large organizations grow, their workforce trends towards mediocrity. Google * takes special care to counter-act this effect. * researches their hiring/interviewing practices just as much as their machine learning. * publishes their methodologies: https://www.workrules.net/
The algorithm in question is discussed in Coding Interviews by Harry He. http://www.apress.com/9781430247616
I feel the original tweet conveyed a bad attitude, was emotional, reactionary, and ultimately a bad career move on the part of OP.
In my younger days I suspect I would have done something similar. I'd like to think I would see the experience as a learning opportunity and be able to react with humility and maturity, but who knows? Hopefully I can think of OP and not tap the tweet button.
Really? Pretty much everyone recognizes that google style interviews weed out perfectly good people. What Max went through is just an example of one such obvious case. It's very much a case of the google hiring algo failing. Lot's of people would have no doubt that Max can cut it when it comes to iOS dev.
That's all it is. Now, if you are going to read "tweet conveyed a bad attitude, was emotional, reactionary," into a perfectly human tweet, holy crap you are judgmental.
>> ultimately a bad career move on the part of OP.
Not at all. I've done a lot of interviews and basically none of them required us trawling twitter. I think it would have to be something pretty heinous for me to not hire someone based on their social media crap. Definitely not something as mundane as this. This sort of hilarious cowardice about expressing feelings just makes me angry. At what point do we stop acting like these trivial, humanizing glimpses into a person are somethign that is a bad career move?
If this is how success and failure are determined at google, it's no surprise how many of their products that people actually use come from acquisitions.
I interviewed once for Google (at their request) and failed. For some reason they interviewed me for a networking position instead of a code one, so questions about TCP internals were not really my forte. I was just launching my second start up at the time and would have declined the job had I gotten it. I admit it does sting a bit to be declined - not just for Google, but for any position - even those you would decline yourself.
Just quibbling: I don't think anywhere near 90% of engineers use homebrew? Google development is done on Linux, and Homebrew is a Mac thing AFAIK. I have never used Homebrew.
Android development can be done on Macs but I doubt they use Homebrew. Certainly not for anything important.
Its just good ol' hyperbole man. No small children will die because its not exactly right, it just helps make a point.
Now, I have no idea what the code quality of Homebrew is, but just because he built something popular doesn't mean he should get a green light in every company. If Google is looking specifically for top-notch software engineers, they are probably filtering them very well with their practices.
Maybe they are only good on paper at that moment and don't have something like "Homebrew" on their Github, their knowledge is sufficient to perform work at Google. So why pick someone who has fame to his name, probably wants to get paid accordingly and thinks he is a hotshot because of his Twitter and Github follower count over someone who proved himself in an interview?
The first is not necessarily better than the latter.
One problem is that only experienced developers can do these kind of interviews, because you need wide experience, be able to talk about various technical topics and tell whether the other person is telling you stories from their own experience or some quickly learned facts.
It's funny, but the best experience I had interviewing at Google and Amazon was talking with the managers.
That's usually what they say, but I've found that if you don't solve the puzzles, you don't get hired ever.
What does bother me is that other companies, who are not even in the same league as Google, start to copy their hiring process.
I remember interviewing with a digital ad agency a few years back and I swear, these guys thought they were Google. The number of academic trivia questions that came up, it was ridiculous.
In a way, I think Google has done a lot of harm to the industry in general by making others believe that everyone should have a hiring process like Google's.
On the exit meeting it was relayed to me that they were having trouble finding people because no one could pass their tests and I was beside myself because I couldn't understand how they would expect someone with that technical ability to want to bang out Wordpress sites all day while there are 100s of people who would love to do that job and be very successful without ever even knowing what basic recursion let along the stuff they had in their test. Bizarre.
interview was brutal - lots of whiteboarding very artificial problems totally unrelated to the business and i just couldn't get excited about it and didn't end up getting an offer
hiring is tough, these things happen
This is like knowing math and/or stats and applying it to a word problem.
I mean imagine the amount of extra work someone has to put in to start interviewing. Take a break from your current work, Prepare your CS basics again, Prepare from interview question dumps online, read up / analyze everything the new company is doing and form reasonable opinions, practice white board coding / coding without an IDE, allocate time for any homework projects given, Psych yourself up if you are introverted etc. The alternative is just to stay in the current role and hope stuff gets better. 90% of the folks I know choose the alternative over the dehumanizing process of interviews. So many folks I know are good engineers get chewed up in interviews (both in my current company and elsewhere) because the process is pretty cooked. We are trying to see how this can be improved, but yeah - I just keep going back to my pet theory :)
I do agree with one of the commenters here though: At one point your resume should speak for itself. These are the kind of problems I would like LinkedIn to be solving instead of finding ways to spam me with recruiter deluge.
I'm sure Google is still a great place to work, but its reminding me more and more of 1999 Microsoft. In fact the similarity is spooky.
Secondly, I don't think this is a hard interview question; it's certainly fair. Did you expect to be asked knowledge-based questions that Google knows you're already good at? Questions specifically geared towards you? Or questions where Google can watch you solve a problem and be comfortable with the fact that you are able to solve coding problems? Did you think Google would hire you to write Homebrew? Or solve problems on teams Google has?
I think this person is just being unreasonable.
Edit: as pointed out by others, the hiring decision probably does not take a few hours, but under an hour. Still, the point is valid.
It seems like this tests a) how much you want to work at Google and b) how good you are at memorize things.
Homebrew is at least one of the best package managers for Mac. I would be very surprised if it was not at least near the 90% mark...
I mean - what's the point of spending 3-4 years in an Academic environment that proceeds to then test and grade students on exactly how good they are, at the time - then only to perform the whole process over again some number of years down the road, with fuzzier results?
Seems dumb to me.
I've worked with people who could likely do very well on algorithmic tasks - (of which most software projects require precisely zero) - but actually deliver something of use... not so much.
I get the puzzlers or whatever for a phone screens (quickly weed out people that are obviously unqualified), or if you're hiring someone junior who doesn't have work experience, but if you're at the point where you're bringing someone in you probably think they're minimally qualified, so it should really be about determining if goals are aligned IMO.
I hate whiteboard programming questions and I don't give them when I interview someone - I give them a laptop with 10 different languages on it, and some data to munge -- I think it's a pretty decent thing for both parties.
From my own experience - people that do well in such interviews are good generalists. On their own they will start discussing performance improvements and ways to parallelize the solution, it's a pleasure to have such an interview.
It's about enjoying problem solving and willing to keep your brain fit. It has nothing to do with memorizing solutions to some existing set of problems.
"We don't look kindly on trademark genericization."??
GEORGE: You know what I do at the Yankees, when one of these old guys is breathing down my neck?
ELAINE: What?
GEORGE: You schedule a late meeting.
The people vilifying Max or saying "duh you didn't get hired, Google requires awesome people" seem to have a totally warped sense of exactly what Google engineers do day to day.
Blows my mind that there are so many people defending this (well-known and pretty much taken as a trade-off) lapse in the Google hiring algo and instead making it seem like Max's fault.
Maybe we are jealous. We wish we had the brains of the people getting into Google. I can say personally that I envy these people.
The fact that they don't mind being treated as numbers maybe says something about these people too. They are cold. Their ego must be pretty big too if they make it into Google.
Someone should do a study...hehe.
Alumni's response that it is exceptionally tough. He shared an anecdote about one of his best hires.
The alumni wanted to test the interviewee's knowledge in different areas. He asked a question on diodes - it might not have been diodes, but for the sake of the story let's stick to diodes. The interviewee replied: "Hold on, I am not one to know about it."
I have not worked at Google and I do not think I'd pass its interview process. It is unlikely that I be diligent enough to make Homebrew. Nevertheless, I am inclined to the idea, that being knowledgeable in all tested areas would not reveal the personal fit necessary to make a great team.
The recruitment process (at least for experienced engineers) should be little more then "can I work with this person". The 6 month probationary period that follows the hiring process should be used for "can this person do the job well". But that's just my experience, and it seems to have worked well.
Regarding the same academic questions everybody gets asked in every development interview, I feel Einstein said it best with "[I do not] carry such information in my mind since it is readily available in books. ...The value of a college education is not the learning of many facts but the training of the mind to think."
If they try to invent something new or in different market they might need different type of people but as of now ads business is cash cow and they would be crazy to try change it.
I think the stated philosophy of interviews like this one is that a false positive is worse than a false negative. Every single one of the responses to that tweet either misses that point or sounds like little more than defensiveness in the wake of a bruised ego.
You might disagree with that interviewing strategy, but you're not addressing it directly.
I don't doubt that the engineers who manage to jump through all those hoops are sensational. Personally in the end it just dawned on me that I didn't want to work at Google that badly.
The whole Twitter exchange is a pitiful sour grapes circle jerk, and I'm surprised that it's provoked such a massive response.
I agree with your overall point (why not both?) but how does the interviewer know they can do both if they are only tested on their ability to solve problems on a whiteboard? I don't have a solution (that scales) to this problem but I tend to agree the types of whiteboard problems typically seen in interviews are a bad way to identify good developers.
First a little deduction. Let's try to be explicit about the theory behind this technique.
It's safe to assume that the job will NOT consist mainly of cranking out binary tree inversions on whiteboards while being watched over. So obviously we're hoping to make a correlation with something else. Assuming the candidate was not tipped off and learned this particular puzzle, perhaps we are correlating to an ability to rapidly create novel solutions of long-solved algorithms without reference tools.
But is that what the new hire will be doing? Probably not.
We could continue down this path, identifying ever more removed correlations until we get to something that the job actually demands. This probably involves solving hard problems like naming things. [1] But by now our theory stands on pretty thin ice indeed.
In any case, all of this deduction is theory making. It's not knowledge until we attempt to falsify [2] it via induction. The human mind constantly induces hoping to verify our deductions. We reason, observe, conclude and repeat. We're good enough at it to survive, but that's about it. Lucky for us, science came along. Today's technical hiring is at best alchemy.
A interesting company called TripleByte [3] is trying to apply induction (first for YC companies). They specially shun on white board coding and puzzle solving tests in general. I will be interested to see how they fare and whether their learnings are adopted more broadly.
[1] http://martinfowler.com/bliki/TwoHardThings.html
Without discussing too many details, I believe the issue with Google's recruiting process is it was designed when the company was smaller and it follows the philosophy that anyone that goes through the hiring process should be ready to be thrown into any of the many Google projects and be able to function immediately. That's not strictly true anymore.
You have some divisions that are extremely hardcore or require very good knowledge of a particular field (think Google Cloud Platform vs. Android Kernel vs. Chrome vs. Search, all completely disparate projects), but there's also work for people that don't need to hold a PhD from MIT (think front-end development.)
On the other hand bad engineer will stay at the company, lower standards, damage morale and set bad precedent to other engineers.
Being engineer myself, I feel much more motivated working in an environment where you can just assume, even before meeting, that the other person is intelligent and motivated. You trust hiring process to filter everybody else so you don't have to subconsciously distrust every person you meet.
This comes at the cost of situations like that.
If you're spending 45 minutes writing 5 lines of code, it is not definitive, but certainly a red flag.
Being a front-end guy, I proactively request for hiring manager or a senior front-end engineer from the team.
Would love to hear people's thoughts and feedback!
Not to mention for a qualified professional candidate, it feels an awful lot like you're asking them to work for free.
> The code is reviewed by the interviewer > but probably never run.
This is what surprised me during an interview with Facebook. I tend to write a little code, run it to test it out, write a little more, test again, and so forth, you know, rapid iteration or whatever the fancy industry term is these days. The interviewer gave me these silly little shell scripting problems (silly in that they ought to have been easy and clearly had nothing to do with real life work) but instructed me _not_ to run the code. How the heck am I supposed to know if I've solved the problem? How will I know if I even have the right approach? I don't consider myself an expert software engineer and so probably not a fit for the position, but that style of interview definitely hamstrung me.
http://www.developingandstuff.com/2015/05/why-i-dont-do-codi...
[1]: https://twitter.com/markmcerqueira/status/608914346706657280
Would they manage to pass the process?
who does dynamic programming on a white board during a work day?
it's like you are recruiting an army, but testing people's gymnastic skills. they should test street fight skills.
Jobs come and go, great work is always great work, but friends are what I remember most.
At a certain point, you learn that there are other jobs out there, and maybe the one with the biggest name isn't the best one. I certainly wouldn't be the least bit offended. Not when the market is flush with high-paying-jobs-a-plenty, particularly for someone with his background.
Walking through the cafeteria made me feel like I was back in CMU CS again, in a bad way.
In my nearly 3 years here, that is my experience as well. I've also spoken to many engineers who have left who lamented the quality of their fellow engineering talent at their supposedly 'hot' startup, compared to their former team at Google. Or having to deal with way more unprofessional (or crazy) management, prima-donna teammates, etc.
Really smart people who are really insecure about their intellect?
If you wanted to work for an organization where everyone likes to show off their skills to one another in the interviews, you'd have gotten the job, and you'd be one of them.
The best interviews I find are like a first day of work (but unpaid). Your experience and skills are established by resume and portfolio. The interview shows whether you can work with the team and wrap your head around the org's problems. If you're having to show off- well, that's how your work will be, too.
Google's interviews convinced me, years ago, that there's no way I'd ever want to work there. And that feeling hasn't changed one bit. Much like FB, it's an org whose coding needs are really pretty trivial and the real work was done and finished a long time ago (but there's plenty of need for debugging, egos especially). If you want to work there and surf the gravy train, cool for you.
I was with you till riiiiiight there. Google isn't just a search company anymore they're working on lots of very interesting non-trivial things all around the company.
Petty, but fuck em. They don't want him, they shouldn't get the fruit of his work.
So, I've interviewed with Google twice. Once was 3 years ago, the other was like 2 weeks ago. They contacted me. The way I see Google's interviews is like referee in Football(soccer or "Futbol"). Sure, you need a certain amount of skill to play in the World Cup but you winning or losing can, and often enough does, come down to a controversial referee call. You end up losing out to a team that did a handball - https://www.youtube.com/watch?v=-ccNkksrfls , but that's just how it is. What makes me like watching soccer so much is the same thing that excites me about Google's interview process. Yes, there is heartbreak and anger. Just like World cup fans get angry when their team loses because they were denied a point for an off-sides call even when the player was nowhere near off-sides.
--- Is Google's interview process fair? Nope.
--- Would I subject myself to their futbol-referee style of judging candidates again? You bet.
--- Do I think they should make their process more fair? Nope, let the drama and _justified_ rant posts continue. Just like I want the unfairness in futbol to stay as is. I was one of the people against putting the microchip inside the ball to know for sure if it crosses the goal line. I want the drama of a ref having to call it and sometimes getting it wrong.
I know people, especially on HN, love reliable & repeatable. I do too, except when it comes to dealing with humans.
But how about Google didn't give them a job because this is how they handle failure, embarrass an entire company on Twitter to punish them.