Optimizing a breadth-first search
snellman.net
snellman.net
Edit: for context, wasn't meant to be "zomg how you so dumb" so much as "everyone should also read, cause is relevant".
Totally happy to believe I'm wrong about that, though :)
The Python version seems to do DFS using function calls, and I could imagine this isn't the most performant way to do it. Also, the description implies that their DFS implementation retains state; not sure what to make of it, and not a Python reader:
> Depth-First should be a bit kinder to system memory, though it'll still chew up quite a bit remembering which game states we've seen before.
I made a non-iterative DFS, but hardcoded the maximum depth to the optimal solution. On a trivial puzzle that should take <1ms, it takes 6 seconds. The BFS visits 303 states, the DFS 9M (but only 237 unique ones). This is with deduping the states on the path. If we allow the same state on the path multiple times, it'd be 12s, 50M visited states, 239 unique states. This is with a random visiting order, it'd be a bit worse with a static visiting order.
This is obviously not fully optimized code (whee, std::unordered_set). But fixing that won't help when we're off by 3-4 orders of magnitude. I think the shape of the search graph of this game just isn't well suited to any form of DFS, there are far too many alternate paths to each state.
Kudos for the encoding. 0.7 bits per state is dense.
I think IDDFS might work really well here, even without a good heuristic. It's worth trying, at the very least.
EDIT: read your reply to the other user. Fair enough :)
You might see some interesting effects using a bloom filter and an IDDFS. It would turn the whole search probabilistic but might be fast enough that you could run it enough times to remove reasonable doubt.
The author probably spent some time writing the code, modifying and extending the algorithm. He wrote deduplication code, memory mapping etc. Wouldn't it be interesting to know how iterative deepening performs?
There are literally problems that were 1) once covered in freshman year, 2) could once be recognized and solved by CS grads in seconds, 3) stump recent CS graduates, 4) prompt HN commenters to say how they could solve it if given a few days, and 5) come up in conversation if you go to meetups and talk to people doing actual work.
How does this relate to BFS? It used to be that someone trained as a computer scientist would look at a data structure or a graph and start running some quick gedankenexperiments: What would happen if I tried to find that with DFS? What would happen if I tried to find that with BFS? Those aren't going to be suitable solutions for all problems, but it's a good place to start thinking. There seem to be a large number of recent grads who can't even get that far.
There's a serious problem in the industry being driven by coding bootcamps; it's really sad to see universities being forced to stoop to compete.
Is that really the case, universities are trying to compete with bootcamps? I don't see much in the way of signs of that.
Interestingly, to me, it seems our industry was dominated by people that got good at gluing things together. To a very large degree. We bemoan this when we talk about how much more responsive machines used to be, but I think there can be very little denying that computers do more.
Granted, I suspect I am at best one of the bad graduates you are referencing. :(
The idea that "computers are fast, my code just needs to work" is a really really horrible way to treat your users' devices. A whole host of similar ideas are now popular, and it's pretty obvious that it's because the industry is now dominated by people that are (just) good at gluing things together. This is a bad thing.
So, seriously, do you have data showing that the bar is lower? Or are you just performing selection and survivor bias to get such a negative view? (Similarly, am I doing the same to get a positive view?)
My main qualm is people that seem to hold incoming folks to the bar that they have today, without acknowledging the growth that was necessary for some of that. As a parent, I fully know my children will be better at most everything than I am. In time. Same for most of the younger generation coming out of college into the industry. Most are or will be better than I am. Pretty much full stop.
Lots of those people who wrote inefficient things were the C students in Comp Sci. A lot of those people writing bad code were home-grown programmers who were completely missing pieces of knowledge. From what I saw as an undergrad, it wasn't the 3.75 GPA CS students who were doing those things.
So, seriously, do you have data showing that the bar is lower?
It's not my job to do such studies, however I do interview job applicants. A disturbingly large number of applicants, who all have 3.75+ GPAs, try to tell me nonsense, like "null pointers take up no data." I'd say I encounter something about as egregious as that with a bit short of half. In my recollections of interactions with classmates, I could take it for granted that CS students who made it past Freshman year knew enough to actually implement algorithms and could actually implement a recursive function. If you had tried to talk to them about such things as some kind of deep esoteric knowledge, they would've just given you a funny look.
I've similarly been doing interviews for upwards of 20 years now. There are some particularly bad interviews I've done recently, but there were some particularly bad ones I did towards the beginning of my career, as well. Worse, some of the senior engineers I used to be under were quite bad, all told. (Which does not take away from many of the ones I was with that were bloody amazing.)
Many of your supposed counterpoints are straw-man. No one is expecting someone to have memorized or to reinvent KMP or this algorithm or that. What we're looking at is basic understanding and conceptual tools.
I'm just not asking anyone to believe that things are truly different than they used to be.
But things are clearly different than they were years ago. The number of CS grads has fluctuated a lot, in response to increased investment and industry bubbles bursting. Things are clearly very different today than they were in the early 90's.
https://nces.ed.gov/programs/digest/d12/tables/dt12_349.asp
We also covered "gluing things together" in the early 90's. The concept was already very well worn even when I was a newly minted CS grad, with Jon Bentley discussing it in Programming Pearls using awk. The thing is this: We would also glue libraries together, but we did that while applying our generalist knowledge. It seems to me that there are a whole lot of CS grads who get through their entire undergrad education pretty much only doing that. Maybe that's all well and good, and they can accomplish many great things this way. However, it seems to me that much of the basic generalist knowledge is now mistakenly labeled as "specialist" and is needlessly missing from the general population.
It can, if those doing the gluing have the requisite knowledge.
The fact that you are classifying actually learned something when we covered BFS/DFS Freshman year as being equivalent to "the best of the best" speaks volumes.
My claim is that some of these optimizations on BFS/DFS style questions were never something most people could effortlessly solve in a few seconds. Which is how I took the claim of the OP.
If it was only the identification that was supposed to be seconds, with solutions taking time, that is one thing. But even binary search was infamous for not having a bug free solution for many many years. (Unless I took an urban legend too literally, of course.)
Not to mention, most solutions people give in seconds are at best a good starting position. Just go look at how a typical sort is actually implemented. Much more involved than what you would want someone to do in a few seconds.
I've literally seen folks that think someone should be able to write algorithms such as Knuth-Morris-Pratt in a standup interview. Which is just bonkers to me.
I've also grown annoyed with interviews that are effectively, "how would you design google maps today?" Which really just comes down to have I already done that. At the least, studied it fairly in depth. Worse, the answer is almost certainly not much different than how it was built.
Incorrect. My point is that some high GPA undergrads don't seem to have the habit/ability to even think along those lines. Of course we aren't going to ask for a difficult and polished algorithm in an interview. We throw simple stuff at them, to see if they know how to approach a problem. There are some common problems which are trivially solvable by imply throwing BFS/DFS at them. These 3.75+ GPA wunderkinds will flub those! At least one HN commenter replied to one by saying they could solve it in a couple of days.
Not to mention, most solutions people give in seconds are at best a good starting position.
Too many of these supposed A students can't even get to that starting position.
I've literally seen folks that think someone should be able to write algorithms such as Knuth-Morris-Pratt in a standup interview.
No. What we ask is more along the lines of: Would this candidate be able to think through what would happen if they used DFS on this graph? And that is just about the hardest thing we'd ask.
My priors would label this as just you getting more exposure to what is out there. Previously, you were likely mainly exposed to your peers. Did you personally perform more of the interviews in the past, or more in the present? At an absolute level, I would tend to agree that increasing the volume increases both good and bad. Where the contention seems to be, is if there is truly a lowering of the bar across the board, or if you are just personally exposed to more of the folks that can't make it?
And I confess I have no numbers. I am terribly optimistic and generally forgiving of interview gaffes.
Don't get me wrong, now that I do interviews, it has been somewhat baffling just how wrong some folks can be. The most troubling are the folks that seem genuinely clueless that they have done a poor job.
However, the "strawmen" I'm asking are not hypothetical. I've literally been asked some of those. My favorite was when I was asked what I knew was a de Bruijn cycle, but confessed I would be unable to code without reference. And even then it would take time. Interviewer insisted I try, and seemed to get frustrated that I didn't succeed. The very next interviewer basically asked me to do A*. Something I feel like I should be able to do, but quite frankly it has been a long time since I did those and I have grown used to reference material.
Similarly, I have been asked "how would you build snapchat/instagram" and "how would you build maps?" Both of which, I feel like I could have better answers for, but the truth is that I would start by trying to understand exactly how they are built today.
Ah, one of the pain points with companies in the Bay Area.
For the longest time one obvious, if not necessarily exclusive, problem with students who can achieve a 3.75 or higher GPA is that they tend to be much better at the rote memorization of concepts, but because they didn't necessarily "slog" through stuff, have some troubles, not try to just memorize formulas and algorithms, they don't have the ability to think through problems.
What about students in the 3.0 to 3.75 GPA range? Or are they considered "too dumb" to pass the resume filter you're using?
I think recruiting around here is targeted at "best and brightest" too much, and over-simplifies just the kind of people fit that qualifier.
I remember applying for internships while I was going to SJSU and many of the top companies in the area would have a drop down selection list of "which university did you attend?" and the list would only consist of the usual top UC's and top private schools.
I'm not saying actually smart people can't come from those universities, but when you're doing things like limiting a candidate search to 3.75 or greater GPAs, or filtering based on only "top" schools, then I think the issues you're describing are going to be much more immediate than otherwise.
There may be some grads that have the problems that you describe but there are also many that have those skills. Look at the ICPC regional contests. Many students competing are very highly skilled in algorithms and hacking things together quickly. That does not always make them good employees.
https://www.pbs.org/newshour/economy/column-how-an-epidemic-...
Interviewing people does not tell you the capabilities of people - interviews are very artificial processes.
I'm not exactly a computer scientist (or a recent graduate), but I work with tradeoffs every day, most of which are data derived, not problem derived & that is only visible from experiments on the data (sometimes across millions of ops).
However, what I really end up doing is actually different from just running experiments - I run the scale tests and go running while it runs.
I do my best thinking when I'm running - about a 10 minute mile pace in sunshine gives me my best ideas, perhaps it is a heart-rate thing.
And even when I have an idea, I will dig through researchgate for a couple of hours before I actually start pulling it apart (often, I find people have described the math which helps me think better) - like HyperLogLog used for IN() estimations.
Nobody's going to allow me to do any of those things in an interview.
Interviews are really about selecting people to hire, not about their abilities.
I don't think I'd know how to interview someone for your position.
I do my best thinking when I'm running
I've had similar experiences. Of course I'm not possessed by the idea that interviews are very good as a general mechanism. I've given allowances for nervousness and pressure. I certainly know what it's like to be on the other side of the table. Even allowing for all of those things, it leaves me wondering.
I hope this is a joke, although it's a little scary totalling up how much Chrome is chewing right now, with just one window and seven tabs open...