Announcing Starfighter
kalzumeus.com
kalzumeus.com
Don't get suckered into using a bot. They may be cheaper in the short run but you know you're going to get lifebanned. My prices are nothing compared to your future earnings.
The open source portfolio really is kind of a golden standard. It is real work that you have done, that has (hopefully) advanced the commons, because you love programming and have something unique to add to the world. Some fake video game programming may indicate something, but is simply not as strong an indicator.
Who are you going to hire- TJ Holowaychuk, who wrote Express, or the guy who placed 3rd on TopCoder in 2012?
Honestly, after mind-taxing eight hour job, you want to go home, watch a movie, read a book or some other relaxing activity. Not to mention that for people with families, it is nearly impossible to find the time to contribute. The rockstar-ninjas with thousand-starred repos on Github are very rare.
Then this person probably wouldn't find much time to do CTF challenges, either.
♯And let's be fair, 99% of what's "open source" is junk anyway, just because you wrote an 80% done static blog generator doesn't mean you're "advancing the commons"
I don't agree. Only a small segment of the talented programming folks I know do open source contributions.
It also depends on the type of problem. If you just need one weird trick to pass the level (e.x. use this SQL injection) then that's easy to game. If you have to write a bunch of distributed systems code (e.x. Stripe's last CTF) that's a lot harder to cheat.
For now, I'm assuming they have good protection against automated solutions. If it were just a matter of detecting plagiarism, you're right that it would be trivial. (Heck, I even cofounded a company that did code plagiarism detection, so I have a very good sense for how easy or hard it is. tl;dr: it's straightforward to defeat a plagiarism detector if you are sufficiently motivated and knowledgeable and willing to put in some time to go through all the code, but people/companies who plagiarize are typically none of those.)
Detecting whether someone else is doing the candidate's work, on the other hand, is impossible. Unless you bend the rules. Fortunately, the rules seem pretty pliable to me -- when a candidate arrives at a company's interview, spot-check them on their understanding of a few of "their" trickier solutions. It's a logistical nuisance, to be sure.
The verification burden shouldn't be on the hiring company, it should be Kalzumeus...
I don't think they'll really need to worry about these things for quite a while, though. Good problems to have, I guess.
Though that's where my scam would be most likely to fall apart. Recruiters typically only get their cut if the candidate doesn't get fired in the first N months. So if they purely cheated their way in, then the hiring company is only out a couple months of useless work. (Which could be very costly, to be sure, but at least they don't have to fork over the $50K. And there are many other reasons why a new employee doesn't pan out.)
I suspect that too, and this is something I'd really want to read more about - how does self-policing work in black markets and how effective is it. If anyone has any good sources, I'd be grateful for sharing.
RE fraud prevention, there were two interesting lines in the Starfighter's announcement:
> We assess for skill first, passively as players play our games and then actively. Our founders — talented technologists — personally reconstruct candidates’ solutions and evaluate them.
> We follow-up with players to ask if they have any interest in a no-obligation chat about career options. If they’re interested, we have an honest geek-to-geek conversation.
I suspect that between the first and the second there will be a place for Starfighter team to verify that the candidate has the skills his account shows. After all, if they want to maintain the access "deep into the hiring funnel", they have a strong incentive not to send bad candidates and waste their clients' time.
The scam you've described may be possible, but it's not very rational for someone to use it, IMHO.
Let me answer the most frequent questions we're getting right here:
"CTF" stands for "Capture The Flag". Conventionally, it's a contest with a collection of "flags" each of which is guarded by a programming puzzle; teams of people compete to collect flags. What we're doing is not a conventional CTF, but if you want to get the flavor of what we're doing (without the whole game dynamic), check out MICROCORRUPTION.COM, which is a more conventional CTF we ran last year.
Security is one of two problem domains we're starting with. But this isn't a "security recruiting" service, and our take on security uses it as a venue for systems and network programming, not for the minutia of SQL query quoting rules.
I'm excited to see this develop.
Apart from the gameplay mechanism with code challenges, how is this better / different to HackerRank for a) "programmers" b) companies?
What do you expect will be the effect of this on hiring women? Do you have any relevant data from microcorruption (what % players were women)? It'd be great if Erin could also chime in.
However, this is what I am hoping and dreaming for: that work-sample testing will completely level the playing field. Regardless of gender, age, or origin, if you can do the work, you should get the job. I believe in this so strongly that I am dedicating the rest of my career to work sample testing and, eventually, training.
Getting women to play, though? I'm not so great at marketing, so still working on that. I am writing a blog post to go up later in the week that addresses some of the issues in hiring women, based on my personal experiences and those of my female peers. The tone of the post is as politically neutral as it can be. My hope is to draw out some productive dialog rather than piling on stink for flies. Specifically, vast majority of the people in my professional network, male, female or in between, are genuinely interested in addressing the "women-hiring problem," but they avoid any discussion of it because ... well, it's always covered in flies.
The challenge will be to make just enough of a stink that it draws attention and not flies.
I am sorry, but that is a line with a negative width: if you so much as hint that this may in part be the fault of males (as a whole or, even worse, individuals) pitchforks will be out. If you do not hint strongly enough that this is the fault of males (as individuals and as a group) pitchforks will be out on the other side. As a more personal point the sheer toxicity of those debates continues to shock me, even after spending more than a decade on the internet. Vi vs Emacs debates don't tend to end in treats of rape and/or genocide but "why aren't there more women in computing" almost always do.
This isn't to say that we can't draw any benefit from your writing, just be aware that of the things you can't say (http://www.paulgraham.com/say.html) this is the thing you can't say most.
That's a nice thought, though it rather presumes that the playing field is level outside of the immediate neighborhood surrounding the hiring process.
I think that neighborhood, though, is probably, while not without gender/race/etc. bias, overrated in terms of the proportion of that bias on the course between birth and getting a software job that it is perceived to contribute.
Discerning invested participants from trolls might help.
I agree that bias is overrated as a contributing factor to this so-called problem. But it is one we focus on because we also believe that behaviors can be controlled for and habits can be rehabilitated. There are a lot of narratives where the circumstances are very different but the dynamic is the same. Someone whose parents want them to be a doctor or lawyer who aggressively derail them from pursuing the arts. Some very intelligent kid growing up in poverty with no access to resources who starts acting out and ends up with a criminal record. A woman who is forced to take HomeEc in high school instead of trigonometry (me).
Taken in isolation, these narratives can be explained away. As examples of a larger pattern, they become flaws in a system that can be engineered away. There's a meta-bias, and I'm still searching for a way to describe it that doesn't trigger an emotional response. It seems that until we can have that rational, engineering-focused discourse to identify the not level playing fields of the world, cataloging their characteristics, finding what things can be controlled for and/or eliminated,... sigh I don't know how to finish that sentence.
You say it's a nice thought. I believe it's a nice thought. More people hope for it than do not. What exactly is keeping all of us who genuinely believe that it's at least worth a try to apply a dialectic method to the problem from banding together and attempting it?
I don't know.
BTW, Tom was great at Matasano on #1, and only moderately successful at #2. We get tons of people who aren't in the industry, but they tend to self-select to be young men. For an individual company, #2 is way harder. It's "easy" to fix yourself, but hard to fight the larger culture.
Starfighter looks to me like something that supports #1 directly (by allowing objective metrics), and enables #2. It's a way for people who aren't welcome into a field to dip their toe in without some roomful of young white men asking them illegal questions about their child-rearing plans. On the other hand, there's still tremendous pressure pushing people away, and it's a deep problem. Still, part of the puzzle and I (personally) highly endorse.
As to the numbers, I've been meaning to dig into the data, but Matasano's hiring as fast as we need to, so it's been hard to motivate myself to do so. Maybe once graduation season ends...
-We die.
Fantastic name for this product.
Your last post on hiring definitely found its legs and I am sure will cited for some time to come. You mentioned that you used to send a sampling of key text books to potential Matasano candidates and was wondering if you could share some references. I imagine Applied Crypto might be on there. Any any other important resources for mastering CTFs you might pass along would be obliged ;)
http://sockpuppet.org/blog/2013/07/22/applied-practical-cryp...
- They are timeboxed
- They are mostly (if not only) about cracking security
- They are targeted towards low-level languages
You've already stated that it won't be timeboxed (which totally makes sense as a hiring "middleman": you're interested in applicants at all times). You already said that security will not be the single domain. Now, will I be able to use Starfighter as an excuse to finally stop procrastinating and learn that shiny new language I've had on my to-learn list for far too long ? (Patrick more or less hinted it shouldn't be the case, but I'd like to know)
I'm really looking forward to this. On the overall point of breaking the interview standard we have, I'd like to say a huge "Thank you". The points you've made in your blog post really resonate with what I can see (the interview process is a joke if you want to hire actual programming engineers). I really hope we can move towards a model where applicants can show skills through a portfolio, of which Starfighter should be a part if I understand things correctly.
The challenges start mostly bite-sized and typical programming problems (think fizzbuzz or "what is the 100th prime") and grow into some really interesting areas. You use whatever language(s) you wish, with no time limit, and are free to skip around to whichever questions interest you (or just do them in order... whatever you like)
I find it fun. You might too.
See http://bentilly.blogspot.com/2010/01/solving-project-euler-p... as evidence that I am not just talking out of my ass about PE.
I'm also really looking forward to this, especially from the pov of Hiring. Ours is an academic initiative, with no plans to monetize ever and its cool to see patrick and tptacek picking up the mantle on such a task.
By this I mean, the article uses some creepy language as only a company enthusiastic about its power to broker reputation can.
>>We can tell you exactly what happened when your candidates tried to implement a REST API.
What, are you going to keep a record of the time I fiddled with it for an afternoon and then use that to disqualify me from a job?
You say elsewhere about MicroCorruption:
>Microcorruption player identities were totally private. There's no way for a recruiter to look someone up, unless they used a very-identifiable username.
Sounds like a good starting point, but what if I don't want my report card stored and used to compare with other people? What if I don't want to feel like my performance is owned by you?
>>We assess for skill first, passively as players play our games and then actively. Our founders — talented technologists — personally reconstruct candidates’ solutions and evaluate them.
If I don't have any control over how and when my information is going to be monitored, I'm forced into an ugly position where I have to treat the whole thing as part of my career and perform competitively. The sense of relaxed exploration is killed, the idea of treating it like a college course or project isn't viable, and honestly I begin to resent the project.
Alternatively, I play on an anonymous account, and then if I decide to use my information about me in a professional setting, I'm best served by creating a new account and just redoing everything.
Edit: I should say that the idea really excites me and I'd love to learn in this format.
Yes. You can participate anonymously or pseudonymously. I think we say in three places that we only give out contact details if candidates ask us to.
What, are you going to keep a record of the time I fiddled with it for an afternoon and then use that to disqualify me from a job?
Our incentive is to find reasons why you're hireable, not reasons why you're not.
What if I don't want to feel like my performance is owned by you?
Your performance is owned by you -- we won't have copyright to your code or anything. Signal we gather, though? Owned by us. That's the trade: we give you a really fun game to enjoy, but we control the universe you play in and, inside that universe, we control everything and see everything. If you do not like this trade, that's your prerogative and I respect your opinion. Nobody will force you to play Starfighter.
If you feel that it is an accusation or a criticism, I would suggest that indicates something unrealistic in how you view business negotiations.
As patio said, this is a trade between interested parties. Generosity is therefore expected to be limited.
---
Fair enough. I just wanted it on record that this is a business transaction for you, not a generous educational endeavor.
Some people want to pretend that they're in it to make the world a better place when that's not truly their absolute top priority. They're probably asking "Why can't it be both?" when I say that this is business, not generosity.
One simply can't have multiple top priorities, particularly with profit and generosity; generosity is practically defined as something that doesn't profit. Something like one of your (patio's) blog posts are given generously in that there is no formal expectation/obligation on a reader to 'give back' some asset, for instance.
Thank you for your clarification.
If your top priority is a profit-making business, you need to provide a product that is (or, at a minimum, people perceive as) valuable.
Perhaps you can't have two top priorities (I actually am not sure I agree that this is the case, I don't see that it is actually impossible for two priorities to be equal in priority and above all others), but even if you can't, its quite possible for "business" to be a means and "education" to be the goal, and vice versa. They aren't incompatible.
Perhaps what you're saying is true in that universities have a financing division, but my point is that this isn't a case of university. The focus is on how they're going to make money as middlemen (and dwarves will sing about their riches), not how their business is subservient to their idealistic educational aims.
EDIT: In fact, socially rewarding companies that do good things incentivizes others to adopt ethical strategies and might do more good than a vow-of-poverty educational service provider.
Of course it's not immoral but still, knowing whether "a useful service" or "making money off it" is a top priotity for the company is important. Most of the companies you and I interact with are of the second type, and I guess this is at the root of throwawaymaroon's worry.
I understand the fear though; most of the companies we interact with (and I'd wager, 90% of startups we see here) are what I call toilet-paper companies - they'd gladly switch from whatever it is they're doing to manufacturing toilet paper if that would render more profit or increase chances of getting acquihired; their top priority is business, not the goal. I hope that Starfighter isn't such toilet-paper endeavour.
I naiively believe that things could be varied enough that you would need to understand the concepts rather than paste answers, in which case "cheating" would mean "learning", and is basically something they say they want you to do. I'm not sure how hard it would be to get to that point rather than being vulnerable to Bob the Super Coder posting walkthrough screencasts, but I trust that the founders are already savvy enough to have been thinking about that.
A certain MMO I play recently had a limited-time event built around figuring out the meaning of different clues (locations to go to for the actual meat of the event), and despite a fairly large number of variations, people had collectively figured out just about every possible clue->location mapping within a matter of hours.
That's not to say you can't prevent cheating, but that even with relatively little incentive (that whole clues thing gave only a single cosmetic item, and anecdotally I've seen very few people actually use theirs) users can and most likely will outpace any attempt to prevent it by means of varying the problem.
English teachers do the same thing for detecting plagiarism. Bring the kid into your office and talk about the essay. If he's completely clueless, someone else wrote it for him.
Overall, I get more value from tests that are constructed so that I can learn positive things about the candidate - as many opportunities as possible for the candidate to distinguish themselves. If the only reason for a particular approach was to provide the opportunity for immoral candidates to weed themselves out by committing obvious plagiarism (negative data), then there's probably a better approach that tells me more about the candidate per unit time.
If I was going to continue, I would use a problem that is (1) more representative of the actual work being done by the team; less of a puzzle (2) custom designed for the team or company; not a preexisting or well known problem. (Even candidates who don't cheat can have an unfair advantage on well-known problems if they have coincidentally encountered it before! Another reason to use unique questions.)
This reminds me of the debate over whether performance-enhancing drugs should be allowed in mathematics. Why do you think it's so important for the candidate to personally invent every aspect of their solution? What if you just told people that it's ok to use external resources to solve the problem?
A class might give exams in any of these ways:
- exams only happen in class, where everyone can notionally be supervised
- exams are take-home, but you can't read the textbook while you're taking one
- exams are take-home, and you're free to read the textbook
There's cheating under all of those models, including the first one which takes the form that it does specifically to prevent cheating. The implicit goal (for the students) of model 1 is to make sure they've internalized whatever is being taught. The implicit goal of model 3 is to make sure that, even if they haven't internalized the material, they're capable of applying it. The implicit goal of model 2 is to make sure they'll comply with arbitrary, unenforceable demands (in this context, usually called "the Honor Code"). That might make sense if you're hiring a cashier -- but is it really your first priority?
Do you feel similarly that the implicit goal of this model is "to make sure they'll comply with arbitrary, unenforceable demands" and still not to test internalization of the material?
One question: for a less "just for fun" site like this, with no time barriers, how do you plan on dealing with people just following write-ups? You can randomly generate the flags, but people could still follow the steps. You can say "don't write write-ups," but write-ups will still be written ;)
It's hard for me to go into more detail on (d) without revealing a whole bunch of stuff about the game I don't want to talk about yet; I should be more comfortable talking about it, but until we announce it officially I have a lot more leeway to slip rev1 features. :)
The shortest simplest answer though is: we're a firm whose whole purpose is to make fun, interesting CTF-style games (well, one game; we're the Blizzard of CTFs, and we're building our WoW), so we can address a lot of these kinds of problems with brute force, because this isn't a spare-time thing.
If it helps to understand where we're coming from:
Chris Eagle, the author of The IDA Pro book, published an IDA Pro plugin for the bizarro-MSP430 that Microcorruption (our last CTF) emulates. There are tools with "microcorruption mode" in them because of all the little ways we broke MSP430. Someone wrote a symbolic execution engine to solve the Hollywood level on Microcorruption and posted it to Github. There is still a #uctf channel on Freenode for Microcorruption.
This. Is. Awesome. It is my favorite thing about Microcorruption.
We did practically nothing at all to foster a community for Microcorruption, apart from Erin starting the IRC channel. That's not an opportunity we are going to miss this time; in fact, doing better on the community and sharing side is part of the thesis of the company.
I reallllllllllllly hope people share code and tools and stuff to make progress in the game. How cool will that be?
This is great but I think the stakes are somewhat different if it's ostensibly about jobs/hiring. I don't recall the crypto challenges being promoted as a hiring mechanism (though they may have been useful for that).
http://sockpuppet.org/blog/2015/03/06/the-hiring-post/
I think we plan on making minimal demands of our users, and none of them involve grooming them for prospective employers.
Look, the reality is, most of the people who participate aren't going to be looking for a job when they do. So all our incentives are to make the experience itself rewarding to participants.
I'm weird about typing those words because very very soon we're going to actually ship the first rev and levels of this thing, and as anyone who ships software knows: right now, at this point in the release calendar, my instincts are to be LOWERING the bar, not raising it. :)
>most of the people who participate aren't going to be looking for a job when they do.
This is weird, and I'm sensing some miscommunication between you and patio.
I'm only hearing about this as a tool for proving my worth. Your major marketing (as far as I'll likely ever be aware) has CAREER CAREER CAREER stamped all over it.
People won't follow the honor system if the stakes are at the 'career' level. You're losing the 'fun and free' culture of Microcorruption that makes people spend their free time building fun tools.
Your understanding of the incentives differ from the incentives that have been communicated to me about Starfighter.
You're fired.
Anonymous Throwaway Account? Yes, you over there. I'm looking RIGHT AT YOU. Yes you.
You're the new CEO.
Get to work. Explain this to everyone else on HN. The clock's ticking!
But I'll play along. I'll be taking Starfighter in a new direction. Most notably, we'll be reorganizing how we react to online discussion. Anonymous critics will be summarily executed unless we cannot identify them, in which case they shall merely be barred from Starfighter for life, which, if you believe our marketing department (AND AS CEO I DO IN FACT I'M SORRY I IMPLIED IT WAS POSSIBLE TO DOUBT THEIR CLAIMS), will make it very difficult for them to find work in the hiring utopia that is the post-Starfighter process.
But that's the use from your (or Matasano's) perspective. I joined the crypto-challenges not at all because I want a job in security, but because I continuously heard people be super-enthusiastic about it (both the participants, as well as yourself, tptacek :) ), because it reminded me of the old Malattia+ 3564020356 puzzles (level 6!), because it seemed more fun than the Euler Project puzzles (which I did enjoy, but you can only solve so many palindrome prime puzzles before it gets tedious) and of course because I would learn things about practical crypto.
Unfortunately I only got halfway the first set of the Matasano challenges, but that was more because I did it in Python and at some point got frustrated by its lack of speed :) (even using NumPy) I did make a rather elegant English-text MLE detector using a log-probability frequency table of only 256 bytes :) I thought that was pretty cool. I might have another go at it and this time use Java instead.
I know this whole thing is still in flux, but does it look like there are going to be any remote work opportunities in the early days? I'm probably in the minority but given my personal circumstances (living in a non-tech-rated US city and not considering moving) I'm really only available for remote.
I don't know exactly what the "rules" are going to be, but what rules we have will have one purpose: not to screw the game up for anyone else.
Thanks :)
My best wishes in your success!!!
I look forward to the results.
Does a company need to be hiring 10 or more engineers to make it worth it to talk to you?
I'm at a small company, that has been trying to hire engineers for the last couple of years, and not having much success; we've hired a couple, had a couple take other offers, lost a few to attrition.
We would love to be able to get some candidates who have already passed a technical screen; less time for us, scheduling an hour with a bunch of candidates who don't pan out, less time for them, only focusing on one more in-depth set of problems rather than a whole bunch of different phone screens.
But we're not hiring 10+ developers any time soon; that would double the worldwide size of our dev team. We're looking for maybe 3 or so at this point.
Would you consider working with smaller companies as well?
Did you read Thomas' "The Hiring Post" which was at the top of HN a couple days ago?
I would love to improve our hiring process, and include more realistic work samples as part of it.
However, I don't feel that a full-fledged work-sample test is realistic, especially if every employer started requiring it. Imagine applying to a few different jobs, and each requiring you to do 20 hours of learning of their material followed by some open ended realistic problem, that could take many hours to complete. If you were looking around at a few different employers, this would be a full-time job itself; some candidates would skip it because they could find a job which required less effort up front easily enough, some would simply not have the time because they are working a full-time job while doing it.
On the other hand, if there is a shared work-sample test like this one, that will be used by a number of employers, it means that it can be more in depth than any single employer could do, while simultaneously wasting less of the candidate's time as they do the one test, rather than one per potential employer.
So yes, in a way I am viewing this as a replacement for a technical phone screen, because I am interested in improvements to that process but haven't come across any potential improvements which seem viable, but this one sounds like it could have potential.
The "read the resume, read their existing code samples, and do a phone screen" approach covers a lot of ground in a little time, so while there are a large number of problems with it, it's pretty hard to improve on without spending a lot of the candidate's time to get up to speed on a more realistically sized problem, and a lot of good candidates aren't willing to devote that much time unless they really, really want to work for that particular employer. We're a small company, working in a niche business, so it's hard to get that kind of dedication from a single candidate.
When you design a test -- any test -- you should make sure that 1/ score correlates with success (i.e. full bivariate correlation), and 2/ if the test is voluntary, that there are no prior biases which would select candidates prior to taking the test.
As a reminder: this is an outreach strategy we deployed at Matasano to enormous success. We had an English professor finish the crypto challenges. We didn't solve all our diversity problems, but we made a palpable dent in them, and we did that by coming up with something that surfaced aptitude that wasn't held hostage to the biases of random human interviewers.
There is, to my mind, no hope for the "random human interviewer" hiring strategy. It's hard to make the problem we have now worse. But we're watching out for it.
I would love to hear more thoughts on how we can address this at the level of game design! We're at a "finishing touches on infrastructure, just starting with level design" place in our design right now.
The best way to address it is to first measure for biases (run a demographic analysis on players), and second actively design for unselected populations with focus groups. I am sure some of this information has already been collected by game studios.
For example, suppose $GROUP is over/underrepresented among top starfighter performers. What will be your new belief about $GROUP's technical talent and the causes of over/underrepresentation in tech in general?
Somewhat relatedly I'd like to add that while the result of an experiment in dynamics will obviously change my theory, it will in no way change my values[1]. Privilege based on an immutable characteristic such as intelligence (assuming such a result were to be obtained) is no more arbitrary than one based on bloodline. A smart person, though she will obtain it, deserves no more power than the average bloke, just as a nobleman, though he will obtain it, deserves no more power than a commoner. This is one reason the original meaning of "meritocracy" is satirical[2], as it does not change anything other than for the worse, by making the wielders of power believe that they actually deserve it (like the nobility in ancient times but unlike more recent ruling classes such as the American WASPs).
So far I have found that people who believe that the variance of some traits such as intelligence between population groups is dominantly the result of genetics do so because they think it provides a moral justification to the social order: things are as they ought to be because nature dictates so. I see no connection between the two. Nature (if it is, in fact, at play) has little bearing on ethics[3], and thus can, at most, explain but never justify an unfair distribution of power.
[1]: http://en.wikipedia.org/wiki/Fact%E2%80%93value_distinction
[2]: http://www.theguardian.com/politics/2001/jun/29/comment
My disagreement with you stems from the fact that I don't believe in any privileged population groups - I only believe in individual rights. I may or may not disagree with you about "power" but so far you've yet to provide a clear definition of it. (I did read your wikipedia links, but they provided multiple disparate and unclear definitions.)
Assuming his game tests for skill and skill alone. Though even if so, it wouldn't explain the difference -- just report it.
> I don't believe in any privileged population groups - I only believe in individual rights
I don't understand. One of these things is normative (individual rights) and the other is positive (privileged population groups). The existence of privileged population groups is a matter of fact[1] -- no one thinks they should (normatively) exist. As to individual rights -- everybody believes in them, too. The question is what would be their nature. For example, I believe that if the wealthy were allowed to wield their power (money) over the poor unhindered, then the poor should be allowed to wield their power (numbers) unhindered over the rich as well. The point is that power, by definition (see next paragraph), means restricting in some way the freedom of others, so to obtain freedom you must either restrict all power or unloose all power.
> they provided multiple disparate and unclear definitions.
Perhaps, but not different enough or unclear enough to preclude study or reasoning. The gist of it is, power = the ability to bend (or sway) others to your will. Power is measured by how many people you can sway, and to what degree you can sway them.
[1]: For example, that white men are more privileged in America than black men is a fact.
As a person who cares only about individual rights, I don't care if membership in some particular group is correlated with lack of privilege.
If you want to argue that lack of privilege is an individual injustice, fine - but then you need to stop discussing race since there are plenty of privileged blacks/women and underprivileged asians/males.
...not different enough or unclear enough to preclude study or reasoning.
You assert that white men are more privileged than black men. The definitions you've provided are insufficient for me to concretely state a test we could run to to disprove that.
For example, being black will sway college admissions officers for you but police against you. How do any of these definitions allow me to say that on balance, these things are negative? At what magnitude would the balance become positive?
Anyway, this is completely tangential to starfighter.
And what if lack of privilege is caused by association with the group?
> If you want to argue that lack of privilege is an individual injustice
You may believe in individual rights, but you can't deny group injustice. Blacks were made slaves not due to any individual selection.
> For example, being black will sway college admissions officers for you but police against you. How do any of these definitions allow me to say that on balance, these things are negative? At what magnitude would the balance become positive?
You can't possibly be serious. But just in case you are, there are clear tests to measure power: money, positions of control in the private and public sectors, positions of control in the media. QED
And insofar as such a thing is an injustice, the fact that it's correlated with some group is irrelevant to me.
If the only way power can be measured is via outcomes (in this case, I guess a high developer salary?), then I'm not sure why we need a new word to describe it. I also don't get what your point is. I guess you are arguing that smart people don't "deserve" the money/control that comes with a developer job any more than dumb people, and therefore starfighter is a bad thing?
That an electron is measured through its effect does not mean that we don't need to describe the electron itself. In fact, it is crucial that we do in order to understand its action. Same thing for power: you almost never observe it directly, but its study, and the term itself, are required in order to understand the workings of society.
And as to this game, I don't think it's inherently bad at all, but if its role in society is not studied it might become an unwilling accomplice to injustice. And, if, one day it is somehow discovered that abilities that convey power are dominantly genetic, I do not think that we should give certain jobs to people unsuited to them, but our society is judged not by the achievements of those born to privilege, but by how it takes care of those who lack it. Exactly how this moral obligation should be carried out is a complex matter in itself, and far beyond what I can write here.
We certainly do. So look at what physicists did. First they came up with a clear definition - "a discrete and indivisible negatively charged particle". Second, they went out of their way to distinguish concepts like an electron from charge as a fluid and other models. They didn't simply declare "well, electric current proves electrons exist", they went to crazy lengths like Millikan's oil drops to distinguish these concepts.
When they can't actually distinguish these concepts intrinsically (as was the case with Maxwell's equations vs Aether), they tend to drop the more complex theory.
In contrast, you seem quite resistant to doing any of these things. I don't quite understand why.
Actually the example I’ve give of the electron is a rather weak one, as the electron is a very specific thing, while power is the most fundamental concept in all the social sciences except psychology (i.e. history, sociology, anthropology and political science). A better example — the obvious one, in fact — is energy. Energy can also only be measured through its effects, and yet it is a very fundamental concept. Describing it not as one thing but as disparate manifestations would take away its explaining power and many of our most useful models (I should really write a book called “social science for physicists”).
I’ve also given more thought to your focus on individual rights. Individual rights are an obvious “good thing”, but here’s where they get complicated: When I think of individual rights I imagine a universe composed of mass but no force. Such a universe will be no more than a cloud of plasma. But in our universe, mass gives rise to force, and force creates the interesting interactions that have, in turn, created our world. Likewise, human society is not made of humans, but of human interaction, interaction gives rise to power, and power, by definition, restricts freedom. Now, this is not a bad thing necessarily, as, if you think about it, all cooperation is basically the voluntary yielding of freedom in order to concentrate power for some common goal that wouldn’t have otherwise been achievable. But that doesn’t change the fact that whether they like it or not, just as mass-ful particles induce force, humans induce power, and both force and power take away freedom from others. So saying something like “everyone should be free” makes little sense, as that is only possible in a plasma cloud. In order to grant freedom, freedom has to be compromised — based on some values — and then managed somehow. Obviously, different people will prefer different compromises. Personally, I’d either like to see all power restricted and controlled, or all power unrestricted (including physical violence).
This is why I think that American libertarianism is either hypocritical or ignorant. It is either ignorant of the fact that there is no freedom without power (and hence, coercion), or hypocritical in calling for unrestrained use of certain forms of power alone (money) and not others (physical violence, preferably mediated by a democratic government that restrains the use of money).
I was therefore delighted (intellectually, that is) to learn recently of a fringe Silicon Valley movement called neoreactionism or “Dark Enlightenment”, funded by Peter Thiel. These guys (few women would join that openly sexist, openly autistic movement) are probably all former libertarians that have discovered that there is no such thing as freedom from power, and now openly call for a tyranny. As someone who’d studied medieval history in graduate school (though I have never obtained my doctorate) I was delighted to see the movement’s leader, a programmer by the name of Curtis Yarvin, analyze some historical document and call for the return of feudalism (he complains that those documents are not studied by historians, which is true for the simple reason that they are false accounts).
The problem with the return to feudalism — even if you were to believe the false accounts of how life was good then (it by no means was) — is that the power structure back then was at least held in check by technology, that is, lack thereof. With the invention of mass media and fast transportation, power can be more concentrated than ever before, which is why the greatest invention of Western civilization was the central government, which rose to contain and manage power (of course, this only made conflicts among those governments more violent than ever before, but that fear of annihilation reduced the number of conflicts considerably). BTW, the modern academic definition of politics is, not surprisingly, the management of power in society.
Feudalism combined with modern technology has only been tried — to the best of my knowledge — once, in nineteenth century America. The US at the time had a very weak federal government with almost no regulation. The result was a period of extremely fast economic expansion but at great social cost: a large portion of the American population was enslaved in all by name by a very small number of slave-owners-in-all-by-name known as the Robber Barons (Rockefeller, Carnegie, Mellon, Stanford, JP Morgan, Frick, et al.). All options were taken from them — they couldn’t migrate (they were sometimes paid in company-issued currency, that was useless anywhere else) and couldn’t organize to concentrate power to improve their lot (in fact, they did organize, but the robber barons had private armies that killed the rabble rousers and intimidated everyone else). The people cried for help, and Theodore Roosevelt rescued them by creating federal regulation.
If you want to make a claim that one system has more energy than another, testing that claim will be easy - just apply the formula. If you wanted me to take it seriously, you should have applied the power formula to figure out whether influencing college admissions officers > influencing police - all you did was scoff.
I assert that your use of the word "power" is pointless. It transmits no information about the world, much like how you use privilege. Why do you insist on engaging in long discussions advocating the use of words that mean merely "any probabilistic cause of social outcomes"?
These "long discussions" that you show disdain for are attempting to introduce a semantic base on which "well defined" terms can be further developed and evaluated in light of new social experiences (e.g., the latest claims of * -ism in SV). pron has set out an extended metaphor, say, which is akin to describing how one would perform an oil drop experiment. That is the opposite of "resistance". Your assertion that "power" and "privilege" are semantically empty is completely without grounding. Just because the metaphoric equivalent of "applying the power formula" has been left as an exercise to the reader, does not mean that the terms of the equation are void of meaning.
The fact that intractable interactions lend themselves less easily to formulations (let alone closed expressions) does not take away their reality or invalidate the model. Much of the work in applied mathematics (non-linear equations) is qualitative, as well. I think it is you who are resisting to admit that the past decades of research have taught us a lot about how society works.
I.e., suppose a group has good outcomes but low power or vice versa. How can I find out?
So, to me, your question sounds a bit like, "how do we know when the planets in the solar system move by gravity, and when they move by something else?", to which the answer is that the planets almost always move by gravity, except very rarely, when, say, hit by a particularly large asteroid; how doe we know when that happens? We look. Same here, if a group has good outcomes and low power and vice versa -- while very rare (as power is at the core of the mechanism), you can either study the case carefully (which is what historians do), or compare it with power's known outcomes to see if it's one of those flukes. But, you'll say, I can isolate gravity and test it in a lab to make sure I'm certain this is how the planets move. Well, experiments like that are harder in the social sciences, but they are done quite regularly. Two very famous experiments in power are the Milgram experiment (testing authority power) and the Stanford prison experiment (testing authority power as well as its effect on those who have it). Many dictator games are experiments in other forms of power.
Besides, I don't see what exactly you're driving at. Thousands of studies have uncovered some mechanisms at the very core of human society. The mechanisms behave similarly enough to warrant a name (kinetic energy, potential energy etc.), and that concept seems to be at the heart of what drives most of society. Not only that, it induces a quantifiable (if sometimes only roughly, or even in theory) property. That mechanism, along with its quantifiable trait is called power. It was found to be roughly "the ability to bend others to your will", and has produced interesting, useful models (qualitative -- not quantitative). You want to give it another name? Fine, call it X. But 100 years ago we did not know about X as much as we do now. If you want to identify X with something that you think has been known for a long time -- you'll be wrong; if you want to identify X with something you think is still a complete mystery -- you'll be wrong again. You want to argue with scientists about the names they choose and then quote someone who says arguing about names is futile -- great. What is it that you're saying?
The concept of power conveys a lot of knowledge that has been gathered over decades. Your responses seem to be like those of someone who's just heard of energy, and says, "If energy is what moving things possess, why not just call it speed? Oh, a ball at the top of the hill also has energy, why not just call it height? Oh, fire has energy too? So energy is everywhere, and if it's everywhere then it doesn't mean anything!" Either that someone decides to learn basic physics, or decides to stay ignorant. But if he decides to stay ignorant, I think you would agree it would be foolish of him to continue arguing.
I turned the page. The answer was, for the wind-up toy, "Energy makes it go." And for the boy on the bicycle, "Energy makes it go." For everything, "Energy makes it go." Now that doesn't mean anything. What they should have done is to look at the wind-up toy, see that there are springs inside, learn about springs, learn about wheels, and never mind "energy."...Now that doesn't mean anything. Suppose it's "Wakalixes." That's the general principle: "Wakalixes makes it go."...It's also not even true that "energy makes it go," because if it stops, you could say, "energy makes it stop" just as well.
I claim that this critique applies equally well to your use of the word "power".
And your comparisons to real sciences are quite inapt - again, as I've pointed out to you before, a discussion with tptacek on crypto or kasey_junk on high speed trading results in the aforementioned posters being very specific while their critics are vague. Kind of the opposite of what is happening here.
Note that you still haven't actually provided an experiment or measurement that could identify a successful yet powerless group (if such a thing existed), or vice versa.
In any case, I don't see how that critique applies to power at all, because, yet again, some of the mechanics involving power are well known and well documented. Nobody says "power makes it so". It's just that explaining the power dynamics of racial neighborhood segregation or sexism in tech would take dozens of pages.
It is not simply that "power is what drives women participation in tech down". I can trace a process -- some documented and some hypothesized -- starting with "classical" gender roles, through the massive transition in gender roles and general separation between the sexes that occurred in Victorian times (they had rooms in houses meant to serve men and rooms for women) and shapes society to this day, through the history of women in computing (starting with the transition of switchboards from being seen as a job for women to one for men), with the more general association of which jobs are for men and for women. That would take me about 50 pages, I guess. But power is the central mechanism. I'm not saying "power did it"; I can show how. Just not here.
> And your comparisons to real sciences are quite inapt
Well, I've been using metaphors, naturally. The intractable sciences are much more complex than physics, chemistry and even biology. There are no closed-form formulas in the social sciences; at least not yet.
> a discussion with tptacek on crypto or kasey_junk on high speed trading
Maybe they're just better communicators than me, and maybe HFT is more amenable to discussion in HN comments than the history of gender roles and the evolution of power in human society. However, if you have specific questions (and they would have to be more specific than "how come there are fewer women in tech") I could try to answer succinctly if it is at all possible. The problem is that these are things that are never even taught to first-year social sciences students (some are only taught in grad school), and unlike with HFT, I don't think you even have the basics.
For example, I don't know if you're at all familiar with the techniques used to study history or sociology, how historical documents are analyzed, how different societies are compared etc., and I really can't lay out an intro to social studies here (BTW, that Curtis Yarvin guy I told you about suffers from the same problem, except he considers himself knowledgable for some reason. His writings read like an Aristotelian scholar discussing quantum mechanics; he's completely ill equipped to handle the materials he's using, which is why he draws such ridiculous conclusions. Of the months spent teaching students simply how to approach reading documents, he doesn't even apply the very first lesson: classifying the genre of the document and identifying the intended audience and purpose)
Now, I'm sure that there are some introductory materials to gender studies that skip the basics of social science, but I doubt you'll find them convincing if you're not familiar with the methodology. If you are interested, I could try to find some online course in history or sociology that seems good, but my guess is that they won't get to gender roles in an intro course (and if they do, it will be by skipping the groundwork, which, again, will make it seem less convincing).
> Note that you still haven't actually provided an experiment or measurement that could identify a successful yet powerless group (if such a thing existed), or vice versa.
You still haven't provided an example of a planetary system whose planets revolve around a star due to a force other than gravity! Gravity is what makes planets revolve around a star, and power is the mechanism by which groups (and individuals) obtain success. Once in a while there are aberrations, to which I have provided examples: winning the lottery. Or, if California is covered by the ocean, then the very powerful people who live their might become extremely unsuccessful. Of course because that population is powerful, various disasters would probably be addressed by the government faster and with more rigor than in other parts of the US, but that may still happen.
I asked for a measurement which could identify such a group, not a measurement that would. I can easily tell you experiments to test this in physics - solve Newton's law of motion and find a celestial body with motion that doesn't agree with it.
If I were advocating for the invisible roller coaster track theory of celestial motion, I couldn't provide such an experiment. The invisible roller coaster tracks are observable only by celestial motion - whichever way the moon moves, that's where the track is.
The only way to refute the theory would be via an alternate method of observing the position of the tracks and then observing whether the moon actually followed that track. If someone didn't provide that alternate method, I'd say he was not even wrong.
However, if you have specific questions...
Besides the one I repeatedly ask, you mean?
Examples from the middle ages include grants of knighthood[1] as payment for some unusual service. While usually a knight would only come from wealthy or noble families (or at least a family with good connections) -- hence, from a position of some power -- sometimes knighthood was granted to brave foot soldiers -- i.e. people with little power. Sometimes, the title came with land (and the serfs that worked it, of course).
In non feudal societies, social mobility was usually achieved through money, although some classes were barred from obtaining any money whatsoever (slaves). You can see groups of immigrants, provided the host society did not block their steps too much, slowly gain money, and later recognition and connections. This process would often take several generations.
Analyzing those processes is helped by the fact that often you can observe power directly. Money and nobility titles are very conspicuous forms of power, easily measurable directly. More hidden forms of power such as connections can also be traced directly (a boy of low background would be taken to the home of a merchant as a gift to his parents in recognition of some service; this lets you trace connections across classes); charisma (which in the middle ages was a great way to attain power in religious circles) could be seen in some extraordinary ascetic acts[2] or visions[3]. The latter was one of the few ways women could rise to positions of power in medieval societies (see Joan of Arc), although others would be marrying, and surviving, a man of power. While it was often expected of widows to remarry, some medieval societies were surprisingly relatively accepting widows, recognized their independence, and allowed them to transact on their own.
[1]: http://www.lordsandladies.org/steps-to-knighthood.htm
AFAIK Thiel's sole, extremely tenuous connection to the Dark Enlightenment is thinking libertarianism and democracy are incompatible. He certainly hasn't funded any of its leading lights. This is less accurate than the belief that the Koch brothers control the Tea Party. (They did have a large impact on its early growth.)
>These guys (few women would join that openly sexist, openly autistic movement)
Ableist. If you want an example of a woman who's been involved (more than periphally but not as an identified adherent) look up Justine Tunney. If you're transphobic then she doesn't count as a woman, ableist.
>are probably all former libertarians that have discovered that there is no such thing as freedom from power, and now openly call for a tyranny.
That's one branch of the trichotomy in case you're interested.
http://www.xenosystems.net/wp-content/uploads/2013/05/reacti...
http://www.xenosystems.net/trichotomy/
>As someone who’d studied medieval history in graduate school (though I have never obtained my doctorate) I was delighted to see the movement’s leader, a programmer by the name of Curtis Yarvin, analyze some historical document and call for the return of feudalism (he complains that those documents are not studied by historians, which is true for the simple reason that they are false accounts).
This is a massive misreading of Yarvin, a.ka. Mencius Moldbug. He's got a hard on for absolute monarchy, not for feudalism. The two are very, very different. Feudalism was basically a Western European phenomenom, Ottoman depotism, Russian autocracy or France during the reign of the Sun King are more his thing.
What false accounts are you referring to? Could you provide some links to the deceptive documents among Yarvin's output?
>The problem with the return to feudalism — even if you were to believe the false accounts of how life was good then (it by no means was) — is that the power structure back then was at least held in check by technology, that is, lack thereof. With the invention of mass media and fast transportation, power can be more concentrated than ever before, which is why the greatest invention of Western civilization was the central government, which rose to contain and manage power (of course, this only made conflicts among those governments more violent than ever before, but that fear of annihilation reduced the number of conflicts considerably).
Autocracy, not feudalism.
> Feudalism combined with modern technology has only been tried — to the best of my knowledge — once, in nineteenth century America.
This is so ridiculous that it makes me question your claims of having studied history at a graduate level. Have you ever heard of the Bolsheviks? They had a successful revolution in the Russian Empire in 1917 and founded and ruled the Soviet Union until its dissolution. They arose in a state that attempted to combine autocratic government with modern technology.
>The US at the time had a very weak federal government with almost no regulation. The result was a period of extremely fast economic expansion but at great social cost: a large portion of the American population was enslaved in all by name by a very small number of slave-owners-in-all-by-name known as the Robber Barons (Rockefeller, Carnegie, Mellon, Stanford, JP Morgan, Frick, et al.).
Are you aware that the USA has been among the richest societies on Earth for its entire existence? People were poor because there was so little to go around, not because people were hoarding for the sake of it. You occasionally say lucid and intelligent things but the US economy was growing insanely fast by any historic standards more or less from settlement by Europeans to around 1970. It's still growing insanely fast but the trend has slowed down. economic growth in North America was labour limited for a long, long time. Things were much, much better in North America than anywhere else on Earth, all this while having large inflows of migrants from much poorer nations, i.e. the entire rest of the planet.
>All options were taken from them — they couldn’t migrate (they were sometimes paid in company-issued currency, that was useless anywhere else) and couldn’t organize to concentrate power to improve their lot (in fact, they did organize, but the robber barons had private armies that killed the rabble rousers and intimidated everyone else). The people cried for help, and Theodore Roosevelt rescued them by creating federal regulation.
Did you study under Howard Zinn[0] or something? Everywhere else was worse. The Pinkertons were awful but the USA has never been a weak enough state to allow private armies anywhere on its territory. It was probably the friendliest country on Earth for labour organising for the period in question.
[0]http://en.wikipedia.org/wiki/A_People%27s_History_of_the_Uni...
As to your description of the Gilded Age, I am not sure what our points of disagreement are. Yes, the Pinkertons weren't actual armies, nor were they entirely ignored by the government. My description was greatly simplified for brevity. As to the conditions of workers at the time, comparisons to other societies are irrelevant, because that economic growth (which was the result of immigration, land expansions and new resources) was not at all contingent on the exploitation that was taking place.
In general, comparisons are often less useful when the framing discussion (in this case, questions of policy) is normative. If a tyrant comes to an island where the population is starving, and feeds them a loaf of bread a day but forces them to do backbreaking work for him, then their position is better than the alternative, but no one would suggest that this is in any way desirable or even ethical. The same would be true for another tyrant who feeds his enslaved population two loaves of bread a day.
Comparisons are useful if the claim is made that no other policy would have been possible, which I don't think anyone is making. Sure, growth would have been slower (concentrated power is always a lot more efficient, as time and resources are not required to achieve compromise and accommodate other stakeholders), but preference of efficiency over other human goals is purely a matter of value.
The Bolsheviks instituted an autocratic form of communism, which is pretty much the complete opposite of feudalism (which is usually the result of a free market, although the terms are anachronistic as no one used the term "free market", when real feudalism was actually in place), and in any case, feudalism is certainly anathema to any ideology promoting equality (like communism).
Hopefully the most reasonable one: that there's no such thing about the technical talent of "$GROUP" per se, though our society might presently be so organized that it's easier for members of $GROUP to excel at certain kinds of tasks.
How society is organized may be a cause for a group having higher/lower technical talent (so might "intrinsic" features, i.e. things not mutable by "society"), but it doesn't mean that talent doesn't exist.
How do you test for noncompetitive but still productive people using a competitive test?
I do think this is significantly better than the alternative, but I’m also concerned that it will just create a new class of people who do unfairly poorly in the hiring process.
Edit: I know this comment is unpopular, but every craft industry does work like this to some extent. As a programmer you have a portfolio, and all else being equal those who spend their free time coding have larger and more impressive portfolios. That's the career advice you always give someone on getting their feet into the industry: code more on your own.
In fact, I think a quality CTF could level the playing field, since each candidate could be judged on their relative abilities (how far they can progress through a game that adapts to their prior knowledge). Moreover, one could offset the time commitment by paying candidates to do the CTF. Certainly paying people a decent wage for 6 hours is worth the benefit of hiring a better long-term candidate. There are all kinds of ways to fix the problem.
I would have expected at least some experienced candidates to drop into that "top tier" of people. What's up with this?
Looking at the first page of the Hall of Fame, I see big names like Alex Sotirov, Russ Cox, Ricky Zhou, Ludvig Strigeus, and many other familiar names/handles usually seen at CTF events. There are also many unknowns, which I suppose was Thomas's point.
This accumulated advantage you describe is present either way. The length of the interview or CTF process is not going to change this.
That said, the perfect shouldn't be the enemy of the good here. The hiring process as it exists on March 9th 2015 is already insanely hostile to people who don't have scheduling flexibility to invest hundreds of hours to doing speculative work. "Hey, could you fly to a different continent for 3 days to do six rounds of in-person interviews?" is considered an attractive, reasonable proposition.
We can extract more signal than that gauntlet gives, in substantially less time, delivered in the candidate's own space and at their own pace. This is an unambiguous win for candidates with commitments.
I guess I can ask you directly: how much time do you spend interviewing people, not counting the work-samples you already require?
As for suggestions, it's really about the next year. Unless you've already been doing this, you have to find people, and place them. And that means working not only with the programmers, but also the companies involved. Part of that would mean coaching them on how to use your system. You want to destroy the current tech interview process? You have to replace it. You want to remove the tech portion? Then I think the best way right now is working directly with the companies you are placing people with. They need to play by your rules, and a year from now is when you'll have a better understanding of where you stand, as the person you've placed is hired, and still employed.
Obviously there is more than just one candidate and company, but unless that happens, I don't see a company just buying into your system without keeping it's own. So you need to teach them what it is you are providing, and what they should do to maximize interaction with the candidate. The interviews will still exist: I want to meet the person I'm working for before I show up. But instead of dealing with white boards and ping pong balls, we are having a conversation about my potential future with the company, and what I can bring to the table.
patio11 said it: "we’re here to destroy it, and create something new and better in its place."
A job interview has two sides, the applicant, and the company, and if you want to destroy the interview and rebuild it, you have to do it with both sides at the same table. Otherwise, you aren't destroying anything, just adding more red tape before anyone gets to the table.
* We demanded the same or slightly less time in total than other conventional-interviewing companies did.
* The scheduling of our demands was totally flexible, unlike interviews, which are rigidly scheduled. If you're a morning person, and you have next Tuesday free, that's when you threw the 2-odd hours you needed at the challenge.
* Obviously, it's easier and less stressful to put effort in from your couch than in an alien office environment with people staring at you waiting for you to answer properly.
* The challenges themselves were fun. They were real work: we didn't have people literally slaying dragons (they were breaking a web app and a client/server app), but they were the distilled enjoyable essence of that work with most of the BS removed. In fact, if you didn't find the challenges fun, that was a huge signal that we were the wrong job for you.
* If you did decide we were wrong for you, you could stop at any time --- randomly, on a Tuesday night, with a slice of pizza in your mouth --- and have none of the weird social pressures that would make you sit through a string of pointless interviews. Which is an experience I have had, more than once.
Regarding certification: pure, chill-filtered hatred of certifications is what got me into the part of software security I wound up in. There is zero chance that we are going to build something with the market dynamics of a certification.
I imagine that one might still get called for a 3 day interview, but I suspect that people using a work-test like Starfighter before asking for that will already have a MUCH better idea about your capabilities than they would have without it. Interviews then seem like they become more about assessing your fit with the company, and less about trying to weed out incompetence -- because they wouldn't invite you for an interview unless you'd already demonstrated your competence in this way.
That sounds a bit bad, if it were for only one company, but what excites me about Starfighter is that it sounds like something I'd want to do for fun __anyway__, and its signals about my competence (I hope! ;)) could then be given to multiple companies -- including ones I might never have thought to apply to.
I'd much rather hear, "Greetings Starfighter, .... we want to hire you to do more like that" than get spammed by random recruiters because my LinkedIn profile happens to have Python or Java on it.
At this point, my biggest fear is that I might not have the technical chops to complete the challenges.
I fully expect that I don't have the technical chops right now to complete the challenges, but I can barely contain my excitement to get started. Learning whatever it takes to overcome a defined challenge is so much more thrilling than learning a topic for the sake of knowing it.
More red tape happens because companies feel like they have the power to ask people to jump through whatever hoops they want. But you can reverse this. If you have the people companies want desperately, you can dictate what the companies can and cannot do (no 3-day interviews, for example). I think that's what Starfighter is trying to do.
Anyway, I'm happy to spend lots of time meeting and mingling with my future team and employer. Most people good enough to surpass these CTF's are going to be using the in-house interview time to be in the driver's seat of reverse-assessing their potential employer, realistically.
The interview for my newest job was spread out over two half-days (my choice versus one full-time day). I came back to visit and talk with people three times after I finished my interview and had an offer extended. My choice. I needed to really feel the culture. And ultimately that's what caused me to join the company.
Directly from the comment I'm replying to.
I'm glad it didn't go further.
I think this sort of kick in the pants is exactly the sort of thing that has to at least be tried to sane-ify hiring in tech, but I'm oddly fascinated to watch the ways in which little bits of status-quo inertia and company politics may act to sabotage it.
It is very easy to imagine companies not really "getting it" and seeing it the way they see github contribs or stackoverflow scores or whatever other metric that some do use as a signal, but which very rarely actually gets someone in the door or even on a particularly fast track.
In an ideal world, any sort of testing like this plus a screen to make sure you're not a psycho, a narcissist or just a general pain in the ass on a personal level should be sufficient for virtually any programming job, but there are lots of obstacles without obvious technical fixes: HR dept politics, the hazing culture ("We all did this stupid interview thing, so should the new guy/girl!"), general distrust ("nobody ever got fired for hiring someone using the old method"), etc.
I will be watching this experiment with great interest (and hoping it succeeds).
Starfighter doesn't make any money unless they actually place someone. Their pitch selects for people that aren't doing well in the traditional hiring process. So if a company tries to use its ordinary process, just putting the candidates in at the front, we can expect they won't do especially well. If a few candidates get sent out to an interview, have a miserable time, and don't get the job, I'd bet Starfighter will fire the client. Even when you are just selling to clients, you need to occasionally fire a toxic one. In a two sided marketplace it is absolutely crucial. The screened, high quality, applicants looking for a new job (or at least willing to consider one) are too valuable a resource to waste on companies that aren't serious.
Companies will process Starfighter candidates expeditiously and with dignity. Why? Because we're the work-sample company, and each candidate's experience is a work-sample of your hiring process. I see no need to introduce the best engineers in the world to anyone but the companies that most have their act together.
(Plus, yeah, obvious confluence of interests there regarding incentive compatibility.)
-Updating a resume
-Contacting a recruiter
-Replying to recruiters
-Searching job descriptions
-Writing cover letters
-Tweaking previously updated resume
-Sending applications
-Maintaining a GitHub profile
-Working on open source projects that may or may not fit a future job description
-Spending 1-5 days interviewing
And that's just off the top of my head. I'm sure we could come up with a few dozen more.
[1] a term I just made up, and may not correspond to something in reality
The fear is that there are a bunch of hiring fiefdoms, all waxing and waning, so that I have to spend a bunch of time on a bunch of them to stay relevant. I don't want to be laid off one day and then find out that these days employers are using StackOverflow scores or wherever and that I should have spent the past year working on that.
OTOH, if 1) I can easily find out that SoftwareCo is hiring through Foo-CTF without having built a Foo-CTF profile, and 2) I can spend, say, 5 hours building a profile at Foo-CTF and get a serious response at the end of that, that's good. It means instead of spending 5 hours doing the silly technical interview dance with SoftwareCo, I'm spending 5 hours doing the serious work-sample test with Foo-CTF. Foo-CTF's value-add is that they're experts at doing the work-sample test.
This fear is unplugged from the reality of recruiting incentives.
It is in the direct financial interests of companies to recruit you directly, not through us. To recruit through us, they have to pay for the privilege.
There are very few hiring mechanisms (including, you know, the whole application and interview process) that don't select for people with free time.
On the other hand, they may want to be a bit more selective to keep great programmers coming to the site.
(Sorry for being coy here. Don't want to jinx contract signing.)
If you had a deal with Google/Apple/Dropbox/NSA that said "If we find someone who can complete this challenge, you agree you hire them at $500k/year + $100k signing bonus doing work on Skunkworks Project X" you would have an incredible flow of hackers.
No one has created a marketplace for world class experts and yet these positions and people exist.
Congratulations on launching and good luck!
This is true for a subset of great hackers who are conveniently visible to the tech industry's antiquated, inefficient, insane, and exclusionary hiring practices. It is very much not true for many very talented engineers in the world.
We're going to arbitrage that inefficiency to zero.
You're geeks who've assumed that the only job a geek wants to have involves finding system-level exploits.
It's all very clever to write a program that reads use it's own bytecode as the secret, but does that feed people? Get them to Mars? Heal people? Does it make application programming easier, less error prone, more accessible? Perhaps there exists some insights at the lowest level of program execution that are only revealed in a career like this (a la _A New Kind of Science_), but somehow I doubt it.
But hey, everyone is welcome to their opinion.
If you want to write systems that improve the safety of application programming, it helps to have a lower-level understanding of what is happening.
As for Mars, if articles like this (http://spinroot.com/dcas/) are any indication, there will need to be a lot of systems programming and especially tool building that requires deep systems understanding.
It doesn't even have to be competitive. It's just... fun.
Virtually everything I know about computer science I learned because of software security.
I don't think you need to care that much about security to benefit from a syllabus of exercises framed by security.
I think you should copy/paste this onto your landing page.
That joy and excitement is totally gone now in Bob's CRUD Shop(TM), so it's phenomenal to have a curated lab to play around with (that won't get you arrested).
For example, if you're an application security analyst at Google and you find a security flaw that could result in users' personal information being leaked, aren't you making the world a better place by finding and repairing that bug?
If the goal is to find the top talent that wouldn't don't normally show up on a firm's hiring radar, then I would expect that includes finding people who aren't in those cities (and don't want to be).
I couldn't find any mention of geography it in your blog post or on the starfighters.io website, but I've been burned many times by initiatives where the absence of geogrphical information is supposed to imply "USA only" (because they forget that the rest of the world exists), or if you're very lucky "USA and select European countries".
- get a degree in any subject (money, family support)
- get a degree in a more pertinent subject (money, family support)
- continually learn new theory/techniques/technologies on your own (time)
- write FLOSS code (time)
- get industry certifications (money)
- get achievements with HackerRank and other 'challenge' services (time)
- go to interviews, answer questions (time, possibly others)
Not to mention all of the 'administrative' things people do to communicate their capabilities (as another comment noted. Those take time, too.)
Something like this seems like it is taking a stab at replacing at least a few of these, but of course it still requires at least some extra resources (in this case, time.) The go-at-your-own-pace nature of it definitely works in its favor. Things like this evolve similar to technology, and its cycles of consolidation and deconsolidation; where once we may have used separate devices (for music, video, reading, etc.), we now use just a couple, or increasingly, one. Perhaps we could think of things like this as the next steps in 'skill and knowledge acquisition consolidation.' Hopefully, things like this will reduce the need to spread efforts so thinly, and save us more of those scarce resources. So while it may be another monkey dance, perhaps it will allow us to dance less, but in a more focused way. Or at least stay on a smaller number of simian dance floors.
If nothing else, it'll be another fun way to learn new things and experiment with theory/techniques/technology you ordinarily wouldn't.
http://www.nytimes.com/2012/09/22/business/to-stay-relevant-...
What are some things a company like this could do to decrease dance-monkey-dance factor?
We're not interested in making monkeys dance, unless they want to, and enjoy it.
But, if someone loves to play CTFs and is seeking employment than its a great match. Me personally I'm a build things I want to see in the world and the job will come ... kind of person.
On hacker news you are getting a very small slice of the tech industry. And by reading HN you are subjected to repeated attempts to make you think there is a clear way to be marketable and to earn a living. Rest assured that there are people in tech that the HN crowd makes fun of that are earning fine livings and enjoying their jobs.
Your question is good though. If I understand you to be saying "what can we do to get you to drop one of those other things or you nightly watching of reality tv?"
Manager: That's because we don't have any "good" developers. We need to find some of those 10x productive developers and get rid of some of the guys we have.
Director: So you're telling me we can replace 10 developers for the price of one good one?
Manager: That should be true, but the 10x guy is going to want a bit more money.
Directory: Let's do this!
(some time later at the interview)
Manager: Were looking for highly productive programmers and we're willing to pay top dollar. We see you've accomplished X, Y, Z so we want you to come and join our team.
Developer: Sorry, I can't...
Manager: How about $50k more salary...
Developer: Wow, ok, I can't really say no to that.
(some time later the first day)
Manager: Welcome to your first day, here is your team. Get as much information from them as possible because we'll be laying off 10 of them in 30 days. Good luck!
Not the OP, but anything where a recruiter can go look my rating up, they will. And then that becomes a positive signal, increasing the speed of the rat race, because to be "hip", you have to do the N+1 things. Blech.
If you want to reduce the "dance" factor, have a sheltered tunnel from Starfighter to the first day on the job. Don't have it be a pipe to the tech interview, don't let it be a pipe to a recruiter's desk, don't let HR do anything but verify employment eligibility. Seriously.
How would you imagine the bizarro-world third party recruiter that would emerge from something like Starfighter could interoperate with employers to shield you from the kind of BS you're concerned about? We're interested in ideas!
"We provide you the names of people interested in your fine company, along with our certificate of credibility. You in turn agree that the interview process will be entirely on the intangibles[1] of interpersonal relationships, career goals, etc".
Along with that would be a rider that would involve Starfighter Recruiter LLC taking liability for providing a measurably crappy candidate, as well as disclaimers all around relating to intangibles.
Personally, I don't really want to futz around with long-winded negotiations and alpha male chest thumping. I want to demonstrate my capability, discuss my career goals and interests, and verify that fine hiring company is actually a reasonable place to work that meshes with me and my career aspirations.
[1] I.e., candidate is a loud-mouthed jackass that can't work with anyone, but can pass any technical challenge with flying colors.
The great employers don't require to much chest thumping and I've had some down right fun interviews. I question the culture of a company that uses a CTF game as a metric to hire people. People who play and succeed at CTFs tend to be hyper-competitive and thats not always a wonderful characteristic in a team member or company.
Thats fair, after reading some of your comments above. I believe you don't intend this to be more work for people. And, I believe hiring on both sides of the equation is a non trivial problem.
The underlying idea here isn't speculative. We used it at Matasano, and it was extraordinarily effective.
But it did seem from the outside that those doing ACM competitions were the upper echelon of the school.
Perhaps a CTF situation would be similar. We can only see what unfolds.
- Let players be paid by your clients to play for ~1 day.
- Introduce a non-game version that's a straight forward coding assessment.
- Make the game un-cheatable.
- Write into your contracts that you won't be the exclusive hiring funnel.
- Keep your clients confidential and don't let them mention your game in job ads (i.e., target people who just want to play a game).
Edit: I have to add, this does seem like a dance-monkey exercise and not something that an actual profession would have people do. My preferred hiring approach would just be contractor-for-a-day arrangements.
I’m not sure that method would be suitable for people who already are employed. At least in Sweden, where I live, you can (AFAICT) get fired for working for your employer’s competitors. So if you don’t get the new job, you might lose your current job as well.
There is, of course, releveant XKCD: http://xkcd.com/927/
I can say that I have hopes that if anyone can stop what you fear from happening it's these people.
One example would be MicroCorruption. https://microcorruption.com Thomas and Erin helped build it back when they were still at Matasano. It's a hybrid of a game -- there's a narrative, flavor text, a progression of difficulty, levels, a leaderboard, etc -- and a programming assignment. The programming assignment happens to be some variant of "Here's assembly code; find an input which exploits the vulnerability we planted in it."
Starfighter CTFs will similarly be a game that one plays by programming. Similarities end there. We can do very interesting things with this.
I fully understand that if your company is looking for pen testers, I'm not the guy you'd hire. That's cool, but since one of the aims is developing and refining skill, how approachable is it intended to be?
It is troubling that the language they use on the site implies that systems and security programming is what all programmers aspire to do, but that's just an unfortunate bias on the part of the authors. Personally, I think they'd do well to change it, but perhaps that's exactly how they feel and exactly the kind of messaging they need to attract the right people.
I have a background in architecture, its a mature industry and there are architects, structural engineers, mechanical and electrical engineers, architectural technologists, quantity surveyors, project managers etc.
Perhaps as coding as a job matures over the decades this process will also take place.
Also, fundamentally, a CTF is a distraction from the core business everywhere it runs.
Except Starfighter.
Or, much more commonly, it's a popular game mode in any number of multiplayer games. You're using some really niche jargon without properly qualifying it.
Wow, I didn't realize that, thanks!
I retract my point about "stealing names". I guess that this application of term CTF just wasn't known widely outside INFOSEC field and that's why a lot of programmers (like me) get confused, as security conferences become more recognizable in the mainstream.
right?
The wiki article has a short discussion of how that morphed into video game terminology, which is the etymology that I'd expect without having researched it.
Actually on further research, here's a nice reference: http://security.stackexchange.com/questions/3592/what-hackin...
Just the first header block read like they were creating some kind of videogame company... and then finally at the bottom of that text block, "Oh, it has something to do with interviews..." and then halfway down the next section, "oh, it's that kind of CTF".
Anyway, seems cool if they can make applying for jobs more interesting.
[0]: www.blueprintdecoded.com/
That sounds pretty bold - some of us are kind of happy to go in and do an interview and talk about our experience and not spend a lot of time playing a game. I could see the game as more likely to work for people with lots of time and not as much experience or proficiency with the typical process. I could see it working very well indeed to find people who might otherwise have been ignored.
In any event, it sounds cool, and I wish you guys the best of luck with it!
I know you mean well, but I improve my development skills by developing applications.
(b) For similar reasons to the ones that make you noodle with a programming language you doubt you'll ever use in production
(c) For the same reason anyone ever did anything with a BeBox
(d) Because for some of the technologies/concepts we work with, our dumb game will be the easiest way to get your hands dirty with them.
I play games for fun and to relax.
"(b) For similar reasons to the ones that make you noodle with a programming language you doubt you'll ever use in production"
On your website, it says that these will actually be real problems for real companies. I would think it's something I would see in production?
"(d) Because for some of the technologies/concepts we work with, our dumb game will be the easiest way to get your hands dirty with them."
Isn't this what open source is all about? Pretty much every job I've ever had involved technologies I could just download or install myself and throw on a Linux box/VM.
I suppose it could involve proprietary technology that I would never see in the wild, but if this is the case, I don't think I could get enough experience with it on your game to prove to a company that I know enough to get a job.
Different people have different definition of fun. Some play Minecraft just to build CPUs out of redstone.
> On your website, it says that these will actually be real problems for real companies. I would think it's something I would see in production?
The key word isn't "production", it's "you". I, for example, don't really expect to ever use Go or Rust in professional setting. Nor do I expect to use Prolog. Then there are real production systems today that use PDP machines (nuclear reactors) or COBOL (banks). I don't expect to work on them either, but sure as hell would like to play with them at some point.
I get the feeling that Starfighter is about giving geeks some hard-core tech game with a side effect of helping them find a job that is not boring.
Read "real" here in the sense of "plausible".
Sorry about that.
Not that boring, if your clients have interesting problems.
Also you can probably expect things dealing with custom VMs, file formats or networking. Say for instance one of the challenges involves a machine with a broken TCP/IP stack, so you have to write your custom client that can talk to it, how often do you do that in real life? Yet it can translate into useful skills.
Part of the problem I had earlier today was with the words "fun" and "player". I think the words I was looking for were "participant" and "rewarding".
And it allows outliers to get potentially amazing job offers from your partners.
It's exciting to think that with today's easy access to cheap cloud computing and good process isolation you can have instanced "raids" in the same way as an MMO so I can get my very own version of a challenge that can be as realistic as possible.
That was just my first impression.
At my age, I kind of know how I stack up: I've met really awesome people like Andrew Tridgell and... I'm not one of them. So I don't feel like I have anything to prove, and I much prefer to hack on open source ( http://journal.dedasys.com/2014/04/27/an-erlang-postgres-dri... ) or side projects where I make and create something rather than playing a game. So I really don't think I'd use it unless I had to as part of a job process. I don't think that's where you're going with it though; it's more of a way to attract interesting people who might otherwise not get noticed, right?
So I'm "not your target market", which is of course natural and to be expected. I think it's a great way to find people who might otherwise slip through the cracks, and have the time and inclination for puzzles.
I'm the Dwarf Fortress kind of game player, when I ever play games, if that gives you a sense of the direction I'm pulling us.
If Starfighter is only rewarding to people like Andrew Tridgell, we've done something very wrong; count on us to adjust course quickly if that happens.
Couple questions:
1) Is this a Stripe-like CTF that happens over some caffeine-fueled weekend? Or is it more of an ongoing Project Euler-style drop-in-and-solve type process? I don't think I could handle the former, but the latter sounds quite enjoyable and something I might do in my free time just to learn. I did that with PE for a while since it was fun to earn completion points.
2) You mention "Let's Play" style videos. How do you make it so that the solutions aren't given away? Does each player have a customized CTF somehow?
You don't have to rush through Starfighter games. Some people will, of course, just like there is a metagame to be the first guild on the server to complete a new raid. Players are gonna play. But if you want to pick up our CTF a year after release and just casually spend a few hours learning a new skill, you'll be able to do that, and we'll fully support you in it.
How do you make it so that the solutions aren't given away?
The same way you do it with math problems. Mastery of the subject material is the easiest cheat code.
Elaborate? Math questions are the opposite of a good example, aren't they? Even with randomized values it's fairly easy to write generic solutions. People do this even for Project Euler. How much more so for something even cooler sounding?
(I'm very excited to see this project and it sounds like a lot of fun. I'd just be worried that it would end up being gamed, since the drive for cheating is so strong everywhere.)
For example, a general solver for "what is #{x} + #{y}?" is relatively easy. A general solver for "what is the derivative of #{random_equasion_with_diffuculty(3)}?" requires a full computer algebra system. Take one more step into advanced mathematics, and things get more dicy -- automated theorem provers, for example, require human guiding and hints.
This, at best, sidesteps the question. If a solution is a string of characters, that string of characters will end up on the Internet.
If a solution is a series of tasks---try this string of characters, observe what happens, then try this other string of characters---people will still share the strings and instructions on the Internet. All this still holds if the strings are personalized in some way; then the general patterns will be shared.
I spent a few years at university as a teaching assistant for a programming course using an interactive IDE where this kind of thing was required: Run the provided unit tests, ONLY THEN write some code, ONLY THEN run these other tests, ONLY THEN make the appropriate modifications etc. The idea was to make cheating more expensive than actually thinking about the issues, and to some extent this succeeded in the narrow sense that cheating WAS more expensive, but it did not succeed in the more general sense: people STILL cheated because they really didn't want to think things through.
The higher the stakes are, the more not-so-great people will be attracted to the system and will chug through based on someone else's solutions. This will be a fun game for those who see it as a game, but I don't see how the using-this-for-recruiting part will not implode almost instantly.
Finally, the whole thing about keeping even the general subject areas a secret is a bit silly. The work the best tech companies in the world need ranges from medical visualization to databases to systems programming to pentesting (one of you wrote this will not be a pentest game!) to static analysis and verification to distributed systems to scientific computing to drawing pretty pictures in web browsers to... You will only cover small sliver of this. Everybody knows this, why be an ass about it?
After leaving his job he works as a freelance online marketing expert. Then quits that despite implying making a lot of money. Instead wants to create online marketing courses to reach bigger audience, but takes forever to produce any content and is now abandoning that track. Creates AppointmentReminder with some good initial success but reading between the lines that is going to be sold/abandoned as well.
Now moving onto yet another project. Seems you have created several great opportunities for yourself but cannot stick and focus on any one thing?
"Contingent recruiting" has an extremely negative connotation to me. Quite literally, I have to constrain my bias when designing hiring pipelines to not throw recruiter backed resumes away. I understand that you are only taking the payment model, and are trying to undermine the business model, but have you run into any problems associating with the industry?
Or, more interestingly, have you encountered anyone on the consumer side (HR, C-Suite management, etc) that has pointed out a non-obvious advantage to typical contingent recruiting. I for one would love to hear why companies keep going back to that obviously terrible well.
I'm excited to see what comes of this.
How utterly irrational.
It's ideas like this that make me question why I bother being a programmer for anything more than leisure these days... except that I have no idea what else I could do to keep a roof over my family's head this late in my life.
You call yourselves, "engineers," but I've yet to work for any company that treated you like one or even invested in your future. You're offered a salary well above average for most working adults. They lure you with frivolous perks and stock options. They never send you to school for training, offer you a pension to keep you secure in your golden years, and I've never seen any unions you could join to demand these things from your employers. Your career is probably not going to be on the line if you merge a patch that degrades performance or introduces a timing error. But don't think they'll care about you when it comes to the bottom line. Someone else seeks to profit off of your talents and abilities and nothing else. If they can hire someone to automate you away, they will.
And we're all the poorer for it I think.
So how you are working to make us all the richer? How can I join in?
Unfortunately there's no clean, quantifiable solution to human problems like labor relations that I can see. High-tech companies want to filter out potential hires to get the most highly-skilled candidates for the least amount of money they can get away with. They ultimately want to make more money with less capital and labor costs in order to maximize profits for their founders and investors. Which seems rather contrary to other engineering disciplines where, it has been related to me, they took in the hires they came across and made them into the engineers they needed.
(I can confidently state that you do believe he would have performed similarly on a different test - if he didn't, this whole idea would be fundamentally invalid.)
Now the important question is to determine, for any particular subtest, how closely a person's score correlates with the hidden factor H. This is, for example, how IQ tests are created. You can take a look at the literature surrounding them. Key starting points would be principal component analysis, clustering, hidden markov models and the like.
There is no simple recipe - every data analysis problem is different in it's own way. But there are general themes. I'm happy to discuss in more detail, feel free to write to me if you are interested.
I could tell if GP was being sarcastic so I deleted my original reply.
The softer side of the equation is that the approach of filtering candidates for businesses allows them to minimize (to some market minimum) the cost for highly skilled individuals. Projects such as this just triggered a bone I'm presently picking over.
Old-guard corporations that hire the legally-liable kind of engineer seemed to have taken a much different approach if the stories I've been told by a retired mechanical engineer who worked for Chrysler are true. A completely irrational approach. They hired people based on their potential and made them into the engineers they needed.
The copy on the announcement sounds like this game will quantify everything about the performance of a participant in the game in order to sell them to a curated list of potential employers. What about this system incentives employers to invest in the career development of these people and strengthens our collective bargaining power as the people who build this stuff?
To be more constructive I might suggest turning down the hyperbole and use fewer adjectives. Keep the pitch to employers for employers. Don't be patronizing to inexperienced developers: inspecting the assembly output of a compiler is not elite and not difficult to explain to someone given the right context and framing. I understand your game is about competition but the "winner/failure" schism it can create is a big turn off for a lot of otherwise intelligent, creative, and capable people. It doesn't have to be about being, "the best," in order to be fun and rewarding.
>Effects of practice on the Wechsler Adult Intelligence Scale-IV across 3- and 6-month intervals. Estevis E1, Basso MR, Combs D.
A total of 54 participants (age M = 20.9; education M = 14.9; initial Full Scale IQ M = 111.6) were administered the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) at baseline and again either 3 or 6 months later. Scores on the Full Scale IQ, Verbal Comprehension, Working Memory, Perceptual Reasoning, Processing Speed, and General Ability Indices improved approximately 7, 5, 4, 5, 9, and 6 points, respectively, and increases were similar regardless of whether the re-examination occurred over 3- or 6-month intervals.
> Practice Effects for the Stanford–Binet Intelligence Scales The Stanford-Binet Intelligence Scales—Fifth Edition (SB5) is a widely used assessment tool for measuring intelligence (Roid, 2003). According to Roid, a key advantage of this intelligence test’s most recent revision is that it includes improved lowend items for better measurement of young children or adults having mental retardation. Sbordone, Saul, and Purisch (2007) report that the range of the SB5 was expanded to allow the assessment of very low and very high levels of cognitive ability. Roid and Barram (2004) indicate that the practice effects on the SB5 were smaller than expected. For example, the nonverbal IQ of the SB5 showed shifts of only 2 to 5 points as compared to the 4 to 13 points on the Performance IQ of the Wechsler scales (i.e., the WAIS-III and WISC-III). Roid and Barram add that the lower shift, and thus practice effect, is even more notable given that the retest period for the SB5 was 5 to 8 days versus 23 to 35 days on average for the Wechsler scales.
I think it is amazing what you guys did here: you got more points on a give me your e-mail, we are building something post, than Stripe got when announcing their company. Really speaks volume on Patio's writing.
Will there also be marketing challenges? Like: Every day patio11 clicks on one email. Your task is to write an engaging headline. Or: Here is some Google Analytics code. Make it show 50.000 visits any way you can. Very hard to cheat. (But so is getting Google security to file a bug report).
I really enjoyed the comments. Especially the ipod-hasnt-got-wifi criticism. You guys must know that what you are doing is terrible and you should feel bad.
Who knew decoding morse in some wave extracted from a corrupted image file in Cicada 3301 could also lead to a job. Will there be an ARG element to the CTF's? Like you pose as an agent from the NSA or something?
If you find yourself wanting something more advanced than that, random forests would be a good match here, as would clustering the candidates using DBNs or simply k-means and then using base rates.
Also, if you haven't looked at https://www.kaggle.com/competitions you really should -- they're similar to what you're doing, only for machine learning.
The science of hiring practices is settled: work-sample tests are the most effective way to assess skill in potential hires.
This is one of the most obvious, yet under-applied truisms in hiring.
If they can apply Starfighter beyond Games, or enable non-Game skills to shine while working on Games, it will be industry changing.
Ok, so lets replace whiteboarding ancient algorithms and useless trick questions with a capture the flag game!! That'll make the industry seem less "unserious."
This is so unfathomably ridiculous. I can't believe you're marketing this as an interview replacement. If I get asked what my "Starfighter score" is before a job interview, I will be running as fast as I can in the other direction before they finish their sentence.
There will be people who become great at this game, and if (god help us) this gains momentum, they will get jobs.
I can only hope the rest who are developing applications and actually writing programs will come out on top, and employers will recognize that some people want to build things and not just improve their ranking in a game.
Without opening up a pointless vein of second-guessing about Patrick's writing: I think the timing of this post has led people to believe that this right here is my answer to The Hiring Post.
No. The Hiring Post itself contains what I believe the answers to The Hiring Post are.
There are two halves of the recruiting problem: OUTREACH and QUALIFICATION.
Companies need to be smarter about qualification. We have a lot of thoughts on how they can be smarter and I am in full-on insane street preacher mode about those thoughts, which are not directly remunerative to our business.
Starfighter is about the OUTREACH side of the recruiting problem. Our goal is to be a credible pipeline of candidates who are not effectively identified by the resume/interview/github process. We are looking for buried talent, and to have fun doing it along the way.
I know HN all too well, and I think the instinct people will have reading this comment is to try to reconcile it with exactly the words Patrick chose to use in his blog post and then object to the perceived discrepancies. Please, no: that's a waste of all our time. Patrick speaks for Erin & I. I speak for Erin & Patrick.
But yes, this post does seem to directly correlate, in timing and in content, to Patrick's Hiring Post. I'm surprised to see you say this is not your response to that, especially considering that it addresses many of the questions we should ask ourselves as outlined in The Hiring Post (namely consistency, data, and the idea of one being a “natural”).
Also the post itself is linked in the statement:
>We’re not here to fix the technical interview: we’re here to destroy it, and create something new and better in its place.
I disagree with the fact that outreach is a problem in recruiting. There is an entire industry of companies and teams created for the very purpose of reaching out to software developers for their skills. How many of us reading this thread get multiple recruiter emails a week?
Are the recruiters effective? Maybe or maybe not, but I (who am apparently one of seemingly only few) just don't feel that a competitive game needs to add itself as another middleman in the realm of software recruiting with the intent of "destroying the technical interview."
That being said, I've got a couple questions
Will all the CTFs be security focused?
One of the most common problems coursera ran into was people copying the assignments of other people. How do you intend to prevent copying for the CTFs? I see that you mentioned you're not going to DMCA tutorials or Let's Plays and instead track their every iteration on the code. I guess you can get rid of outright copying that way but I'd love to hear your other strategies.
Other than that this seems like a really interesting way to do recruiting. I can't wait to see the actual CTFs.
Also you've got a tiny typo in the WHY IS STARFIGHTER THE RIGHT TEAM FOR THIS? section: "I have a folder in Gmail saving messages from geeks who used by career advice or salary negotiation tips to their advantage." I think the by should be my
Besides cryptography, what are some other programming problem domains you'd enjoy playing with? We're particularly interested in problem domains that are hard for ordinary programmers to pick up on a whim. We want to make it possible to seriously engage with interesting problems while in your couch with the Daily Show running in the background.
For non-direct-games things fancier AI like deep learning, audio synthesis, making compilers/parsers/transpilers, math stuff like project euler, puzzle/board game solvers/bots (https://gist.github.com/christopherhesse/51e9baf0e3d440d8aff...) come to mind.
Some of the stuff in the 3rd stripe CTF was pretty interesting like making a search engine or dealing with distributed system consensus problems (raft).
Maybe using the coolest new framework is fun for some people, like a challenge could involve angular.js so you could get some experience using it without having to go full-on side project or put it in at work without having any real idea if you should.
Implementing things that you are familiar with but that you don't understand the internals of has been fun for me in the past (for instance like https://www.mikeash.com/pyblog/friday-qa-2015-02-20-lets-bui...). Maybe you always wondered how this thing works, and now you can figure it out and solve this challenge at the same time.
(To elaborate a little, Richard Bartle noticed four types of players of MUD. Achievers are just one.)
I didn't do so well on the microcorruption game. I'm curious to see what else they can come up with, hopefully both less specialized, more generic, but also more advanced...
Still... if it works, and is actually better than most other attempts at code-gaming this may be fun. Profitable? They will have to see.
Also, what are the plans outside of security related (if any)? Can you do a CTF style process for non-security?
Stripe is going to hire a couple of hundred developers over the next decade (assuming things go well). It sounds to me like Starfighter's goals don't translate to a couple of hundred X 20-40k. They seem a touch more ambitious.
Will this be structured so I can use it as a proxy for a programming mentor? I think it would be awesome as a replacement for MOOCs, which I enjoy but find inefficient for the time invested. Books are efficient but not as much fun.
Most / much of the success of a valuable developer is not in their specific technical talent, but in areas like expounding a vision, documenting, running a team, navigating politics and fund raising. (Bit vague here I admit, personal experience of success is problem for me)
Patio11 made a name for himself saying "tech is a necessary but not sufficient condition for business success - now learn about basic marketing"
Something similar seems to be applied here - how can a CTF program ever handle politics or investor problems or documentation issues.
Getting good technical capable people is important - but not sufficient.
Will be interested in seeing the CTF program however ... Good luck.
It sounds like you're mistaking "developer" for "tech startup founder". Someone looking for a developer needs at most 2 of those skills (documenting for sure, navigating politics in an organization full of bureaucracy).
Developers are expected to build resumes at work via daily routines, then at home via github, hackerrank, side projects. System administrators are expected to do daily system administration routine, know OSes, tools, cloud/iaas/paas providers inside out and code side projects and build a coding portfolio.
With years of experience, one expects to be respected and not lose time on games, tests, unwanted side projects. Interview process is built towards young gals/guys.
GitHub, BitBucket, HackerRank, TrueAbility, Certifications, Education, Jobs. It's becoming of a norm that if someone puts more personal time to show off, they are better engineers. Also that's another wrong position in IT. You can not have personal life. I do not want to be a marketeer, I'm an engineer.
If you ever ask any of us around here, perhaps most will tell you a scary story about the competition and the time constraints the competition brings on people.
Eventually with every racing/gaming/ranking tool out there it's becoming to be harder to get a good position in IT without investing in more and more personal time.
1) Of course there will always be other ways to get technical work. You needn't feel compelled to use their product unless you think the time input is worth the reward.
2) If they haven't made the process fun, that is something you want to spend your valuable free time on, they have failed (at least in acquiring you as a product).
The next company expecting me do some obscure online automated coding test will need to pair me with an engineer inside.
And, by the way, if you don't have a great resume and you're looking to work in a field directly related to the CTF we're running: making you more expensive is exactly what we're going to do. :)
By the way nowadays shiny resumes are not considered relevant. It's the bottom line of screening. I have a great resume in my country and I am expensive here. But it would not matter for any SF startup.
The Starfighter team is trying to change the ACQUISITION side of the pipeline. They are stepping in to the space where traditional recruiters feed applicants into the pipeline. In marketing terms they are generating leads. Their value proposition to the employer is that the leads they generate will be of higher caliber than the traditional contingent recruiters. Their value proposition to you is that Starfighter is a) fun and/or b) at least better than dealing with a traditional recruiter. Your Starfighter experience replaces your resume/cover letter that gets you into the pipeline. Not your ability to navigate the pipeline once you get into it.
If your resume/traditional recruiters/responding to web ads/word of mouth is more valuable than Starfighter CTFs fun - hassle + employer payoff than you are quite simply not a good fit for the Starfigher Recruitment agency. Just like you wouldn't be a good fit for a DBA centric recruitment agency if you weren't a DBA.
If I'm disappointed about anything in the announcement, its that it is obvious to me anyway, that they are NOT trying to replace the current qualification steps in a hiring pipeline. Because to me, getting rid of that time wasting/unproductive process is the big value add, not shoveling more devs into the top of the pipeline (management at my company may feel differently).
We need participants to make the game fun (for all of us). We can't do things that make participants wary of us.
This is fuzzy because I didn't let Patrick write about what the game was (that's entirely we're-about-to-ship-itis). It'll be less fuzzy soon.
I may seem to be against StarFighter, but no. I'm not. Actually I'm on the mailing list and waiting for the service to be offered. But I am not fully sure that I will be a good contender. I have not been a good gamer for a long long time.
I believe you fully understand my concerns. It's about the interviewing process, not the tools. Every tool has it's uses and quirks and every institution may use the tools in different ways, intended or unintended.
This "best of the best" stuff was meant to be empowering, but in a lot of ways it didn't come off that way.
I described our goals a little clearer here:
http://www.reddit.com/r/programming/comments/2yhcqn/were_not...
> We come here not to serve technology recruiters, but instead to replace them with a small shell script.
That would be uhm... i don't even know the right adjective.
The more interesting question is if they will ever end up teaching at YC.
If you're talking about the "winners" having personality issues, then hopefully that will be caught in the interview process.
Perhaps I'm completely missing your point though.
I guess there's value is all in assembling a pre-qualified pool for employers to recruit from, though.
But in that way this is kind of like the 2015 version of CNE. Except it's free, which is good. And hopefully less miserable.
I grew up near a small town surrounded by family farms. I've really enjoyed manipulating code and finding bugs in games and other software since middle school, and I suspected I was the only one when I was younger. There were very few "nerds" at our school who focused on computers, and even those that did almost never discussed the discovery or exploitation of bugs. Although I stumbled upon a few communities online that shared my interests, I hadn't found one that just clicked. I happened upon HN while learning more about startups, and it was just right for me. I'm grateful I found HN, but I wonder why it took me so long to find the community I was looking for.
I didn't even know about CTF games until tptacek's recent hiring post, and it was another "How could I have been oblivious to this awesome thing for so long?" moment. There are many people who are a great fit for your clients, and they won't know about Starfighter unless you cast a wide or deep enough net. How do you plan to address this challenge?
Good luck, and thank you for building this!
Having competed in the CCDC for 6 years in college, I found it pretty insane when I saw that employers don't use real hardware during interviews.
Looks like I picked the wrong side project now this team is working on it. However, interviews are broken enough that there is plenty of room in the space =]. Can't wait to see what you launch.
Competition is a signal that you picked the right field. Just about everybody gets this backward.
Amen.
BTW, I completed microcorruption, foobar and two stripe CTFs. It was lots of fun and I am really looking forward to Starfighter!
This theme (test your coders, don't interview them) is spreading about, I keep getting spammed by them as a way to "only hire top talent!" its an interesting proposition and I can't wait to see a bit of history behind it. At the very least it will weed out people who think they are programmers when they are not.
I am curious to see how this will work for those like yourself that make CRUD apps. I didn't go to school for programming, but self-taught. I'm afraid it'd be over my head?
The best line: We haven’t done any consulting in a while. We will continue doing no consulting, to the best of our inability.
Having done other CTFs and puzzles like Matsano / Stripe's, I have found the research part is quite enjoyable. I have little to no knowledge of crypto but researching and solving some of the Matsano challenges was extremely rewarding.
With that said I found the information is pretty thinly spread and it is often hard to find relevant information to the problem domain, particularly with regards to crypto, so I hope they include good starting points for problem research.
http://en.wikipedia.org/wiki/Cicada_3301
I love what you're doing but I agree with other commenters that presenting it as a game to programmers but a recruiting channel to companies might not align well. What do you do if a lot of your top players aren't interested in getting a job? There are probably exceptions, but do you think people who are good enough to beat your CTF have that much trouble getting a job?
Even if you are recruited on topcoder you sometimes still need to submit your resume to some online resume eater "to get you in the system". And then you go on the same time wasting interviews as everyone else.
Personally, I don't play games of any sort but I like to write software and solve puzzles so I find topcoder SRMs more appealing than a programming game, but I'll definitely check it out.
It works great for them, but I believe you attract a certain kind of developers with these contests; you attract people who enjoy puzzles, and people who enjoy measuring their abilities against others. It could be that those are the traits that make good employees; but I myself would never take part in such a competition.
Then again, I don't actually want to work for anyone else at all.
My prediction is that Starfighter will become something akin to HackerRank where companies host their own CTFs to recruit people. While it sounds fine, making CTF problems is significantly harder than making algo problems. I'm sure the team is right for this, so it might just work.
I hope they do release some work in the open (such as their sandbox environment).
Is it purely security stuff?
Be fearlessly willing to pick up programming in new languages.
No security knowledge is required.
Security is a backdrop for what we're doing, but not the only backdrop; it is one of two problem domains we're starting out with.
We use security mostly as a venue for systems programming.
What's the second problem domain you're starting out with?
I hope it's to do with graphics/parallelized matrix math sort of stuff.
Do you plan for there to be an option for employers that aren't asking you to refer candidates to still see some view of the performance of a player? Either by paying some smaller fee or by allowing players to have public profiles with some sort of scores?
I fear that starfighter will overlook goood candidates simply because they are not good at gaming, which makes the second approach by ortask more interesting and maybe balanced.
- Is it going to hurt?
There will be a subset of participants for whom it's worth their time and ours to have a conversation. Our job will be to identify that subset and start that conversation.
You can reasonably assume "What skills does the market want to hire for?" is a good proxy for what we'll be assessing for. I love security, but that's 0.01% of the software market. We have... grander ambitions.
Hint-hint-nudge-nudge
/endOCDpost
I want to point out trueability.com that does same thing with Linux. I find it a very good and fair source for judging Linux admin candidates.
What an absolute waste of time. You can do all of those things as hobby projects which are yours and actually do something useful in the world besides exist as an achievement to be unlocked.
Show your future employer your github, not your starfighter score.
Spartacus, open your mouth, I want to see your teeth.
What is going to happen with Appointment Reminder now?
That's a non-answer though. Let's see how the answer about the fate of Appointment Reminder would evolve over time.
In any case, that's an extra proof that being solo-founder while possible - is a disadvantage.
Edit: rather, (9 characters long) @gmail.com
I'm wondering, what is your business model? Are the users doing free bug fixes and work for the employers while they are proving their skills?
Our business model is explained in the post, under the heading How Will Starfighter Make Money. Briefly: we'll find the best engineers in the world, then broker their introduction to companies who want to hire them. Companies will pay us a substantial amount of money for this service.