Comment by quot66555 | Hacker News Reader

Comment by quot66555 | Hacker News Reader

I'm very interested in Starfighter and am looking forward to the release. However, there's something from your original announcement that still worries me. In this post you said:

The designed operation of our trading levels is “Play them, lose horribly, write a program, reset the level, run the program, have it crash with a bug, fix and re-run, lose modestly, adjust your approach, re-start the level, run the program, win.” (Folks have asked whether we’ll hold this against you. No, no, of course not. We log everything under the sun, but losing/restarting levels is planned. That capability is an advantage of doing this in a rich simulation as opposed to real life — you can go Knight Capital as many times as you want and no one loses their job!)

This feels good. But earlier, this was said [1]:

We will track player behaviors and skill at incredible levels of granular detail, instrumenting them like they were built by the Orwellian MiniPeace... We can tell you exactly what happened when your candidates tried to implement a REST API. We can compare their performance against hundreds of other talented engineers (including your current employees) on the same task.

Can you please elaborate on what exactly is going to be tracked? Is any of this configurable by the player? Personally, I'm going to have a hard time enjoying and learning if I feel like every mistake I make, or every extra piece of time something takes is going to be judged and count against me later on.

[1] http://www.kalzumeus.com/2015/03/09/announcing-starfighter/

Remember their incentives. They want you to perform well (or more precisely they want performing well in their game to correlate with you being a good employee).

It is not in their best interests to count against you, quite the contrary they want to sell you to their clients.

What is a better way to sell a candidate to clients than to say she is in the top 1%? They have every reason to count out as many people out as possible. Nobody looking at this kind of service wants to hire candidates who are not the best.

Because the population of devs who don't play at all is gigantic. So they can make the argument that "this person played and got through 65% which has x correlation with being a good employee" and that still puts that person in the elite status.

Just knowing what stockfighter is puts them into some extremely small percentage of devs, which in the current market is a drop in the bucket for demand.

That's not how our business model works. We're not going to make extra money based on where people rank.

I understand that, but if you want to sell any kind of screen you have to represent that the screen discriminates "better" candidates from "worse" ones. The people who are willing to pay for this screening want the top 1% or 10% or whatever (even if the way it goes down is that they request data dumps and then make up the scoring function themselves). That is how the whole industry works, it's not your fault and you almost certainly can't do anything about it.

Edit: even if you present yourself as a recruiter, it doesn't make any real difference. The value you are adding and charging for is that you are handling part of the recruitment process so that your clients don't have to. Nobody wants a recruiter who doesn't hand them the best candidates.

We're not selling a screen. We're not a job interview. We're just smarter recruiters.

Pardon me. You're selling recruitment services, which happen to involve preliminary screening through Stockfighter. I am venturing the very risky speculation that these recruitment services would have no value to your clients if they were not somehow selective. If all you want is to sift through a million resumes, that has been cheaply available for years.

Sorry, it's just not that simple.

I see that you aren't finding this productive to continue. That's okay. Thanks for your responses anyway.

Answered up here: https://news.ycombinator.com/item?id=10095199

I think the key point is that they don't want the top 10% or 1% or .1%, they want as many people as possible above some particular skill level. The client is just as happy if that happens to be 50% of players or whatever.

They don't want "the best" candidates, they just want competent ones.

Yes, that's closer to the idea.

In particular: clients cannot pay us for a list of people who have exceeded some threshold in SF.

Clients would be irrational to pay for a recruiting service which removed 0 bits of uncertainty as to the quality of presented candidates (by the client's definition of quality). It's safe to generalize that recruiting services which survive reduce uncertainty or at least give a strong impression that they do, i.e., they discriminate among candidates in some way (I don't mean discrimination in the legal or pejorative sense).

This isn't an accusation of wrongdoing, just a simple observation about reality. The incentive for a recruiter using a game like this really isn't for every candidate (err, I mean, player) to be seen as equally recommendable. Even if you were so suicidally idealistic that you saw it that way, clients wouldn't. But you aren't stupid, you know that.

quote66555 is right to think that there is an incentive for extensively recorded details to be used as a signal of his individual "quality" as a candidate. And if he is not in the top n% (where n < 100) then at best he will not be recommended. So it is possible that his little delays or minor fuckups will count against him, assuming he cares about being recommended. It's a reasonable thing for him to think. He isn't scare-mongering.

Look: this is easy.

We promise: your data is private. Nobody sees anything you do unless you want them to. It doesn't matter what employer incentives are; revealing a log of how you've engaged with this CTF would involve us breaking a promise to you.

Meanwhile: what we're doing as a business is opt-in. If you don't think we're trustworthy, there is absolutely no reason to work with us to help find jobs. Just goof around with the game and ignore the fact that we do other stuff. We are no company's job interview.

>>>Most of the best performers on this CTF aren't going to get placed by us.<<<

It's frustrating talking about this, because we've had our hands absolutely full just getting a CTF that's better than Microcorruption done. We haven't had a spare moment to think about how to make money from it.

Catch me sometime when I'm not debugging a weird interaction between my emulated SPI bus and my emulated UART. :)

I feel like this is largely an issue about expectations and disclosure more than it is about logging per se, right? We have a privacy settings page, which I'll show you in a minute, but here's the public view of myself on the staging server. I have it configured to Maximally Public (and will have that on in prod).

https://www.evernote.com/l/Aacj0YC7tCRIdbau2AgTrBORPc6kgX442...

The redactions are just to avoid spoiling levels. The "replay" link is a UI templating bug. (It isn't for showing a replay, it is a convenience method for replaying the level, and it's now supposed to show unless you're logged in and able to play the level yourself. Fixing this in the next 5 minutes.)

Now, if I (as a player) log into and view my own profile, I get a better sense of some stuff the system knows about me:

https://www.evernote.com/l/Aae3c5I6TdZA7ZtONMeM_z-ammvZA_3FD...

That is obviously not all the information the system has about me. If a sparrow yawned in a world simulation that I took part in, that fact is dumped onto an S3 bucket, somewhere. To be maximally explicit: every order I sent in. Every execution I got back. Snapshots of the GM's memory state as I passed through the levels. etc. We grab all that. That isn't an exhaustive list. Assume if it is amenable to being discovered by a computer that it will be discovered and persisted until the end of time.

This is not exposed publicly.

Here's the preferences UI. Verbose but it gets the point across, right. Defaults are "Anonymous" and "No Public Profile", respectively.

https://www.evernote.com/l/AadESN5EzgdCzpFAQnR0rIuxA5Ghb7pKX...

Now to a question which may be implied: "OK, I now understand that my info is locked up pretty tightly within Starfighter. But you want to tell people that eventually, right? That's how you justify your service to clients." At the point I'm on the phone with the CTO of $CLIENT I am a contigency recruiter. That means I only get paid if that CTO decides to interview you and ultimately hire you. You can bet that I will be doing my best Enterprise Sales work and showcasing your performance in its most positive truthful light. If I did not feel like that would be possible, that phone call never happens. If the CTO had previously mentioned to be "Be on the lookout, specifically, for people who are good at analyzing mountains of data. We need them for our fraud team.", and I had relevant signal on that question from level X, you can be reasonably assured that if you did really well on it I'd open the call with "The reason I'm recommending Foobar to you is because you told me you wanted folks who are really good at analyzing data. Let me walk you through the solution Foobar came up with -- it's a hoot."

(Worth re-iterating explicitly -- that that point, we've talked on the phone and you've explicitly given me the go-ahead on placing you.)

To the question: "OK, so I get that people outside the system don't get arbitrary read on my history, and I understand that you're essentially a firewall which is incentivized to only allow the egress of the best possible true packets: what about you? Aren't you going to be influenced by, you know, taking a look directly at my stream of HTTP requests and seeing that I fumble fingered a URL the first time I played the level?" To which I say: I don't care and don't have time for that nonsense -- I'd much rather concentrate on things which actually matter to an engineer.

Thanks for the thorough response. Yeah, pretty much. It's good that you all have thought about this stuff. There's always a PVP aspect if you're ever going to put yourself public or on the leaderboards, but that's just part of the game. I can always start out anonymous and go public if I turn out to be any good...

account1: practice.

account2: 'for-realzies'.

Succeeding by whatever metric because of this? Let the true games begin!

That "account2" isn't going to do you any good that figuring things in "account1" didn't already do for you, but knock yourself out. ;)

As a piece of unsolicited UX feedback, if you chose a different word for one of "cleared"/"cleaned" it would make those two radically different actions stand out from one another in the list. I do see that "cleared" is accompanied by a score, so it's not the end of the world, but I did warn you it was unsolicited feedback. :-)

Also, do you know why evernote chooses to send "Content-Disposition: attachment; filename=skitch.png" forcing my browser to download instead of display the png inline?

Reply on news.ycombinator.com