1,615 karma · joined February 20, 2007
Bootstrapped family biz: elbobooks.com
Spring Science founder / CEO (now part of Genentech)
Khan Academy founding engineer / VP eng
Fog Creek early engineer / VP eng
This reduces inbox noise because it increases the percentage of emails in my inbox that people expect me to read completely and/or do something about. No more "might as well just CC Ben for the heck of it." Those goes elsewhere, for me to quickly browse the subject line if and when I choose.
Remote internships have been done in the past too, but we strongly encourage being in Mountain View both for our sake and yours. The summers are a blast with all interns in-office together and enjoying various team activities.
I suggest you apply!
I'd like to also emphasize how much we lean on our community. Khan Academy's goal is not to be perfect, but to iterate towards perfection — it'd be impossible to do that across such a broad array of content without building up a strong community.
Beneath our videos the community creates Q&A to help address (we hope) misconceptions and follow-ups about each topic (fun example: https://www.khanacademy.org/science/cosmology-and-astronomy/...).
When the original WaPo article was posted, it mentioned a user's question ("So is .02009 greater than .0207?") as evidence of KA failure. But to me, the fact that there were already helpful, patient responses to this question from our community...well, that's a success. We know KA can't magically teach everybody everything, and building a community of support is one of our biggest goals.
Of course, we also use the gems in this community to help us iterate on content.
Sal goes back and reworks videos often, but this is difficult to scale so we recently added the ability for us to make "official clarifications" from our community's content. These are video annotations that clarify important or confusing points for videos we haven't had time to redo yet (see an example at 12:47 in https://www.khanacademy.org/science/chemistry/acids-and-base...).
Overall, Sal has imbued the entire company, starting at the top, with a belief that shipping beats perfection.* We're building tools to move as fast as possible any time we can improve our content — but that won't stop the occasional valid criticism from surfacing. We'll make use of it.
On a transparent/personal note, I take issue w/ many of the assumptions and tone behind this letter...but my shortcomings here won't stop our team from plucking some value out of it.
* https://sites.google.com/a/khanacademy.org/forge/for-develop...
Was pointed to http://michelf.ca/blog/2005/mac-sub-menus/ by @gecko this morning, which brings up the opposite (and legit) problem: very, very long submenus.
Mega dropdowns w/ equal submenu height are a nice use for this, though.
We're mostly looking for longer-term non-volunteer commitments when dealing with things like our data/analytics, but our open source repos are very volunteer-friendly: https://github.com/khan/khan-exercises
Transparency about all past internships, their expectation and accomplishments: http://bjk5.com/post/36067210182/khan-academy-internship-sum... http://bjk5.com/post/8826207372/khan-academy-internship-summ... http://bjk5.com/post/15500332273/khan-academy-internship-fal...
Transparency from previous interns about their work, which I believe speaks for itself: http://david-hu.com/2011/11/02/how-khan-academy-is-using-mac... http://dylanv.org/2012/09/14/the-khan-academy-internship-exp... http://jamie-wong.com/2012/08/22/what-i-did-at-khan-academy/
If any piece of evidence from the above points to an organization that values quotas over excellence in interns and the work expected of them, well please raise the alarm.
We need to get past our default response being a knee jerk assumption of discrimination.
We focus incredibly hard on our internship program.
Just made this page the other day to share more: http://www.khanacademy.org/careers/interns
When we launched Precorder, we got little (and so-so) press by describing a newly launched app w/ X Y and Z features.
When we started telling writers that we saw the camera technology used to capture great white sharks jumping out of the ocean and ported it to iPhones, we were covered by Kottke (http://kottke.org/11/01/precorder) and other influential photography bloggers who immediately triggered a long wave of follow-up press.
Not only does it have obvious and enormous personal benefits for Jamie, but it's galvanizing for the team to look back and read through just how much has been accomplished in a few months (I'm proudly on Jamie's team, am speaking firsthand).
The only thing I'm suggesting is that looking at a graph that is wildly fluctuating up and down can be more helpful and easily accessible than asking someone who isn't an expert in stats (I raise my hand) to look at a number of participants and immediately understand its effect on this specific experiment's variability.
No, we are tracking the number of people that the target event happened to -- but we've also tracked the number of times it happens, and as mentioned in the article, we are aware that these specific metrics are extremely outlier-prone and are looking into ways of improving this. All advice welcome, your tips are much appreciated.
A historical graph can show you that 12 hours ago, there was a stat sig in favor of alternative B (we've seen this plenty of times).
Some metrics take a while to stabilize, and the more metrics you have, the more likely you are to run into these particular situations (as you mention above). A graph helps you understand recent variability in the metric.
The point about using historical graphs instead of single significance numbers reduces, but does not solve, the likelihood of making a mistake due to one of these chance results.
I would like to correct a persistent misconception or two.
Persistent misconception: "...we suggest that Khan Academy desperately needs voices of teaching experience. Khan could tap into any number of existing networks..."
Truth: We have four ex-teachers as full-time employees. We have two high school math teachers as consultants. One Harvard Doctoral candidate in Education and one post-doc in neuroscience at Stanford are in residence. One UPenn Professor is also likely to begin a sabbatical with us. We have a 3 person team dedicated to working with and getting feedback from our 50 pilot classrooms and the 15,000 teachers actively using KA in classrooms.
Persistent misconception: "...it certainly requires more than just “two minutes of research on Google,” which is how Khan describes his own pre-lesson routine."
Truth: Go read Sal's AMA response (includes the sentence "When I did organic chemistry, I spent 2 weeks immersing myself in the subject before making the first video") before taking one of these "two minute" snipped quotes at face value: http://www.reddit.com/r/IAmA/comments/ntsco/i_am_salman_khan.... I've seen Sal's face light up when he gets an unwieldy new shipment of textbooks to start studying in preparation for his videos. Does he dive right into some videos? Absolutely. Is claiming that his "pre-lesson routine" can always be dismissed as two minutes of Googling disingenuous and patently false? Absolutely.
However, what I'm specifically referring to in this blog post is the ability to keep relying on App Engine's datastore for the everyday work involved in serving our application (forget the mapreduce stuff) while gaining more flexibility to run non-App Engine pieces of software on the virtual servers without suffering the App Engine-to-EC2 latency pain.
A trivial example would be Lucene (right now we have to run it on EC2 and communicate back'n'forth). Another example would be our own memcached servers that we control the size of.
We don't have to worry about it, because performance scales with the size of each query's result set, not the size of our data.
On any other system, you'd have to worry about it. That's a (for now) unique opportunity Google has w/ compute engine, IMHO.
ben@khanacademy.org
I'm willing to take some of that blame for being shallow, but not all of it. I know I wasn't the last teen diabetic w/ similar thoughts.
I seriously doubt if I would've been up to doing either while managing injections instead of button presses.