mshron

I've maintained for awhile now that the distinction isn't between "big" and "small" data, but between coarse and fine data. Now that everything is done through the web, previously common data sources (surveys, sales summaries, etc) are being supplanted by microdata (web logs, click logs, etc). It does take a different skill set to analyze noisy, machine-generated data than to analyze clean, survey-like data; it's a skill set that is more biased towards computational knowledge than classical experimental design, hence the shift in emphasis.

Since the new administration decided to start gaming the rankings. Admissions rate now stands at 8.8%, down from mid-forties when I was admitted only eight years ago. Part of that was from switching to the Common App, part of it was riding the coattails of Obama.

It is a great school. But until recently, rankings weren't really something that the administration paid much attention to.

> The $1 billion in PAC money in 2012 seems to have had very little impact on the election, other than to pump up the people who already agreed with the messages in the ads.

There are at least three ways that money can have an effect that come to mind. The first is determining who wins; this is the kind you're talking about. The second is determining who can run; the winners of the primaries in both parties have as far as I can remember also been the top fundraisers going in to the primaries. Is it sticky which is the cause and which is the effect? Of course. But I have no doubt it's an amplification.

The third way is, I think, the most perverse. Only a very narrow range of political perspectives are capable of making it to the mainstream, since they first have to win the money election in order to make it anywhere.

Frankly we have no idea what the political landscape would look like with citizen funded elections. Maybe it would look exactly the same, in which case your hypothesis that the government we have is already representative would be proven correct. Right now we have no idea, and that's why I support citizen funded elections.

1) I think Lessig makes a fair case that we should broaden our notion of corruption. Congress folk spend 30-70% of their time raising money. Roughly half of congress goes in to lobbying after their terms end, with an average pay hike of about 14x. You think that doesn't "corrupt" their loyalties? Okay, it's not Dhaka. But I don't live in Bangladesh, I live in America, and I have different expectations for one than the other. I don't buy your claim that calling grossly perverse incentives corruption somehow lumps Washington in with Moscow.

2) Most of what the government does is not big issues, it's small ones that actually have the biggest effect on our lives. How should airlines, financial firms, telecommunications, etc., be regulated? Most people have very vague ideas about what they want, and are strongly swayed by poll wording. Organizations being regulated, however, have very strong ideas about what should and shouldn't happen. In theory, they should be one voice of many; in practice, we get regulatory capture.

I think Coursera has struck a very effective balance between synchronous and asynchronous learning. Once you sign up for a class, you're able to go back and watch videos, get assignments, even submit assignments for automated grading.

For those of us who find the synchronous style to be a sufficient motivator, it's a great benefit. For people who fall behind but want to cover the material, they're welcome to do so. The grades don't actually matter, remember? Coursera is already "gamified." It's just that the game is done so well that most of the time we're not aware of it.

Not the OP but:

* Problem definition

* Infrastructure

* Data transformation

* Exploratory analysis (arguably part of model work)

* Results presentation

Then again, this is an ongoing disagreement I have with the Kaggle folks over what constitutes "data science," where I'm pretty confident that "applied machine learning" is a better explanation of what their contests are about.

Citation needed? If true, that's a pretty crazy story.

I'm surprised nobody has mentioned Meteor yet. As far as I understand, developing for Meteor means having a single codebase that erases the distinction between server and client.

Your web pages become live views of the underlying controllers. Modifying state in one client automatically modifies it in another. And I'm pretty sure it supports hot-swapping.

http://meteor.com/

Everyblock was a great way to find out about what was going on in my neighborhood. I wish they had given a little more warning; I would have gone back and read some things before they closed their doors. Then again, hard to complain about such an excellent free service.

They lost their founder only a few months ago. I have no inside information, but I can't imagine that this is unrelated.

I wish the whole team the best of luck! They did a great job with Everyblock. I hope they do something else amazing soon.

Assuming that you're comfortable with 2d histograms (hexagonal binning is the standard) I completely agree. Otherwise scatterplots are pretty crucial.

In what way were you hindered? I deleted mine years ago and the only downside is that I occasionally can't see a photo album someone sends me. It's not like photos only exist on Facebook.

I can't speak for other people, but the effort it took to maintain a Facebook presence (deidentify unflattering photos, take a few minutes every day to get caught up on people's minutiae) was far greater than the benefit it brought me.

People who want to talk to me know my email address and phone number.

(edit: clarification, first paragraph)

I'd like to suggest another reason for large military expenditures that have nothing to do with corruption. The US has the only two-ocean navy, and a very extensive one at that. A lot of the burden of protecting shipping lanes for members of the "global system" is covered by the States. That's really, really expensive.

Of course, that means that countries that don't want to play by US rules (North Korea, Iran, Cuba) get shafted, but it's still a better explanation of the crazy military costs than either fear or corruption.

To those of you that have forgotten: ↑↑↓↓←➝←➝BA

... where there are serious externalities. Isn't this where government regulation is most suited?

I love the idea, but I don't think the page does a good job of selling itself on design merits. Your eyes immediately go to the tertiary copy ("The new eBook for bootstrappers..." etc) because that's where the highest contrast is highest.

The ostensible header ("Become the designer...") is almost invisible (I didn't see it for the first 30 seconds). My eyes keep wanting to drift back down to where the contrast is higher.

So paint me as skeptical.

NumPy! Woo! This is a big deal for writing data-intensive backends. There was a great outcry of happiness on the ticket system when this got announced.

Now all we need is SciPy and matplotlib and we'll be in business...

Permutation tests! Easy to use and easy to explain.

The two examples have the same mean and median, but differ substantially in their min/max/standard deviation/median absolute deviation. Pick your spread, they differ. I got this by poking around in iPython.

Find the standard deviation of A. Reshuffle A and B together, taking half of the resulting list, calculate the standard deviation, record it and repeat that a thousand times. See that you basically never get a standard deviation as large as you did for A alone, so they meaningfully differ.

Repeat on each of the data sets, and you see that D and E are plausibly similar to A whereas C and F are not. Repeat for B to see if we're being fucked with, and lo and behold it all seems to work out, though D is a little iffy.

I used numpy and ten lines of glue code. Happy to post if there is interest.

I can see why the Dreyfus article would be controversial, but so far (first ten pages or so) I found it very enlightening.

This really struck a chord with me:

> What the learner acquires through experience is not represented at all but is presented to the learner as more and more finely discriminated situations, and, if the situation does not clearly solicit a single response or if the response does not produce a satisfactory result, the learner is led to further refine his discriminations, which, in turn, solicit more refined responses.

Comparing that to the meaning of "learn" commonly used in ML to mean "finding optimal weights for some features" and you can see the gulf pretty starkly.

I think the typography and design are easier to read than http://sleeptyi.me, but it is uncomfortably similar wording and navigation if they're not made by the same person.

It turned out to be a bug in their testing framework.

I just finished "Hamlet's BlackBerry," which is exactly about taking time to disconnect to get "depth." A little preachy some times, but thought-provoking.

Basically, we haven't developed the social norms needed to put some space between our brains and maximal connectedness, but we can look to history and personal experience to try and figure out what some of them would look like.

It's got some good historical bits about how different great thinkers have dealt with technological change (writing, printing press, telegraph were as disruptive as the Internet). It's also got some personal anecdotes of the author's decision to have an 'Internet Sabbath' every weekend, where from Friday night to Monday morning there is no connectivity in his house.

I'm not sure that engineering (including social engineering) is going to work in a situation like this, but it's probably not very harmful to try.

Got it. For some reason I was expecting some sort of Mario AI or something. Thanks!

I'm confused. All I see is a copy of Mario in HTML5. Can someone enlighten me?

I don't think the claim is that something other than net calories are what regulates your body fat, but instead that trying to limit intake through calorie counting isn't the right way to do it.

That's what I took away, anyway.

Put an LED behind the temporal cloak. Look at it from the outside. You see an LED. Look at it through the temporal cloak. You see an LED. Now turn the cloak on and blink the LED (very fast). If you're looking through the cloak, you won't see the blink happen. If you're looking from the outside, you will.

Strangely encouraging analogy. Microsoft is still a hugely dominant player, but eventually new technologies caused their lead to erode and other, more innovative companies like Google started leading the pack.

If you're tired of Google-as-platform, just wait a few years.

There is such a project already, called PhoneGap: http://www.phonegap.com/

This is really neat. I was under the (mistaken) impression that there was not much for sale in BTC.

Right, I misread it. Also, I'm playing with it right now and you can put people into multiple circles.

I haven't seen the demo, but the indications from the press is that circles can (and should) include groups of people.