danger

I use them, but there are plenty of libraries out there.

http://compbio.cs.huji.ac.il/FastInf/fastInf/FastInf_Homepag...

http://robotics.stanford.edu/~sgould/svl/

http://code.google.com/p/factorie/

http://people.kyb.tuebingen.mpg.de/jorism/libDAI/

http://research.microsoft.com/en-us/um/cambridge/projects/in...

BTW, factor graphs are generally more popular than Bayes Nets these days, at least in research circles. (Bayesian networks can be seen as a special case of factor graph.)

I don't think that's true. See, for example, Finding Maximum Flows in Undirected Graphs Seems Easier than Bipartite Matching (Karger-Levine, 1997): http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.52.3...

The method sounds very interesting, but the title seems to overstate the work a bit. This is an approximation algorithm, meaning that it's an improvement on an _approximate_variant_of_ a fundamental problem, right?

There are so many possible directions that somebody might want to go, though. If you're ultimately interested in reinforcement learning, you don't immediately need to understand EM in order to "get" Q learning. Or to work on discrete problems related to computer vision, having a basis in linear programming and algorithms will get you further than logistic regression or SVMs.

I'll agree that anybody doing machine learning should understand the basic idea of maximum likelihood learning (e.g., logistic regression). But how far do you go beyond that? This is really the heart of the issue, I think. Yes, there are a ton of things that are really useful to know, and any machine learning student should learn them at some point. If you're trying to do an interesting project that teaches you something about research (and thus looks good on a graduate school application), though, what is the best use of your time?

+1 for this thread

your idea sounds interesting, but i think it's getting at a different comparison. as presented, the contest is about who can design an algorithm to get the most out of a set (but large) quantity of data.

what you're suggesting is where to get the best data from. i think it's also an interesting question, but it would need to be in a separate category at the very least. also, wouldn't the best bet (pun intended) just be to go with the bookie odds? people who are putting money on the games presumably have better information than somebody just tweeting their prediction.

You think you can build a model that is better than the bookies's?

That sound fun. I think the rule should be a bit more hardcore, though: that the predictions have to come from raw data. i.e., no meta-algorithms that use information about seeding or expert predictions, but if somebody wanted to gather, say, play-by-play data and use that, it'd be ok.

Great, timely article!

I've tried that one before and--no joke--the person got very excited, and started going on and on about aliens and life on Mars. I couldn't figure out why until I realized that he had mistaken AI for ET.

A few memories of Sam from a Toronto machine learning student: http://blog.smellthedata.com/2010/01/sam.html

Here's a bit more on the subject: http://news.ycombinator.com/item?id=846514

SVD doesn't work when you have a sparse matrix as input. You could use some sort of imputation method (like EM) to fill in all of the missing values then run SVD, but this would be extremely inefficient, since only a tiny fraction of the user x movies matrix of ratings is observed. It also probably wouldn't work very well, because your true signal from the data would be overwhelmed by all of the missing values that you'd somewhat arbitrarily filled in.

PMF is a generalization of SVD that is able to work directly on the sparse structure of the input. It does gradient descent -- modifying the low dimension latent vectors directly to minimize error, and never does any imputation of missing values. This makes it very efficient, and it can easily handle millions of ratings. See http://www.cs.toronto.edu/~amnih/papers/pmf.pdf for an application of PMF to the very large Netflix problem.