I would say, to be succinct, that the main trick in ML is Occam's Razor (http://en.wikipedia.org/wiki/Occam%27s_razor).

It has been found that, for most problems, a simple model which well represents previous experience should be accepted instead of a more complex one with marginally better representation. I would claim that the reason this generally works is an empirical discovery, as opposed to a mathematical result, but probably has philosophical implications in its success.