Matthyze

This is really great. It's really awesome to have free access to such resources.

That should be very doable, shouldn't it be? 2200/(4*365) ~= 1.51 pages per day. Of course, that doesn't exclude weekends and holidays.

I wonder what would happen if the US got involved in a war against the Mexican cartels. Is that feasible, if even desirable?

I think you might've taken my point too strongly. Of course math is very useful, and certain contributions are purely mathematical. I just don't think it is as hard of a requirement for innovation as was claimed.

I think CNNs follow very naturally from the notion of shift/spatial invariance of visual processing. That doesn't require a mathematical understanding.

Are math skills really? Most aspects of deep learning don't require a deep understanding of mathematics to understand. Backprop, convolution, attention, recurrent networks, skip connections, GANs, RL, GNNs, etc. can all be stood with only simple calculus and linear algebra.

I understand that the theoretical motivation for models is often more math-heavy, but I'm skeptical that motivations need always be mathematical in nature.

I'm Dutch and genuinely confused by what you're getting at. There are foreign datacenters here but AFAIK not particularly many.

> Each time there is an algorithmic advancement, people quickly rush to apply it to both existing and new problems, demonstrating quick advancements. Then we tend to hit a kind of plateau for a number of years until the next algorithmic solution is found.

That seems to be how science works as a whole. Long periods of little progress between productive paradigm shifts.

Not the greatest at economics, but the 'Implement tariffs -> market crash -> rollback tariffs -> market recovers' cycle immediately makes me skeptical. This might work once, but subsequently, investors will simply anticipate future tarrifs (implementations or rollbacks) and stop responding to them immediately. We've already seen this when the stock market didn't react positively to the rollback of Mexican tariffs.

It is. Psychological torture is torture.

That's amazing. Really awesome work.

In some sense, I supose, but that's now how I woud characterize traditional media covering e.g. natural disasters.

25% or 10% additional tarrifs. Not sure what the current tarrifs for China or Canada are.

Spot on. Reminds me of that old approach by evangelicals to frame scientific consensus as 'just a theory.'

And most people believe there's too much crime. The DOGE answer to that is to round up people indiscriminately.

(Not entirely indiscriminately — gotta get those political opponents!)

That's insane. How did this ever get approved?

Wikipedia is useful for reference, but not education. Not sure whether that's intentional.

Indeed.

“Where processing is to be carried out on behalf of a controller, the controller shall use only processors providing sufficient guarantees to implement appropriate technical and organisational measures in such a manner that processing will meet the requirements of this Regulation and ensure the protection of the rights of the data subject.”

Tools can be crutches — useful but ultimately inhibitory to developing advanced skill.

Law isn't synonymous with prosecution. Think decriminalisation.

Thanks! Now might actually be a great time for me to pick up the material. If anyone has suggestions about particularly good free/online mathematics courses for ML, I'd really love to hear it. Or books!

I'd humbly like to ask people who've read the paper whether it's worth trying to understand it without a great math background. The paper looks intersting but daunting, and I'd hate to sink a lot of time into it and leave defeated.

It sometimes sucks being in ML with 'only' a CS background. Feels like all the math and physics grads are running around having fun with their fancy mathematics, while I stand here, feeling dimwitted.

Announcing your plan to soon buy a large amount of bitcoin sounds like a great way to pay more for that amount of bitcoin — right?

Perhaps the reason that this approach works well is that, while the LLM gives you good general-purpose language processing, the decision tree learns about the specific dataset. And that combination is more powerful than either component.

The NFL theorem (for optimization) has a mathematical proof, FYI. But I agree that there's a lot of room for interpretation.

That's essentially the No Free Lunch (NFL) theorem, right?

The thing about the NFL theorem is that it assumes an equal weight or probability over each problem/task. It's impossible to find a search/learning algorithm that performs superiorly over another, 'averaged' over all tasks. But—and this is purely my intuition—the problems that humans want to solve, are a very small subset of all possible search/learning problems. And this imbalance allows us to find algorithms that work particularly well on the subset of problems we want to solve.

Coming back to representation and maps. Human understanding/worldview is a good example. Human understanding and worldview is itself a map of reality. This map models certain facts of the world well and other facts poorly. It is optimized for human cognition. But it's still broad enough to be useful for a variety of problems. If this map wasn't useful, we probably wouldn't have evolved it.

The point is, I do think there's a philosopher's pebble, and I do think there's a few free bites of lunch. These can be found in the discrepancy between all theoretically possible tasks and the tasks that we actually want to do.

So, if I understand the approach correctly: we're essentially doing very advanced feature engineering with LLMs. We find that direct classification by LLMs performs worse than LLM feature engineering followed by decision trees. Am I right?

The finding surprises me. I would expect modern LLMs to be powerful enough to do well at the task. Given how much the data is processed before the decision trees, I wouldn't expect decision trees to add much. I can see value in this approach if you're unable to optimize the LLM. But, if you can, I think end-to-end training with a pre-trained LLM is likely to work better.

Thanks for the thought-out reply. That said, we disagree on quite a few points. A 'lesson' is something to be learned from, and Rich Sutton explicitly mentions in his conclusion what we should learn from this lesson. But it's indeed not law. His argument is also not limited to ANNs, nor even ML. His first example concerns state space search and chess. In general, I think the field is moving further away from expert/domain knowledge and towards reliance on copious amounts of data and computation. End-to-end learning embodies this. LLMs incorporate essentially no domain knowledge about human language (processing). But of course it remains a matter of degree — certain domain knowledge remains very useful.

So, what's the use-case for PINNs, then?