ActivePattern

I believe that Chris Lattner (Mojo lead) has said that its going to be open-source eventually. He's followed the same process as with Swift, where they did't open it up until major design decisions had been worked out with the core team.

AFAIK, it's a general purpose language. I don't see any reason you wouldn't be able to use it for applications outside of ML.

I think the reason that they focus on pure Python is because a big use case would be pure Python code bases that can be incrementally ported over to Mojo for better performance, without needing to totally abandon the Python language and ecosystem.

If it's training on the same data that it generates, there's no new information being added into the system. You'd be reinforcing everything that it already gets right and wrong, which would lead to zero improvement.

That said, it's common to use large models to generate synthetic training data for training other smaller models. In this way, we're able to transfer knowledge from one model to another.

If we are considering the function to be the neural network with an argmax applied to the output probabilities, it's not overfitting at all. Its classification accuracy over unseen data (validation set) continues to improve.

The issue here is one of calibration: https://en.m.wikipedia.org/wiki/Calibration_(statistics). That is, the output probabilities of the neural network do not reflect the true (observed) probabilities. If it is systematically underestimating the probabilities, it is termed "underconfident", and if overestimating the probabilities, "overconfident".

Note that in these cases, it may still be improving as a classifier on unseen data, while still showing higher validation loss as calibration degrades.

I wouldn't have guessed that bamboo has a strong taste. I've used other wooden cutlery before and it doesn't really affect the taste of food at all.

It's already been reverted.

I'd be curious how much of the link you read.

What they demonstrate is a neural network learning an algorithm that approximates modular addition. The exact workings of this algorithm is explained in the footnotes. The learned algorithm is general -- it is just as valid on unseen inputs as seen inputs.

There's no memorization going on in this case. It's actually approximating the process used to generate the data, which just isn't possible using k nearest neighbors.

I suspect the model is fundamentally the same underneath, but that various tricks like quantization are being performed in the deployed model to improve inference speed/cost at the expense of output quality.

Then simply train the model to predict whether a human can interpret it as a unicorn.

I would imagine that all new ideas in chess (particularly openings) come from analyzing positions deeply with the computer. It is an indespensible tool at the highest levels.

The computer often plays moves that go against our human-crafted chess "principles", but that happen to work for reasons that only make sense if you can calculate 40 moves into the future.

Yeah, that's closer to the definition I'm familiar with. It's a species of "correlation does not imply causation".

What the author is talking about here seems to be how to recognize and avoid confirmation bias.

No, your expected value is indeed positive over repeated iterations: (1.05^i)*w, where i is the number of iterations and w is the starting wealth.

The intuition for why this happens is that the losses of the majority are made up by the big gains of a minority. You can even see that after two iterations:

Case 1: two heads -- 225% wealth Case 2: heads, tails -- 90% wealth Case 3: tails, heads -- 90% wealth Case 4: two tails -- 36% wealth

So, most people are losing money but a single lucky guy makes enough money that the expected value is still positive.

If you run a simulation with a low sample size, you might not see this effect because the "lucky guy" case is rare (and gets more rare with more iterations).

As a Canadian, it does feel like stepping out of a time machine when you pay at restaurants in the USA. Instead of using a terminal at the table to pay yourself, you need to give the server your card and wait for them to manually process it somewhere. Maybe things have progressed in recent years. But we haven't done it that way in Canada since the early 2000's.

Right, and that's a legit argument against Technical Analysis. The meteor argument makes no sense, for the reason above.