ersiees

It’s same price for lower latency.

Sub two hour marathon is a more recent example of this

Meta, please go back to publishing weights. Your models aren't top-tier, this wouldn't hurt you at all at your current position in the rankings. They wouldn't have the best open-weight models, but the best western open-weight models.

Isn’t it too late for that? Won’t that rather cement the oligopoly we have right now?

I think the European urinals already follow the proposed designs closely. They only compare to very old chunky designs.

I don’t love fennel, it usually dominates the whole taste of a dish for me

So, 100$ Nike shoes will soon be 125$.

Author here!

Thanks :)

1. Only for the first version, not for this version. I am sorry! 2. Yeah ours is guaranteed ok, as we wrote code to generate it basically just from plain torch ops. The code to run inference is available, just not the training code and data generation. 3. We have put it to work on time series data, which is very business relevant for example https://github.com/liam-sbhoo/tabpfn-time-series, and we have a table in the Appendix with all datasets we evaluate on in our main analysis to give you some ideas for possible datasets.

Yes, there are normalizations applied before the features are fed to the neural network. Additionally, the neural network is trained on a very diverse set of artificial datasets.

Author here: The new introduction of attention between features did make a big impact compared to the first variant of TabPFN. The old model handled every feature like it was completely different to be feature 5 vs 15, but actually features are typically more-or-less permutation invariant. So the logic is similar to why a CNN is better for images than an MLP.

To put it very simply, the trick is that while the others train a new model for each problem, TabPFN is pre-trained to handle any kind of problem on the fly.

To draw a parallel to NLP: previously people trained a neural network for each kind of text classification they wanted to do, but then LLMs came around that pre-trained to learn to perform new tasks on the fly. Similarly, TabPFN learns to do new tasks on the fly just from the context (dataset) given.

Training and prediction in these models is by default one and the same, similar to how the prediction of the next token in an LLM is not split into learning from context and then doing the actual prediction. There is a way to split this even up, though, then the predictions, I believe, take something like 1/10s for medium-sized datasets.

Tuned per dataset

No, it is *much* stronger, a different architecture and scales to 10x the number of examples. It can also do regression now, and handle categorical features. Please, have a quick look at the abstract before making such claims.

Another clue: there is no way to download the latex, while you can if someone uploaded the latex on arxiv.

What is the alternative to AI twitter? Is there some incumbent where it moves?

And there is scratch right?

Why is it only for 7 days available?

I think this is also interesting to keep in mind when thinking about the changes to the job market that are possible due to technology (e.g. AI).

Most of care will not be automated any time soon. And it is a huge part of the economy.

Someone having a non paywall link?

I think I was not very specific, but I think there is a lot of video on YouTube that does not make any money for the producers and in the past YouTube also did not show ads for these videos, but now they show Ads even if the producers of the content don't receive any money. I watch mostly lectures and niche videos on YouTube, I am pretty sure for most of these videos, the only entity getting Money out of it is YouTube.

Check out Lektor.lol an open source wrapper around ChatGPT I created just for that :)

more of a rural myth, I guess ;D

This trick “they found” is part of the standard torch implementation of multi head attention, namely it is called, add_zero_attention. They add a zero to the logits, resulting in a one in the denominator as e^0=1 https://pytorch.org/docs/stable/generated/torch.nn.Multihead...

Very interesting that someone finally tries out muP in the real world. Do I understand the usage correctly:

MuP is only used to get around choosing an lr for each size? Here I wonder how it compares to standard heuristics like the one in the OG scaling laws paper by OAI and tricks like back winding a few steps after loss explosion.

For some reason muP was not trusted with the largest trainings? Why is that?

Why do we call people expats and not rich migrants or something like that?

What about inside trade? Isn’t there a big risk that people at institutions you use as source take bets last minute before they announce the critical information? Or maybe even change what they publish (maybe pretending it to be an honest mistake, which is corrected the next day) to win?

Can I also trade “Will Chelsea win against Barcelona next weekend?”

I can’t stand this twitter behaviour: something new comes out, they try to find examples where it fails and say it is generally bad because of these failures. Have some respect for the work of others.

Not

We did try it a bit a while back, but did not have conclusive results. I expect you can bend it to perform better for larger datasets, too, but how exactly I cannot say for sure. The bootstrapping is definitely a good candidate for this.