70 karma · joined May 24, 2012
Seriously, whats different this time?
>We made several new observations on scaling behavior during the development of Llama 3. For example, while the Chinchilla-optimal amount of training compute for an 8B parameter model corresponds to ~200B tokens, we found that model performance continues to improve even after the model is trained on two orders of magnitude more data. Both our 8B and 70B parameter models continued to improve log-linearly after we trained them on up to 15T tokens. Larger models can match the performance of these smaller models with less training compute, but smaller models are generally preferred because they are much more efficient during inference.
Can someone experienced please explain this. Does this mean, a lean model with more training time and/or more (or better) training data will perform better than a fat model?
Is this generating videos as streaming content e.g. like a mp4 video. As far as I can see, it is doing that. Is it possible for AI to actually produce the 3d models?
What kind of compute resources are required to produce the 3d models.
In the context META, here is a quick summary Besides the sequential increases of MAU, DAU, DAP, MAP, One thing that stands out to me is:
"We anticipate our full-year 2024 capital expenditures will be in the range of $30-35 billion, with growth driven by investments in servers, including both non-artificial intelligence (AI) and AI hardware, and data centers as we ramp up construction on sites with the new data center architecture we announced late last year."
Thats a lot of investment mostly in NVDA hardware. Expect NVDA to rise. This also means good for other suppliers, AMD, INTC, TSMC. The benefit for META will be apparent in a few quarters.
Good Luck!
[1]: https://www.sec.gov/ixviewer/ix.html?doc=/Archives/edgar/dat... [2]: https://s21.q4cdn.com/399680738/files/doc_earnings/2023/q3/e...
So far it has been the best decision(investment wise) for me.
The only effort I did was to read their IPO prospectus and the latest 10q(at that time).
Thanks Gustavo.
Fun times!
That's likely the reason majority bought FSD