HNHacker News
TopNewBestAskShowJobs

MurizS

673 karma · joined August 21, 2018

Training multimodal models. Here goes nothing:

https://hesiod.ch/

submissionscomments
MurizS··on Still using Firefox – but not because of its vision
Native since v146, about:config -> browser.tabs.splitView.enabled
MurizS··on Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
Which in turn lets the people at Moonshot AI worry about that for them, the only provider for this model as of now.
MurizS··on Who invented deep residual learning?
I think what you're referring to is also known as Stigler's law of eponymy [1], which is interestingly self-referential and ironic in its own naming. There's also the related "Matthew effect" [2] in the sciences.

[1] https://en.wikipedia.org/wiki/Stigler's_law_of_eponymy

[2] https://en.wikipedia.org/wiki/Matthew_effect

MurizS··on Procedural knowledge in pretraining drives reasoning in large language models
I think GP was probably referring to "Scaling Data-Constrained Language Models" (2305.16264) from NeurIPS 2023, which looked first at how to optimally scale LLMs when training data is limited. There is a short section on mixing code (Python) into the training data and the effect this has on performance on e.g. natural language tasks. One of their findings was that training data can be up to 50% code without actually degrading performance, and in some cases (benchmarks like bAbI and WebNLG) with improvements (probably because these tasks have an emphasis on what they call "long-range state tracking capabilities").

For reference: In the Llama 3 technical report (2407.21783), they mention that they ended up using 17% code tokens in their training data.

MurizS··on What made Dostoevsky's work immortal
Most likely "The Brothers Karamazov".
MurizS··on LLM-generated code must not be committed without prior written approval by core
For context: https://lkml.iu.edu/hypermail/linux/kernel/2401.3/04208.html

HN discussion (4 months ago): https://news.ycombinator.com/item?id=39191899

MurizS··on What Is Applied Category Theory? (2018)
It is not that the HN community is diminishing your credentials in applied category theory or in any other fields, more so the inherited snarkiness in your short comment provides no one a favor and surely doesn't add any value to the discussion.
MurizS··on Ask HN: Best Talks of 2020?
Funnily enough, someone tells am after a minute he should "slow down and breath a little bit".
MurizS··on Life Lessons from a 97-Year-Old Lobsterman (2019)
Out of curiosity, what was your first act and what made you transition to software engineering?
MurizS··on Physicists Pin Down Nuclear Reaction from Moments After the Big Bang
Your comment reminds me of a study [1] published in 2019, demonstrating that people with the strongest views against genetically modified foods know the least about science but believe they know the most.

Implicating that the Big Bang Theory enjoys its popularity because it "sells" is, I think, showing complete arrogance to landmark discoveries made in the past century - such as the cosmic background radiation [2] (for which the Nobel Prize 1978 was awarded), supporting the thesis of the Big Bang.

[1] Extreme opponents of genetically modified foods know the least but think they know the most. https://www.nature.com/articles/s41562-018-0520-3 [2] https://en.wikipedia.org/wiki/Cosmic_microwave_background

MurizS··on YouTube Down
Maybe speedtest.net downloads a YouTube video to measure your speed /s
MurizS··on YouTube Down
Maybe speedtest.net downloads a YouTube video to measure your speed /s