HNHacker News
TopNewBestAskShowJobs

smaddox

1,728 karma · joined January 15, 2015

submissionscomments
smaddox··on Why can't you multiply vectors? [video]
Yes. In physics. E.g. Maxwell's equation reduces to a single equation in geometric Algebra: https://peeterjoot.com/archives/geometric-algebra/maxwells_g...
smaddox··on GaussianDreamer: Fast Generation from Text to 3D Gaussian Splatting
Gaussian splats are like 3 months old. They currently can't be relit, and there's currently no way to turn them into a mesh+texture. If/when either of those problems can be solved, I certainly would expect them to be used in the games industry.
smaddox··on AI hype is built on flawed test scores
> And that's because, despite all of its training data, it's not capable of actually reasoning.

Your conclusion doesn't follow from your premise.

None of these models are trained to do their best on any kind of test. They're just trained to predict the next word. The fact that they do well at all on tests they haven't seen is miraculous, and demonstrates something very akin to reasoning. Imagine how they might do if you actually trained them or something like them to do well on tests, using something like RL.

smaddox··on Visual Node Graph with ImGui
> This felt like a good solution; I could instantly see how most features could map to this setup and it most definitely wasn’t a visual programming system…

Huh? I don't get this. The example right below seems a lot like coding with nodes to me. Perhaps they mean it's less procedural, more declarative? But that's the case with all of these visual programming interfaces. At least until you run into the limits of the declarative sub-language.

smaddox··on Inside the Matrix: Visualizing Matrix Multiplication, Attention and Beyond
If you're interested in developing an intuitive understanding for how information (items and relations) can be represented in vectors and matricies, I highly recommend this paper titled "Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors":

http://ww.robertdick.org/iesr/papers/kanerva09jan.pdf

smaddox··on Relational is more than SQL
> Automatically managed, application-transparent, physical denormalisation entirely managed by the database is something I am very, very interested in.

Sounds a bit like Noria: https://github.com/mit-pdos/noria

smaddox··on Relational is more than SQL
Common enough to have a name: the N+1 query problem.
smaddox··on How would you say “She said goodbye too many times before.” in Latin?
Roughly the same number of syllables, though.
smaddox··on The first observation of neutrinos at CERN's Large Hadron Collider
This appears to be the reason:

> Until now no neutrino produced at a particle collider has ever been directly detected. Colliders copiously produce both neutrinos and anti-neutrinos of all flavors, and they do so in a range of very high energies where neutrino interactions have not yet been observed. Nevertheless, collider neutrinos have escaped detection, because they interact extremely weakly, and the highest energy neutrinos, which have the largest probability of interacting, are predominantly produced in the forward region, parallel to the beam line. In 2021, the FASER collaboration identified the first collider neutrino candidates 13 using a 29 kg pilot detector, highlighting the potential of discovering collider neutrinos in LHC collisions.

smaddox··on Ask HN: Where do I find good code to read?
Rust stdlib code is quite high quality although not particularly dense due to large amount of comments. Start from the docs, and click any source link: https://doc.rust-lang.org/std/vec/struct.Vec.html

Sqlite is supposedly high quality C code: https://github.com/smparkes/sqlite

For videos of someone (Casey Muratori) writing video game code and debugging it, Handmade Hero: https://handmadehero.org/

A blog post about how to write code by the same author: https://caseymuratori.com/blog_0015

For how to implement a fairly advanced type system, Typing Haskell in Haskell: https://gist.github.com/chrisdone/0075a16b32bfd4f62b7b

But, honestly, you're probably better off writing code yourself and learning by doing.

smaddox··on An Old Conjecture Falls, Making Spheres a Lot More Complicated
Looking beyond the Lovecraftian mysticism, this is actually pretty fascinating in and of itself. I'm not a true mathematician (only a lowly electrical engineer with some training and study in mathematics), but I would not have expected that equality to hold for complex values, let alone quaternions, etc.
smaddox··on An Old Conjecture Falls, Making Spheres a Lot More Complicated
> I always wonder what a popular science/math magazine would look like if it were oriented towards hackers. In this I mean people who have little background in the field but also the type of person who is used to bluntness and knows to RTFM.

Expensive.

smaddox··on GIL removal and the Faster CPython project
Could you? What enforces only a single thread having access to a given reference? What about global variables?
smaddox··on Do Machine Learning Models Memorize or Generalize?
> I have yet to see a neural network actually learn representations outside the range in which it was trained

Generalization doesn't require learning representations outside of the training set. It requires learning reusable representations that compose in ways that enable solving unseen problems.

> On generalization - its still memorization

Not sure what you mean by this. This statement sounds self contradictory to me. Generalization requires abstraction / compression. Not sure if that's what you mean by memorization.

Overparameterized models are able to generalize (and tend to, when trained appropriately) because there are far more parameterizations that minimize loss by compressing knowledge than there are parameterizations that minimize loss without compression.

This is fairly easy to see. Imagine a dataset and model such that the model has barely enough capacity to learn the dataset without compression. The only degrees of freedom would be through changes in basis. In contrast, if the model uses compression, that would increase the degrees of freedom. The more compression, the more degrees of freedom, and the more parameterizations that would minimize the loss.

If stochastic gradient descent is sufficiently equally as likely to find any given compressed minimum as any given uncompressed minimum, then the fact that there are exponentially many more compressed minimums than uncompressed minimums means it will tend to find a compressed minimum.

Of course this is only a probabilistic argument, and doesn't guarantee compression / generalization. And in fact we know that there are ways to train a model such that it will not generalize, such as training for many epochs on a small dataset without augmentation.

smaddox··on How RLHF Preference Model Tuning Works (and How Things May Go Wrong)
Startups often have blogs in order to increase their visibility. It can help with both marketing and recruiting.
smaddox··on 50M tons of water vapor from Tonga's eruption could warm Earth for years
I read something about this recently. Clouds indeed reflect sunlight, but they also trap IR radiation from the surface that otherwise would make it out in to space. The clouds also radiate IR into space, but the higher the clouds, the cooler they are, and the less they radiate. Apparently, for very high clouds, the net effect is heating (i.e. the heat trapping effect is larger than light reflecting effect), while for low clouds, the net effect is cooling.
smaddox··on Oregon decriminalized hard drugs – early results aren’t encouraging
Interesting review, thanks for sharing.

There's no doubt that there are dimensions along which sugar and cocaine are different, but your conclusion is unsubstantiated, IMO. The paper's conclusion is much more restrained:

> Given the lack of evidence supporting it, we argue against a premature incorporation of sugar addiction into the scientific literature and public policy recommendations.

Lack of evidence is not necessarily evidence of lack. Most of the arguments the review makes are that a mechanistic link between sugar/fat consumption and addictice eating behavior has not been substantiated by the literature. The review does not argue that such a link is impossible or improbable, nor does it argue that addictive eating behavior is in a different stratosphere from addictive drug use behavior.

smaddox··on Oregon decriminalized hard drugs – early results aren’t encouraging
> Though for sugar, I haven’t personally seen people pawn off their childrens possession or rob people to get a fix, so it might not be as bad as drugs.

Sugar is both legal to possess and readily available at very low cost. No one needs to rob people to afford it.

Rats prefer both sucrose (table sugar) and saccharine (artificial sweetener) over cocaine: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1931610/

smaddox··on Semiconducting Transport in LK99 reproduction attempt
This is a very nice summary, especially for those of us who cannot read Korean. Thank you. If you posted it as a top level comment, it might get some more visibility.
smaddox··on Semiconducting Transport in LK99 reproduction attempt
If the DFT paper is correct, then annealing would let the Cu atoms relax to the Pb(2) sites, which doesn't induce potentially superconducting bands.

Metastable alloys (roughly, alloys that are only stable at room temperature because the atoms get frozen in place too quickly for them to reach the energetically favorable positions) are quite common in materials science, both for bulk metals and for semiconducting crystals. There are far more possible metastable alloys than stable alloys, so the range of material properties possible in metastable alloys is far wider than that in stable alloys. So, considering we have never discovered a room temperature superconductor, so far, it wouldn't be particularly surprising if it requires a metastable alloy.

smaddox··on The first room-temperature ambient-pressure superconductor?
Copper is diamagnetic, not paramagnetic
smaddox··on Ask HN: Which recent research paper blew your mind?
Ahh, I guess that's been done, too: https://proceedings.neurips.cc/paper_files/paper/2018/file/6...

Now we just need an iterative solver over both the structure and the "weights", and we get both architecture search and training at the same time

smaddox··on Ask HN: Which recent research paper blew your mind?
Nice. I was wondering if something like this is possible a few days ago. The next step would be somehow extending the discrete->continuous concept to layers.
smaddox··on Generative AI space and the mental imagery of alien minds
> The assumption that everything about our physical universe is described by computational rules is not a given, and is never explored nor justified.

That's in the linked article about the ruliad, among other places in his writings about their physics project: https://writings.stephenwolfram.com/2021/11/the-concept-of-t...

smaddox··on Age of universe at 26.7B years, nearly twice as old as previously believed
When the model doesn't fit, add an extra free parameter!

Seriously though, I wonder what the ramifications to other parts of astrophysics would be if this is true.

smaddox··on Scaling Transformers to 1B Tokens
> Recurrent neural networks (RNNs) do this implicitly, but we don't yet know how to effectively train RNNs on ultra-deep sequences.

What would you call "ultra-deep"? [1] shows how to train an RNN in a GPT-like mode, using parallel scan, with great performance on Path-X, which has a sequence length of 16k. It's based on prior papers doing the same thing but from a state space model perspective.

[1] https://arxiv.org/abs/2303.06349

smaddox··on Supreme Court Rules Web Designer Can Refuse Work on Same-Sex Wedding Announcemen
> Smith elaborated in a sworn statement that she “received a request through the ‘contact’ webpage on my website from a person named, ‘Stewart,’ reference number 9741406, to create graphic designs for invitations and other materials for a same-sex wedding (‘same-sex wedding request’).”

So is she going to be prosecuted for perjury?

smaddox··on Rise of pill-popping tech execs
What processes/procedures are in place to prevent polls like this from being completely fabricated? Why should I trust this data any more than a tobacco company's studies showing it's perfectly safe to smoke cigarettes?
smaddox··on A trick to eliminate 2π (sometimes)
> You should call it d/dx bar.

Or put the modifier on the denominator so that the product and chain rules are obvious (the modifier only persists on the dx, not on the dy)

smaddox··on H100 GPUs Set Standard for Gen AI in Debut MLPerf Benchmark
So it would be more like 46 hours to train GPT-3 from scratch.

    (300BT / 1.2BT) * 11 min * (1 hr / 60 min) = 45.8 hr
Still pretty incredible. That's an 18.8x speedup over 36 days.
← PreviousPage 3 of 29Next →