In theory, if you grew the RNN's state quadratically vs sequence length, you could likely achieve comparable performance to transformers, but it would likely be less efficient than transformers.
1,728 karma · joined January 15, 2015
In theory, if you grew the RNN's state quadratically vs sequence length, you could likely achieve comparable performance to transformers, but it would likely be less efficient than transformers.
Also known as an anti-reflection coating. Definitely not unprecedented.
Cool project, though.
Still not clear what it would be useful for outside of clocks, but definitely a very cool physics project, and I'm sure someone would find other uses eventually.
This is news to me. I'm trying to think where non-determinism would come in at temperature zero, but coming up with nothing. What am I missing?
Perhaps doable with a free electron laser, but probably not with traditional lasers, due to the energies involved.
But, yeah, not sure what the use would be. Maybe a form of lidar that allowed measuring speed of objects to extreme precision, by measuring the dopler shift of reflected light? (Assuming light at such a short wavelength is reflected sufficiently, which it probably wouldn't be).
If there happened to be one atom/matrix that could be tuned to the transition energy of another atom's fission transition energy, then you could use it to burn nuclear material / waste. But you could just do that with the pump laser directly.
I suppose you could maybe pump such a laser with a very high temperature plasma (like fusion temperatures hot), rather than with a free electron laser. Then maybe it might make more sense.
We know that helion is able to recover 95% of the energy of every pulse. And they've measured the scaling laws. There doesn't seem to be anything that will prevent net positive energy in the reactor they're building right now.
RetNet was published in August (although I only learned of it last week): https://arxiv.org/abs/2307.08621