HNHacker News
TopNewBestAskShowJobs

dawnofdusk

353 karma · joined September 8, 2022

submissionscomments
dawnofdusk··on Learning Theory from First Principles [pdf]
First principles doesn't mean easy to read unfortunately
dawnofdusk··on Learning Theory from First Principles [pdf]
Depends on what you want from a (text)book. In my mind books should be authoritative, they should include things that have had some thought put into them and are fairly well studied/verified. Modern advances in deep learning/ML are exciting but are very often not this. I would not a read a book which is just some recent hype papers from NeurIPS/ICML stapled together.
dawnofdusk··on In Defense of Simple Architectures (2022)
Enough of this strawman one-liner "complexity for the sake of complexity". Unnecessary complexity is introduced because someone once thought it was necessary. It may not be necessary anymore, or (even worse) it might have not even been necessary at the time and the person who introduced it was just wrong. But all complex architectures start because people think it will bring benefits.
dawnofdusk··on Grand-Master Level Chess Without Search: Modeling Choices and Their Implications
>they'll talk about the "structure" of the position, and then confirm the structure via some amount of search

Yes that's sort of the point of the approach. Imagine when the board is any position, you have a list of all possible moves which are ranked based on how good they are (i.e., how likely they are to lead to you winning). In order to make this ranking you've "flattened" the future outcomes of that move into a single number which is used to rank the list. Then, you can play optimally without search: simply pick the best move at every board position. (Richard Bellman showed that, for the right definition of "best", this gives the optimal strategy for playing entire games, not just single moves.)

Now the question is, how does one actually calculate this sorted list of the best moves? The straightforward answer is essentially search and various techniques to make the search computationally tractable. What Deepmind's approach is use such a list computed by Stockfish (which explicitly searches) to compile a dataset which is then trained on in order to approximate the function that produces this ranking (called the Q-function). But the point is that search is just an algorithmic strategy to estimate the Q-function, mathematically speaking it is merely a function of the current board state and therefore in theory could be determined without any search. So, maybe a deep neural network could learn how to do this?

dawnofdusk··on Grandmaster-Level Chess Without Search
It doesn't get the actual optimal Q values computed from Stockfish (presumably this takes infinite compute to calculate), in fact it gets computed estimates from polling Stockfish for only 50ms.

So you're estimating from data a function which is itself not necessarily optimal. Moreover, the point is more like how far can we get using a really generic transformer architecture that is not tuned to domain-specific details of our problem, which Stockfish is.

dawnofdusk··on Grandmaster-Level Chess Without Search
Even if it's "implicit" I'm not sure if that matters that much. The point is that the model doesn't explicitly search anything, it just applies the learned transformation. If the weights of the learned transformation encode a sort of precomputed search and interpolation over the dataset, from an algorithmic perspective this still isn't search (it doesn't enumerate board states or state-action transitions).

>performing some function of the similarity of the input to those previously seen.

This is indeed what transformers do. But obviously it learns some sort of interpolation/extrapolation which lets it do well on board states/games outside the training set.

dawnofdusk··on Physics for Mathematicians – Introduction
Skimmed some of the articles, particularly those nearer to my field. Seems like a generally good set of informal notes.

Random comments:

>when the states evolve in time and the observables don’t we are using Liouville’s picture; when the observables evolve in time and the states don’t we are using Hamilton’s picture.

I have never heard this terminology, I have only heard Schrodinger's picture vs. Heisenberg's picture.

>This means that, very unlike on a Riemannian manifold, a symplectic manifold has no local geometry, so there’s no symplectic analogue of anything like curvature.

Perhaps the only enlightening comment I have ever heard about the tautological 1-form/symplectic approach to Hamiltonian mechanics.

dawnofdusk··on Physics for Mathematicians – Introduction
>Is there a translation of this into category theory terms?

It's essentially the same as the relation between covariance and contravariance in category theory.

dawnofdusk··on Wear OS's most consistent OEM quits: Fossil stops making smartwatches
For runners and other athletes (depending on the sport), it seems fairly useful. I struggle to think of other use cases that justify it.

Obviously, a smart watch is strictly better than a regular watch in the sense that it has more features, but I think the downside of having to charge it so regularly makes the value-add not worth it. I am always sad about the downfall of Pebble watches, because the e-ink screens were excellent and gave you a watch that rarely needed to be charged and still gave the benefits of a smart watch (however small they may be).

dawnofdusk··on The VAE Used for Stable Diffusion Is Flawed
It is a good thing for deep NNs to be expressive enough to do this, because it is this level of expressivity that let's it find answers to otherwise ill-posed problems. If it were not able to do this there would be no point in using them.

The only thing that is scary is the hype, because this will make people sloppily use deep learning architectures for problems that do not need that level of expressive power, and because deep learning is challenging and not theoretically well understood, there will be little to no attempts made to ensure safe operation/quality assurance of the implemented solution.

dawnofdusk··on The VAE Used for Stable Diffusion Is Flawed
This is one of the cool things about various neural network architectures that I've found in my own work: you can make a lot of dumb mistakes in coding certain aspects but because the model has so many degrees of freedom it can actually "learn away" your mistakes.
dawnofdusk··on Calculus on Computational Graphs: Backpropagation (2015)
Nice blog. I'll be provocative/pedantic for no good reason and say that what's described isn't "calculus" per se, because you can't do calculus on discrete objects like a graph. However, you can define the derivative purely algebraically (as a linear operation which satisfies the Leibniz chain/product rule), which is more accurately what is being described.
dawnofdusk··on Japan's airport control towers stop using 'No.1' instruction for plane takeoffs
qī and sì sound nothing alike in any dialect of (Mandarin) Chinese I can think of, to Chinese speakers. Shi and shichi literally contain the exact same phoneme.
dawnofdusk··on Does DNA have the equivalent of IF-statements, WHILE loops, or function calls?
Analog computing is trendy again, though, especially for AI.
dawnofdusk··on Does DNA have the equivalent of IF-statements, WHILE loops, or function calls?
I just mean that cells in an organism experience much higher noise rates than a CPU, but we still consider them as capable of sophisticated computations. It's hard to peg a number at exactly how much more, but at least for reference many bacteria have a gene transcriptional error rate of like 10^-4. Apparently fiber optic engineers target 10^-12 error rate, and I assume the error rate of transferring data between CPU registers is even lower. So it's probably a decent estimate to think that biological "computations" occur under at least a million times more noise than a computer.
dawnofdusk··on Does DNA have the equivalent of IF-statements, WHILE loops, or function calls?
An interesting question because all of this genetic encoding and expression is probabilistic at its very core (it has to be, otherwise there wouldn't be evolution), whereas it would be very bad if basic operations in a CPU had similar level of error rates.

Interesting to me how the top comment has talked about constructing logic gates out of biological circuits. I wonder if anyone has done the opposite, i.e., write a probabilistic programming language whose operations are under the same amount of noise as a cell?

dawnofdusk··on Flakes aren't real and cannot hurt you: using Nix flakes the non-flake way
This blog post is one of the best ones I've seen on Nix tbh. Brings a lot of clarity on things people are often confused about ("What is a flake?" "What is a module?"). Hoping it doesn't get drowned in the SEO for people
dawnofdusk··on Benchmarking 20 programming languages on N-queens and matrix multiplication
Why would uncompressed bytes be better? Using a good compression algorithm better approximates the statistical entropy of the code which is at least correlated with e.g., Kolmogorov complexity.
dawnofdusk··on Fail2ban Sucks (2020)
I'm surprised no one has ported it to some container-friendly runtime.

In any case your threat model for container workflow is already 100x more complicated than some one-server blog with an open SSH port, I would give up on fail2ban immediately

dawnofdusk··on Unix as IDE (2012)
Unix shell utilities as an idea/philosophy are a really good IDE, but in practice a lot of them need major UX overhauls. `find` is awful, for example, if only because it uses single hyphens followed by full words for it's flags (different from almost all other conventions...)

Doing anything which involves more than one pipe or xargs is just painful. Bash as a language is not ergonomic at all.

I don't know if there's a nice alternative that keeps the same spirit. My feeling is something like emacs + eshell (a shell which can execute elisp expressions)? Elisp is not a very ergonomic language either, of course... but something like this.

VSCode was a revolution in giving an IDE with a good UI, every feature one could want, and fairly good performance. In 2023, I feel like everyone's on containerized/distributable work flows so there is a big need to actually return to "UNIX as IDE" type work, and now would be a great time for some new CLI tools that improve on the past.

dawnofdusk··on On the link between Lyme disease and bioweapons (2019)
Not saying that I believe in the conspiracy theory, but can't the disease exist naturally while also being the subject of bioweapon research? The only way to know would be to figure out more precisely what kind of research was being done on the Lyme bacteria... I'm not sure if there are signatures one can look for which indicate that some strain is certainly a wild mutation and not from concerted research (similar to how archeologists distinguish which sharp rocks are ancient human tools and which ones are just coincidentally sharp rocks).
dawnofdusk··on LK-99: The live online race for a room-temperature superconductor
Type-II super conductors may exhibit "flux pinning". Type-I super conductors do not.
dawnofdusk··on It's 2023, so of course I'm learning Common Lisp
Agree. ThePrimeagen had something to say about this on one of his streams, responding to someone who said "The programming language doesn't matter, only the programmer." He said something like "If that were true, let's just all go back to writing C, it's pretty good. But then you'd say 'well not exactly...'"
dawnofdusk··on Rust vs. Julia in scientific computing
Personally a lot of the points in this article are pretty decent, but I think the solution is not Rust but something like LFortran (https://lfortran.org) which has an interactive REPL, can be AOT compiled, no performance footguns, and is Fortran (a big plus for adoption by scientists).
dawnofdusk··on Rust vs. Julia in scientific computing
Computer power is expensive and constrained for many scientists...
dawnofdusk··on Wasm_of_OCaml
Yeah it's the French word. You can always say just "chaîne" when it's unambiguous. Or you can just use the English word
dawnofdusk··on Sidebery – A Firefox extension for managing tabs and bookmarks in sidebar
Missing Vivaldi. Native vertical tabs is why I switched from Firefox to Vivaldi.
dawnofdusk··on What dif­fer­enti­ates front-end frame­works
Literally just Angular if you use RxJS, which the author evidently thinks is too hard to understand and so dismisses it.
dawnofdusk··on A physicist who bets that gravity can’t be quantized
Yeah the "gravitational hum" would not seem to be the same as gravity itself having some inherent randomness. On the flipside, in a very real sense if the cosmological history of our universe is such that we exist in a universe with an always present indeterministic hum, then for our intents and purposes this is the same as gravity itself being random... unless we can devise experiments to study gravity in other universes with different cosmological initial conditions.
dawnofdusk··on The controller pattern is awful, and other OO heresy (2013)
Yes but modules are very unergonomic for some of the use cases listed in the article, with their dependence on `__init__.py` and imports being wonky.
← PreviousPage 4 of 5Next →