HNHacker News
TopNewBestAskShowJobs

jbay808

7,340 karma · joined October 22, 2018

I'm a Canadian professional engineer and consultant, specializing in feedback systems and motors, and with expertise in several other domains.

https://thesearesystems.substack.com/

submissionscomments
jbay808··on Oxford PV sets new solar panel efficiency world record
Do you have a spec sheet for your MPPT chip that you can share? The white paper and video presentation that you linked appear to be completely unrelated.
jbay808··on Friends don't let friends make bad graphs
Great talk and very well presented.
jbay808··on Poland may seek extradition of Ukrainian Nazi WW2 veteran Hunka from Canada
No, but if a Canadian engineer put their stamp on work that was this sloppy, an apology wouldn't suffice. They'd be suspended at least, or likely even lose their license permanently. Even if the work was prepared by their subordinates and just handed to them to sign off on it.

Why should the Speaker of the House be held to a lower standard of diligence, especially when their actions have international geopolitical consequences?

jbay808··on PID Control Challenges
Absolutely not! The choice of fixed- vs floating-point for a controller should be based on domain knowledge of the problem, and the right choice may vary depending by operation. For example your PID's integrator needs to accumulate a running sum. Fixed-point addition is well-behaved for addition, with only the need to deal with overflow (which can be dealt with; the integrator likely needs to saturate anyway to respect a physical limit, so you choose a datatype that doesn't overflow before then). Floating point addition has much more nuance; adding a small value to a large one can result in no change at all to the output, causing a controller's small-signal transfer function to change based on the system state. This can lead to all sorts of unexpected bad behaviour, like vibration and limit cycles, which might happen only in edge cases after the controller has been running for some time, and so are very hard to catch in testing.

There are also many cases where floating point is the clear winner, but again, you should think about it and not just choose floating point reflexively because it seems easier. Its convenience makes it very easy to sweep errors under a rug, and they always come back to haunt you later.

jbay808··on Toyota’s advanced battery technology roadmap
It reminds me of the old story about the inventor in the late 1890s, who claimed to have invented a car that could cross the United States running on just a gallon of water. After failing to deliver a tangible product for years, investors pushed him to either produce his vehicle or return their money. The inventor refused, saying that he'd now come up with an even better version that could cross the United States on just a single thimble of water, and it would be beneath his professional pride to release an inferior product, so he had no choice but to continue his research.
jbay808··on Researchers use AI to find new magnetic materials
Iron nitride is in many respects not better than neodymium, but certainly very promising nonetheless.
jbay808··on Graph of Thoughts: Solving Elaborate Problems with Large Language Models
It is only difficult for a LLM to sort a list of numbers if the list is longer than half of the context window. (Source: I tested this myself[1]). The sorts are not error-free every time, but with sufficient training they become error-free the vast majority of the time, even for long lists. This is not especially surprising because transformers are capable of directly representing sorting programs.[2]

[1] https://jbconsulting.substack.com/p/its-not-just-statistics-...

[2] https://arxiv.org/abs/2106.06981

jbay808··on Ask HN: Could you share your personal blog here?
That was my concern as well...
jbay808··on Flashing elements alienate users
As a Substack writer, I find it very distracting. Part of the reason is that when my eyes are looking at the body text, the motion happens in my upper peripheral vision. That's also where tab heading notifications are (eg. an incoming Google Hangouts message). So as I write I keep subconsciously looking up to see what's happened, only to find it's just another "draft saved".
jbay808··on From word models to world models
> that's one preprint on arxiv, that makes a wild claim about a new concept that they acronymise as "RASP". It's not any kind of established terminology, nor is it anything but a claim.

Would you change your mind for a different link, like this one? http://proceedings.mlr.press/v139/weiss21a.html

I think you would enjoy learning about RASP, rather than taking such a hardline skeptical position.

> a function is a mapping between the elements of two sets, whereas an algorithm is a sequence of operations that calculates the result of a function and is guaranteed to terminate

I'm aware. Transformers (and RASP programs) are guaranteed to terminate; that's one of their nice properties.

> Is there a good reason not to do that?

Balanced against the value of my unpaid time, a probability of 10^-70 is low enough for the purposes of a quick and fun test.

Speaking of which, I'm going to enjoy my weekend now. I hope you enjoy yours!

jbay808··on From word models to world models
I agree a neural network can certainly learn the conditional distributions that let it make that choice correctly every time. Once it has done so, then do you not have a sorting algorithm?
jbay808··on From word models to world models
> In your comment above

Oh, I see, good catch. I think that comment was a result of a botched edit; I do that sometimes. Too late to change it now. Sorry for the confusion!

> Where does this use of "n-gram" come from? I mean, did you see it somewhere?

It's shorthand for n-gram Markov model. The same way it is presented in, for example, A Mathematical Theory of Communication.

> "Defined an ordering", where?

In order for a set to be sortable, you need to define an ordering over the elements. So for example, defining that the letter 'A' is greater than the number '99'. It's easy to take for granted that 1 < 2, but the neural network doesn't know that a priori, because the tokens are just index values. It doesn't have any way to know that token number 5 represents the character '5'.

> if you take your model that's trained on two-digit lists of integers and you run it on lists of any other type of elements it will not be able to sort them correctly.

To reiterate, the token dictionary basically just contains the characters "0123456789,():[]_\n". If you try to ask it to sort '(Tuesday, Monday)', it's just going to throw an exception because 'T' isn't a recognized token; it doesn't have a corresponding index. It's not even a question of whether it can sort them correctly or incorrectly.

> "Any input list"? How so?

I think the meaning is pretty clear. No algorithm can sort a list of elements that aren't members of a totally ordered set, so I wasn't attempting to imply that any input list meant that a neural network could somehow supersede this limitation.

jbay808··on From word models to world models
> The (sequential) distribution of digits amongst sorted numbers is tiny

This is why 10^80 random lists gets reduced to only 10^36 sorted lists. However, 10^36 is still very large with respect to the size of the model.

jbay808··on From word models to world models
> what do you think would happen if you gave your model an alphanumeric list to sort? Did you try that?

The tokenizer would throw an exception, because it doesn't have any tokens to represent alphabetical characters. But you tell me - if I had tokenized alphabetical characters and defined an ordering, would you expect the results to be any different?

> You say e.g. that "LLM is learning an n-gram"[...] you can't "learn an n-gram".

Where do I say that? I don't think I make any reference to "learning an n-gram", which is a relief because I don't know what it would mean to "learn an n-gram".

> There's plenty of error in the figure where you show its accuracy (not clear if that's training or test accuracy).

Test accuracy between training iterations (not part of the training process itself, which uses its own separate validation set which is split from the training set). And yes, I agree, it is not error-free, and I wouldn't expect it to be, especially after so little training. What the figure shows is the percentage of sorts that were error-free, and how rapidly that decreases. I've since repeated the test with finer resolution, and the fraction of imperfect sorts continues to decrease about as you expect, which is enough to satisfy my curiosity, although I'm a little curious to see if there is some point where it falls completely to zero.

jbay808··on From word models to world models
> it sure has not learned any "algorithm". At best it has learned to approximate a sorting algorithm. That's what statistical machine learning models do, they are function approximators; not program learners.

Transformers are RASP programs, which includes sorting programs. See the Weiss paper (https://arxiv.org/pdf/2106.06981.pdf).

> Also, Machine Learning 101: you test your models on a test set that is disjoint to the training set. To clarify, we do this not because it's in the book and that's the rules, but because, by testing the model on held-out data, we can predict the error the model will have on unseen data

The probability of a test list existing in the training set is less than 10^-70.

jbay808··on From word models to world models
> If LLMs "learned algorithms", the best compression would be on the order of bytes.

Yes. Except:

(1) the model size is fixed during training, it would be impossible to obtain a bytes-sized result regardless of what it learns to represent. One might even open the thing up and find bubblesort* inside followed by 599 MB of junk DNA; that size is dictated by how it was initialized.

(2) I'm not claiming this model is a minimal size; I started with the biggest model I could train on my wimpy GPU and succeeded on my first and only try, which I think is a fairer representation of how GPT-4 was built than if I'd started by proving the minimum size of transformer that could represent the task** and then (surprise!) obtained it.

(3) Compared with the size of a map of all 10^80 unique input lists to all 10^36 correctly-corresponding sorted outputs, 600 MB is a remarkable compression ratio, even if it's not reducing it all the way down to exec("sort(input)").

(4) Nowhere do I make any claim that transformers are minimal or even space-efficient representation of an algorithm (or a world-model); in fact, they seem quite terrible in this respect, especially compared to arbitrary code. And doubtless there are a bunch of weights that got trained to near-zero and could be trimmed to make the matrices more sparse, or quantized, which is the kind of thing people do to compress an LLM itself but I didn't bother. What transformers do seem to do very well at, despite the overhead, is the differentiability that allows them to be trained in the first place, and also the flexibility to handle different kinds of problems. I could have trained the same blank-slate starting model to one that shuffles or reverses each list, or perhaps to do one or the other depending on whether the first number is odd or even, or any number of other tasks.

> You're showing the system vast amounts of numbers being sorted, so it learns the distribution of that data, so it can replay those sorts.

It's almost definitely the case that every list it's tested on, and sorts 100% correctly, is a list it has never seen in training (unless it's a very short list, but I control for that). My training dataset is only about 100 MB; given the number of random lists, it's vanishingly unlikely that it's seen almost any of them, let alone the 100% of them that it is able to sort correctly. (The tests, of course, were not drawing from the validation set either; I test the model by generating new lists on the fly, because that's easy to do).

> statistically approximate the empirical distribution of the training dataset structure

Can you provide more details about what you mean by this distributional structure that can be compressed without a generally-correct sorting algorithm? How would you define a similarity measure between distinct random lists that allows for this kind of interpolation?

* Well, probably RASP-sort, not bubblesort. Also, it would need to include definitions of things like the comparison operator between all tokens, because it doesn't have a numeric datatype built in, or even the idea of numbers as an ordered set; it has to learn all that.

** (the Weiss paper does this, and lo and behold, transformers can indeed sort).

jbay808··on From word models to world models
> Redness is not in the structure of those sentences.

Sure; it's in the spectrum of reflected light. (Or perhaps, the retina's trichromal responsivity). But that physical concept can be meaningfully described by sentences. It doesn't require an infinite number of them to create a coherent world-model, which can do things like predicting that a blue object will become red if it moves away from you at a high enough speed. Which is something a human might be surprised by even after many years of visual experience with red objects -- unless they've read sentences about the Doppler effect in a physics textbook.

If you can manage to trick GPT-4 into revealing that it doesn't have a world-model of the concept of 'red', please show us!

> At a quick glance of your article it feels like you havent formulated Hyp1 correctly -- P(CorrectSort | f(HistoricalCases)) is perhaps arbitrarily high if some statistical f() can be chosen well.

Keep in mind, the LLM's structure was not hand-crafted to do well on this mathematical task. It was built to be good at language modelling, and initialized with essentially a uniform prior over all token sequences. Even if a dataset is efficiently compressible, that's no guarantee that the LLM will be able to compress it efficiently. In fact, many people would probably be surprised to learn that it can do this problem at all, let alone so well with so little training. But do think about the statistics of sorting a bit more. I think it's not as easily compressible as you think it is, except by an actual sorting algorithm. Again, you can compress it a bit with monotonicity and so on, but nowhere near the amount you'd need to sort a long list without errors, using so few parameters. I compute the number of sorted and unsorted lists in the footnotes.

One of the things that makes sorting tricky for an LLM is you always need to look at every item in the input list. Even if the previous output token was '99', you can't be sure you're now at the end of the list; you still need to count how many '99's were output already and how many are needed.

(The dataset itself, of course, does not contain the notion of sorting, a description of sorting, a test for sortedness, or any algorithm for sorting. It only contains a large but finite number of examples of sorted and unsorted lists. It's up to the LLM, and its training process, to discover the mechanism that generated these results.)

jbay808··on From word models to world models
> you havent formulated any hypotheses that can be tested. You've just looked at the graph and "reckoned something"

Let's not be so hasty. I think I do put it as clearly as possible. I'm comparing essentially your Hyp1 and Hyp2, where Hyp1 (aka the stochastic parrot) is expressed a little bit more clearly as the LLM is learning an n-gram that produces correct sorts through rote memorization of statistical correlations in the training data, like that sorted lists tend to start with '0', end with '99', and increase monotonically; and Hyp2 is that the LLM's training molds it into representing an actual sorting algorithm that would correctly generalize to any input list.

> But it's really obviously false for natural language. The properties of the world are not the properties of word order... being red isnt "red follows words like..."

This is not really obviously false. Yes, being red isn't "red follows words like...". But a word order should still map to properties of the world, especially if those words are to be meaningful to a listener. Being red is "a surface reflects or transmits most of the light in the 600-800 nm spectrum and absorbs most of the rest". Of course, it won't do to just echo those tokens; once you've nailed down the concept of "red", you need to make sure that concepts like "reflects", "light", and "spectrum" are represented as well. It's an open question as to whether this sort of knowledge graph can be properly bootstrapped from a large volume of text descriptions, but I am strongly inclined to believe it can. If you dismiss it outright you're just begging the question.

jbay808··on From word models to world models
> It is absolutely trivial to show Hyp2 is false

To investigate precisely this question in a clear and unambiguous way, I trained an LLM from scratch to sort lists of numbers. It learned to sort them correctly, and the entropy is such that it's absolutely impossible that it could have done this by Hyp1 (sampling from similar text in the training set).

https://jbconsulting.substack.com/p/its-not-just-statistics-...

Now, there is room to argue that it applies a world-model when given lists of numbers with a hidden logical structure, but not when given lists of words with a hidden logical structure, but I think the ball is in your court to make that argument. (And to a transformer, it only ever sees lists of numbers anyway).

jbay808··on Scientific colour maps
On this note, is anyone aware of any two-dimensional colourmaps that are perceptually-uniform (or close to it), or perhaps periodic?

I find I often have to hand-roll these myself.

jbay808··on Instant Brands, maker of Instant Pot and Pyrex cookware, files for bankruptcy
They do not handle the rapid temperature differentials a home chef often encounters when they aren't careful enough. Especially when they are expecting the resilience of borosilicate glass.
jbay808··on Store epoch times as 64-bit floats
I strongly disagree with this. It only makes sense to use floats if you want time to have increasing precision the closer you get to 1970. From a symmetry group perspective, that makes no sense; the amount you care about the incremental precision of time measurement is invariant with respect to the present date. (Translation invariance). This means you want uniformly distributed time samples, hence an integer format.

If you think microsecond precision is good enough for anybody, then you could consider using an integer where the LSB represents 1 microsecond. But don't use floating point time, unless your specific application cares more about time scale invariance than time translation invariance.

jbay808··on It's not just statistics: GPT-4 does reason
I won't take a hard stance on what counts as "reasoning", which I picked in the title for lack of a better summarizing word; I am open to alternatives. So if you think that making abstractions and implementing a sorting algorithm does not count as reasoning, I will not disagree with that position. Where I am going to take a hard stance is on what does a stochastic parrot cannot do. And a stochastic parrot, defined as "stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning", cannot sort lists of 127 characters.
jbay808··on It's not just statistics: GPT-4 does reason
I just can't imagine how a stochastic parrot could repeat back a correctly-sorted list that it hasn't seen in training, without actually implementing a sorting algorithm in the process. It seems (and, by calculation, is) phenomenally unlikely that it would just stochastically happen to pick every single number correctly.

When that is combined with the fact that transformers provably can implement proper deterministic sorting algorithms, it seems that the benefit of the doubt should go to the transformer having learned a sorting algorithm?

LLMs aren't plastic in the sense that they don't learn anything when they aren't being trained. But they can be trained to execute different programs depending on the contents of the context window, like if it contains "wrong, try again:" so maybe they can learn from their mistakes in that sense.

But if you could teach an LLM to sort by explaining it in the context window, the network would already have necessarily learned and stored a sorting algorithm somewhere; the text "here is how sorting is done: [...]" would just be serving as the trigger for that function call.

jbay808··on It's not just statistics: GPT-4 does reason
I would interpret this as showing that matrix multiplication code is carefully engineered to correctly implement... well, matrix multiplication. Stumbling on that specific mapping of 96 input bits to 96 output bits would be hard to pick out of a hat by chance, from the set of all possible mappings. Learning that precise mapping, starting from a uniform prior and only given a finite set of examples, could be seen as an impressive task, although less impressive than sorting. If a model learns the correct mapping -- and better yet, needs only 9 parameters to implement it -- then I think it's fairer to say the model does matrix multiplication, rather than that the model convincingly imitates the statistics of matrix multiplication.
jbay808··on It's not just statistics: GPT-4 does reason
I'm glad to see someone express this view, because I think this gets to the heart of the question. How does a stochastic parrot learn to sort lists?

Embeddings are part of the compression-by-abstraction that I'm explaining in the first two parts, but the embeddings generated by an LLM go beyond the normal word2vec picture that most people have of embeddings, and I believe are closer to whatever "understanding" means if it could be formally defined. It would be quite a coincidence if GPT-4 happened to solve the riddle merely by virtue of "Moonling" and "cabbage" being closely-located vectors.

jbay808··on It's not just statistics: GPT-4 does reason
That's right, but if you did that compression, it wouldn't be an n-gram anymore. What I'm attempting to get across is that you could model GPT-4 as an equivalent 8000-gram in an abstract sense, but that's not a good mental picture for how it actually functions. Internally, GPT-4 is no more an 8000-gram than Stockfish is a giant lookup table of chess positions. GPT-4 is learning RASP programs, not statistical text correlations.
jbay808··on It's not just statistics: GPT-4 does reason
Author here. Happy to see this discussion. Absolutely, GPT-4 sometimes has trouble reasoning and doesn't reason perfectly. I'm impressed by its successes, but I agree it's not at the human level yet, and I would not make the claim that it is.

Counting is a task that transformers can do, per Weiss.[1] But it's not surprising that transformer networks in general have trouble counting characters -- the tokenizer replaces common sub-strings, so the number of characters will not in general be the number of tokens. The network might have little way of even knowing how many characters are in a given token if that information isn't encountered elsewhere in training.

[1]: https://arxiv.org/abs/2106.06981

jbay808··on Why I'm done with Mouser Electronics
I also always checked Digi-Key first and then Mouser second if Digi-Key is out of stock on something I need. (Unless it was a TI part; their direct online ordering process is quite nice). But going forward I'll reconsider Mouser.
jbay808··on Google launches PaLM 2, its next-gen large language model
> Bard isn’t currently supported in your country [Canada]. Stay tuned!

Too bad for us...

← PreviousPage 2 of 34Next →