versteegen

Chollet is a leading skeptic of the generality of LLMs (see arcprize.org). He surely isn't doing a startup to train another one.

Humans have a massive pro-human bias. Don't ask one whether AI can replace humans and expect a fair answer.

Agreed, I've wondered why it's so ignored since I first learnt to program, and relatedly, why are programming languages strictly English-based?

Those slides are my introduction to Fortress. Sounds nice, sorry to hear it's discontinued. I'm struck by how very similar to Julia it is (which is officially inspired by Fortress), for example the focus on performance and maths, zero-cost abstractions, type specialisation, very similar syntax, ASCII<->Unicode equivalence, encouragement of unicode identifiers and operators, and some (small) amount of eliding '*' for multiplication. (You can write "2(a + 1)a^2" but not "(a+1)(a+2)" because the latter is a function call. Whitespace between multiplicands not allowed.)

It seems it would be easy to write a package to display Julia code as HTML or Latex, although I didn't find one. There's already a library to print expressions as Latex, and of course in a Jupyter notebook results are displayed in HTML.

Testability as you describe it seems to give you more than just knowledge, but also some amount of understanding: understanding of consequences (not necessarily understanding of causes) — you mentioned the ability to make predictions about the consequences of actions (e.g. 'tests'). (Aside: it seems that you can say you know something, it's a narrow enough concept to be sharp, while understanding something can only ever be true to a degree: it's broad without limit!)

But you may have conflated 'testability' and 'tested'. Can I know there is a cow in the field if I don't check? Seeing it was already evidence, testing just collects more evidence, so how can that matter? Should we set a certainty threshold on knowledge? Could be reasonable.

Maybe prediction-making is too strong to be necessary for 'knowing', if we allow knowing some fact in a domain of knowledge of which you're otherwise clueless. Although very reasonable to not call this knowledge. Suppose I learn of an mathematical theorem in a field that's so unfamiliar that I can't collect evidence to independently gain confidence in it.

> their advancement in formal theorem proving is hindered by a lack of training data

'Formal' which here means inputs to a proof verifier/proof assistant, and I think most people would accept it as a given that by the extremely data-hungry standards of LLMs there is a severe lack of such data, and that this is a severe hindrance to getting an LLM to generalise. The question is how they might scale with more training data.

Trivia: at least in some BASICs (QBasic, QuickBasic, FreeBASIC) it's not necessary for every GOSUB to be matched with a RETURN, since the GOSUB is a single 'call' instruction that pushes onto the stack (and RETURN is a single 'ret' instruction, hence GOSUB-RETURN is highly efficient!) so it gets removed when you exit the function if you didn't RETURN, like alloca(). I actually re-implemented GOSUB this way using macros and inline assembly. I called it GOSUB/RETRACE.

Inference/learning of densely connected graphical models is a huge difficulty. It's not a big problem for MRFs with nearest neighbours only, but it is for anything interesting (e.g. long-range connections), so I felt quite jealous of neural networks with simple feed-forward computation. Using feed-forward computations in one direction (likely observed->latent) of hybrid models is really appealing to me. Instead of having to use approximations (e.g. Contrastive Divergence (CD) instead of actually sampling from an RBM) which are easy to work with, why not just build your model around the easy thing in the first place so it's not an approximation, and use approximation in the other direction only. I'm not very familiar with Helmholtz machines but they seem to do that, but (unsurpisingly) seem just as difficult? I'm not sure. But I don't like Helmholtz machines and VAEs with Gaussian priors over latents, that seems like making zero use of the power of graphical models. I want the best of both!

Bose-Einstein statistics, which originated purely from finding a derivation of an equation, without immediately understanding of the true physical meaning of the derivation.

And then Bose-Einstein condensates; a thought-experiment consequence of Bose-Einstein statistics.

Radio waves etc., from formulating Maxwell's equations.

I should mention Restricted Boltzmann Machines (RBMs, which Hinton got the Nobel for): it's really natural to use hidden variables in formulating a problem like this: one lattice of binary variables for GoL time step 0, a second for timestep 1, and optionally a third of continuous variables (or eg 256-level greyscale) for the unquantised target image. (Keep stacking RBMs for more timesteps.) Then you have a bi/tri-partite undirected graphical model. RBMs can be generalised in many ways and one of them is to replace the 2nd-order (pairwise) interactions between variables with higher-order interactions like these 9th order ones -- their most essential property is being bipartite so you can infer each variable on a layer independently given the other (or neighbouring) layers. The article didn't say how the target (timestep 1) configuration was picked, but you can simultaneously optimise for the timestep 0->1 error and the timestep 1->image distance, which could give a better looking result.

And when you have a (two-layer) RBM you can integrate out the hidden variables and get a MRF without them (as I first described), or vice-versa. Which is more useful depends.

Or if you're more interested in textures, RBMs are used for texture and image modelling (e.g. [1]), and higher order statistics are far more interesting. In virtually all papers they take the form of the responses of linear filters (like that 3x3 filter in OP, often wavelets for natural images). But you can use completely different statistics. I found that long-range local binary patterns (LBPs), used for texture recognition, are good for texture generation too.

I've switched away from textures, but energy-based models are harder to escape. A surprising intersection of old and new topics at [2].

[1] R Liao, S Kornblith, M Ren, D Fleet & G Hinton, 2022, "Gaussian-Bernoulli RBMs Without Tears" https://arxiv.org/pdf/2210.10318

[2] C Zhang, B Jia, Y Zhu & S-C Zhu, 2024, "Human-level few-shot concept induction through minimax entropy learning" https://www.science.org/doi/10.1126/sciadv.adg2488

Simulated annealing (basically MCMC sampling with a temperature schedule) is how you optimise or sample the equivalent MRF, which I discussed in my other comment. You can hope to escape local minima using annealing, and lower the temperature to zero to fall into a local minima, minimising the noise added by annealing. In practice if you're trying to produce something that looks like a target image as in the article I'm pretty sure the results will be indistinguishable. If you actually cared about how many individual pixels are correct, yes, annealing is better than gradient descent. That's why stochastic gradient descent is used in ML.

The objective function here defines a Markov random field (MRF) with boolean random variables and certain local statistics of nearest neighbours, either uniform if the target is a white image, or varying with location to produce an image. MRFs define Gibbs probability distributions, which you can sample from (which will already produce a good image here) or perform gradient ascent on to reach a local maxima. The negative log-likelihood of the MRF distribution is equal to the loss function of the original optimisation problem, so the maximum likelihood estimate (MLE) (there will often be multiple due to symmetry) of the MRF is the optimal solution(s) to the original problem. (But in general the MLE can look completely different to a sample.)

The statistics are 9th-order (of 3x3 blocks of pixels) but of a simple form which are hardly more expressive than 2nd-order nearest neighbour statistics (in terms of the different textures that they can reproduce) which are well known. In the approximate case where you only care about the average value of each pixel I think it would collapse to 2nd-order. Texture synthesis with MRFs with local statistics is discretized (in space) Turing reaction-diffusion. I did my PhD on this topic.

Probably the most influential early paper on this kind of simple texture model, where you will see similar patterns, is:

Cross & Jain, 1983, PAMI, Markov Random Field Texture Models

OpenCyc is mostly just the core ontology (knowledge graph). I'm told it is nothing in comparison to Cyc. None of the interesting algorithms (just some basic inference). It was so terribly named that they had to discontinue it.

But I imagine it's still a great resource. Haven't played with it.

(The source for both AM and Eurisko were discovered at the same time.)

I was reading through the source code for both (and the Interlisp manual, but mostly Eurisko) earlier this year and found it quite inspiring. Some of the most meta(-meta-heuristic search) code ever written! Now I'm working on a similar project and am looking for parts of AM's and Eurisko's algorithms which I can incorporate! I'll write something up if I have any success.

Eurisko is far more abstract than AM, but the data structures (structures with named 'slots' (members)) it uses makes the code vastly more readable (AM is pretty damn unreadable despite plenty of comments). Lenat himself said the better representations are what made Eurisko possible, unsurprisingly. If you do want to understand the implementation of AM in incredible detail you can read Lenat's PhD thesis. No detailed documentation of Eurisko exists. Also, both programs contain lots of code to print out what the system is currently doing, which is incredibly helpful for deciphering the code.

The landmark Deep Belief Networks (stacked RBMs) paper in Science was in 2006 [1]. DBNs were completely obsolete quite quickly, but don't deny the immense influence of this line of research. It has over 23k citations, and was my introduction to deep learning, for one. And cited by the Nobel committee.

You're completely incorrect to say RBMs were of theoretical interest only. They have had plenty of practical use in computer vision/image modelling up to at least a few years ago (I haven't followed them since). Remember the first generative models of human faces?

Edit: Wow, Hinton is still pushing forward the state of the art on RBMs for image modelling, and I am impressed with how much they've improved in the last ~5 years. Nowhere near diffusion models, sure, but "reasonably good". [2]

[1] G.E. Hinton and R. Salakhutdinov, 2006, Science. "Reducing the Dimensionality of Data with Neural Networks"

[2] "Gaussian-Bernoulli RBMs Without Tears" https://arxiv.org/pdf/2210.10318

That was my almost immediate reaction to the headline! Rarely credited but rarely forgotten.

Mira joined in 2018. OpenAI was founded in 2015.

The fact that OpenAI created and released o1 doesn't mean they won't also scale models upwards or don't think it's their best hope. There's been plenty said implying that they are.

I definitely agree that AGI isn't just a matter of scaling transformers, and also as you say that they "may not be the best" for such tasks. (Vanilla transformers are extremely inefficient.) But the really important point is that transformers can do things such as abstract, reason, form world models and theories of minds, etc, to a significant degree (a much greater degree than virtually anyone would have predicted 5-10 years ago), all learnt automatically. It shows these problems are actually tractable for connectionist machine learning, without a paradigm shift as you and many others allege. That is the part I disagree with. But more breakthroughs needed.

You forget that, at that time, OpenAI pushed SoTA LLMs massively forward by scaling them up ~3-4 orders of magnitude when others didn't think that would work or weren't willing to spend the money. But not just that. Following their example, Google and nVidia also attempted to scale up transformers but without really managing to push the SoTA.

So, I agree instead with meowface, and think it could even have been a 5+ year delay rather than 2 or 3. If you look at breakthroughs rather than incremental improvements, 5 years is not a long timescale. (And if OpenAI hadn't have made their breakthroughs, production of the highest-end GPUs/TPUs would be nowhere near where it is today.)

(I'm not attempting to justify OpenAI's structure or behaviour, just want to comment on one point.)

> That so many here are fans of screwing the world over for a buck makes this kind of comment completely unsurprising.

I'm very confused by your attitude towards that comment. Do you think that Mozilla's non-profit/for-profit split organisation was a bad idea?

Whoa. You need to write about how you've accomplished that. How long have you been working on it?

There's probably no good reason to put code and data on the same page, it's just one extra TLB entry to use two pages instead so the data page can be marked non-executable.

> I'm happy to talk about this all day!

Oh really :)

I'd like to ask how applications should change their memory allocation or usage patterns to maximise the benefit of THP. Do memory allocators (glibc mainly) need config tweaking to coalesce tiny mallocs into 2MB+ mmaps, will they just always do that automatically, do you need to use a custom pool allocator so you're doing large allocations, or are you never going to get the full benefit of huge tables without madvise/libhugetlbfs? And does this apply to Mac/Windows/*BSD at all?

[Edit: ouch, I see /sys/kernel/mm/transparent_hugepage/enabled is default set to 'madvise' on my system (Slackware) and as a result doing nearly nothing. But I saw it enabled in the past. Well that answers a lot of my questions: got to use madvise/libhugetlbfs.]

I read you also need to ensure ELF segments are properly aligned to get transparent huge pages for code/data.

Another question. From your link [2]:

> An application may mmap a large region but only touch 1 byte of it, in that case a 2M page might be allocated instead of a 4k page for no good.

Do the heuristics used by Linux THP (khugepaged) really allow completely ignoring whether pages have actually been page-faulted in or even initialised? Is a possibility unlikely to happen in practice?

Nice. I agree that LLMs will be game-changing for easing the writing of specifications/contracts for verified code; it could become the standard. Interesting that you're using Isabelle/HOL for code generation. I'm working towards auto-formalization of maths using LLM translation to an intermediate language between English and the proof assistant (for which I'm leaning towards Isabelle/HOL) and then a toolbox of techniques (i.e. ATP) for the remaining translation. Formalization is too painful at the moment. It's nearly the same problem as yours. But I'm starting out on simpler problems.

So I'm really interested in the "filling up" part, if you have anything to say about it.

Those assertions are a long way off absolutely true!

The undecidability of Turing machines in general doesn't say you can't reason about any programs, as you seem to claim, it says there exists programs which have unverifiable behaviour.

(Although the proof provides unnatural examples, in practice simple recursive mathematical functions such as the Collatz conjecture will probably always be beyond automated analysis.)

Of course software that was not written with verification in mind can be verified correct, provided it doesn't contains things such as an interpreter or complex conditional recursion. There are static analysis tools for analysing programs in languages such as C++ which make use of ATPs (automated theorem provers). For example, most of zlib has been proven correct using Coq. Yes, verification of the entirety of complex programs is in general impossible, but parts of them can still be verified. It's the most common use of ATPs, and an active area of research!

I'm guessing what they meant is that they use toolchains that are retargetable to other GPUs (and typically compile down to PTX (nVidia assembly language) on nVidia GPUs rather than go through CUDA source -- GCC and clang can both target PTX). For example XLA and most SYSCL toolchains support much more than nVidia.

> cheap piecewise affine approximation that is achieved by adding the bit representation of the floating point numbers together as integers ... with little to no performance impact

> we show that we can eliminate all multiplications in the entire training process, including operations in the forward pass, backward pass and optimizer update, demonstrating the first successful training of modern neural network architectures in a fully multiplication-free fashion.

This reaches demoscene levels of crazy/impressive!

Largely because they feared that large language models would be useful for large scale disinformation, election interference, impersonation, automated phishing, etc.

https://openai.com/index/better-language-models/ See "Policy Implications" and down.

Yes, there are many components to train noise.

I used to live by rails with a small incline, and the diesel locomotives made a loud low noise accelerating uphill from a station, while downhill the trains moved far faster and the engines were quite insignificant compared to noises due to speed: mainly the wheel noise and going over the expansion joints.

> I've saved a Welling paper from rejection from two reviewers who admitted to not knowing PDEs

Thank you for fighting the good fight.

This is why I love OpenReview, I can spot and ignore nonsensical reviewer criticisms and ratings and look for the insightful comments and rebuttals. Many reviewers do put in a lot of very valuable work reading and critiquing most of which would go to waste if not made public.

Yes, attention (in transformer decoders) looks backwards to internal state at previous tokens. (In transform encoders like in BERT it can also look forwards.) When they said "stateless" I think they meant that you can recompute the state from the tokens, so the state can be discarded at any time: the internal state is entirely deterministic, it's only the selection of output tokens that involves random sampling. What's also a critical feature of transformers is that you can compute the state at layer N for all tokens in parallel, because it depends only on layer N-1 for the current and all previous tokens, not on layer N for the previous token as in LSTMs or typical RNNs. The whole point of the transformer architecture is to allow that parallel compute, at the cost of directly depending on every previous token rather than just the last.

So if you wished you could implement a transformer by recomputing everything on every token. That would be incredibly inefficient. However, if you're continuing a conversation with an LLM you likely would recompute all the state for all tokens on each new user input, because the alternative is to store all that state in memory until the user gets back to you again a minute later. If you have too many simultaneous users you won't have enough VRAM for that. (In some cases moving it out of VRAM temporarily might be practical.)