This is due to the tokenization scheme. These LLMs can’t see individual characters, they see chunks of characters that are glued together to form tokens.
It’s impressive that they understand individual-character-level questions as well as they do.
6,984 karma · joined January 26, 2013
It’s impressive that they understand individual-character-level questions as well as they do.
Have you tried getting your formatted JSON out via the new Functions API? I does cure a lot of the deficiencies in 3.5-turbo.
gpt-3.5-turbo is the model behind ChatGPT. It's chat-fine-tuned which makes it very hard to use for use-cases where you really just want it to obey/complete without any "chatty" verbiage.
The "davinci-003" model was the last instruction tuned model, but is 10x more expensive than gpt-3.5-turbo, so it makes economical sense to hack gpt-3.5-turbo to your use case even if it is hugely wasteful from a tokens point of view.
You can do flows like:
- Upload CSV of stock data
- "Graph the closing price each day"
- "On the other axis, add a line showing volatility"
- "Filter out to just days with above-average volatility"
etc
Pretty useful for analyzing raw data.
For instance, one could imagine an RNN that learned to do some approximation of tree search for game playing Chess and Go. But we have very good reason to think that tree search is basically exactly what you want, so even systems like AlphaGo have the tree search implemented outside the neural net, but still using a learned system to heuristically guide the tree search.
The reference to the bitter lesson here is that feature engineering has, thus far, typically lost out to more general end-to-end methods in the long run.
This paper tries to do feature engineering by hand-coding an exponentially decaying mechanism, where tokens further in the past are assumed to be less important.
My comment is that this type of hand-engineering will lose out to methods that are more end-to-end learned. These methods do not necessarily need to be hugely computationally intensive ("buy more GPUs").
That said, I could see it being the case that in the short term, we do just buy more GPUs, learn a general end-to-end algorithm, but eventually figure out how to re-implement that end-to-end learned algorithm in code significantly more efficiently.
When you abandon O(N^2) attention, you are forced to start adding heuristics to choose what to correlate. Any time you see one of those giant context window LLMs, you need to be asking what heuristics they added, what is getting correlated, and what is not getting correlated.
This paper chooses an exponential heuristic where tokens further in the past get exponentially less attention. This heuristic is fine for certain tasks like responding in a chat room, where the most recent tokens are the most important, but bad for tasks where tokens are roughly equally important throughout the text, such as a dense academic paper or a reference manual.
The bitter lesson [1] is going to eventually come for all of these. Eventually we'll figure out how to machine-learn the heuristic rather than hard code it. Recurrent neural networks (RNNs) do this implicitly, but we don't yet know how to effectively train RNNs on ultra-deep sequences.
Another possibility is learning a heuristic for non-recurrent LLMs via reinforcement learning, such as in [2], which is basically a reinforcement learned "auto-researcher" that was trained in a style reminiscent of AlphaGo.
[1] http://www.incompleteideas.net/IncIdeas/BitterLesson.html
https://en.wikipedia.org/wiki/Claude_%22Bud%22_Lewis_Carlsba...
https://www.water-technology.net/projects/sorek-desalination...
50 cents per day for a fully desalinated water supply is... incredibly cheap.
If you're interested in water policy and water management / engineering, I cannot recommend enough reading the book "Let There Be Water: Israel's Solution for a Water-Starved World".
Consider a simpler case: a small neural network that takes 2 numbers and adds them together, producing 1 number as output.
This network is very obviously feedforward, and probably very tiny with few layers.
Say I have a list of numbers [1, 2, 5] that I want to sum. If I send 1 and 2 through the network, get 3 as a result, then I send 3 and 5 through the network, and get a final answer of 8, my network has not suddenly become non-feedforward just because I fed the output back into it.
The key distinguishing factor between feedforward and non-feedforward is if the network itself loops back around and, at training time, it learns how to make use of this ability to pass data to itself to maintain some hidden context between passes.
There is no such learned hidden context in my addition example, and none in GPT.
---
There actually is a tiny caveat here: the RL fine-tuning process OpenAI has done on its models ("RLHF" & friends) actually does allow for a very, very small amount of information leakage between passes because you are rewarding whole responses, so the model can learn little patterns of what tokens in the beginning of the response led to certain tokens at the end of the response and reinforce those patterns.
The model could learn to encode small bits of "hidden" information in particular token choices that the human raters wouldn't notice. In this case, there is a (small but non-zero) amount of learned hidden context. But this is not what Hofstadter is talking about -- the non-RLHF'd base model is just as intelligent, just harder to use.
Using https://github.com/awslabs/shuttle which works on our real Rust code.
Recurrent neural networks have the recursion as part of the training regime. GPT only has auto-regressive "recursion" as part of the inference runtime regime.
I think Hofstadter is surprised that you can appear so intelligent without any recursion in the learning/training regime, with the added implication that you can appear so intelligent with a fixed amount of computation per word.
Go look at the GPT training code, here is the exact line: https://github.com/karpathy/nanoGPT/blob/master/train.py#L12...
The model is only trained to predict the next token. The training regime is purely next-token prediction. There is no loopiness whatsoever here, strange or ordinary.
Just because you take that feedforward neural network and wrap it in a loop to feed it its own output does not change the architecture of the neural net itself. The neural network was trained in one direction and runs in one direction. Hofstadter is surprised that such an architecture yields something that looks like intelligence.
He specifically used the correct term "feedforward" to constrast with recurrent neural networks, which GPT is not: https://en.wikipedia.org/wiki/Feedforward_neural_network
Definition of Feedforward (from wiki):
``` A feedforward neural network (FNN) is an artificial neural network wherein connections between the nodes do not form a cycle.[1] As such, it is different from its descendant: recurrent neural networks. ```
Hofstadter expected any intelligent neural network would need to be recurrent, ie looping back on itself (in the vein of his book “I am a strange loop”).
GPT is not recurrent. It takes in some text input, does a fixed amount of computation in 1 pass through the network, then outputs the next word. He is surprised it doesn’t need to loop for an arbitrary amount of time to “think about” what to say.
Being put into an auto-regressive system (where the N-th word it generates gets appended to the prompt that gets sent back into the network to generate the N+1th word) doesn’t make the neural network itself not Feedforward.
We live in a democracy and it's up to a majority to decide what to do with the public commons. It is telling, to me, that no state that has banned billboards (Alaska, Hawaii, Vermont, Maine) has any serious political movement from any part of the spectrum to reverse the bans.
Advertising in public places is privatizing the commons, and, taken to the extreme, is negative-sum.
Consider this billboard on a boat that ruins the view at a public beach: https://www.reddit.com/r/ABoringDystopia/comments/i939xz/thi...
Yes, there might be small positive economic benefit to the one person on the beach who actually needed the advertised product, but a huge, diffuse negative externality to everyone else.
Most public advertising is not this bad, but if you drive on the highway in a state that has banned billboards, such as Vermont, and then you cross the border into a state that allows them, you notice it, and not in a good way. We'd be collectively better off if they were banned everywhere.
Allow advertising in private spaces, but ban in public.
An LLM is "just" a document completer. Given text, it predicts the following text.
So there is nothing stopping you from bolting on a system which works like:
- Given the current streaming input, continually update a pool of N likely continuations (predict what the user will say)
- For each of those N likely continuations, pre-generate responses
- If the actual user continuation matches one of the continuations, use your pre-generated response
This is trading off compute for latency, as you won't use at least N-1 of those pre-generated responses.
Eleven Labs is the first voice synthesis that is good enough that I'd listen to an audiobook generated from it, but pricing is such that it would cost $100 to synthesize a 10 hour audiobook. A little too expensive. If they could get it down to $10 I'd cancel my Audible subscription and just synthesize audio from ebook text.
So if I can get a locally running voicebox model and just leave it running on my laptop over night transcribing an audiobook, that's even better.
The Windows snapping UI is much more intuitive, though.