He has the ingredients of being able to keep LLMs in perspective (is there really an “I” or just the illusion of one?), but he doesn’t understand what the computer is doing; it’s not “mechanical” enough for him.
3,476 karma · joined April 10, 2007
He has the ingredients of being able to keep LLMs in perspective (is there really an “I” or just the illusion of one?), but he doesn’t understand what the computer is doing; it’s not “mechanical” enough for him.
While technically, SAT has exponential algorithmic complexity, in practice, SAT solvers can solve problems with thousands of boolean variables quite quickly. “Guess and check” on these same problems would take the lifetime of the universe. CPU vs GPU does not remotely cover the difference.
Basically, I have no idea what this author is going on about, this could be a joke for all I know.
He doesn’t side with Steve or find Steve’s statement to be historically informed, but there are some similarities (and differences) to note if you compare moderators to lords and Reddit/Steve to a king.
I think in this case it isn’t the axioms but the “style” of philosophical discourse that requires the reader to take the message with a big grain of salt. Like when a YouTuber says, “In the next ten minutes, I’m going to lay out the exact formula for how to write a perfect screenplay,” it’s not literally true. Tolstoy’s musings are a little deeper than your average YouTuber, on par with a good one, similarly overblown and oversimplified, and more moralizing.
When I criticize Google, or some other company, I am not forgetting that they are motivated by making money.
WebSockets also don’t ensure messages are delivered (once) across network hiccups. And browsers have different limits on how many there can be open at once across tabs for a domain (IIRC). In the end, I think it makes sense for them to be the main transport for a real-time app, but they are just one part of creating a conceptually simple model for real-time communication between client and server.
Does Jamsocket have or use a client socket library?
I'm not an expert in the field, but, there are lots of previous algorithms for predicting the next token in a series (Markov chains, autocomplete). None of them felt so much pressure to make an accurate prediction that they had no alternative but to teach themselves arithmetic! It seems what is different about LLMs (as far as the post goes) is that we can anthropomorphize them.
More seriously, I guess I just feel like a meaningful sketch of an explanation for why algorithm X (where X is LLMs in this case) for continuing a piece of text is good at problem A should involve something about X and A. Because it is clearly highly dependent on the exact values of X and A, not just whether A can be posed as a text completion problem and humans would prefer the computer learn to solve the underlying problem to produce better text. For example, it could help to imagine a mechanism by which algorithm X could solve problem A. The closest thing to a mechanism (something algorithm X, i.e. LLMs, might be doing that's special) in the post is the talk of necessity being the mother of invention and "a deeper understanding of reality simplifies next-token prediction tasks," and the suggestion that if you were an LLM you might want to use "the rules of addition."
It's true that modeling arithmetic in some way could help a LLM account for known arithmetic problems in the training data, which could help it on unseen arithmetic problems, but what problems an LLM can solve is a function of what it can model. Anything an LLM can't model or can't do, it just doesn't. LLMs are really bad at chess, for example. The patterns of digits in addition may be similar enough to the hierarchical patterns in language the LLM is modeling. But it's not clear if the LLM is using the "rules of addition" or not. As far as I know, we don't actually understand why LLMs are able to store so much factual information, produce such coherent stories, and do the specific things they can do.
I don’t feel like the points made here align with any insight about the workings of LLMs. The fact that, as a human, I “wouldn’t know where to start” when asked to add two numbers without doing any addition doesn’t apply to computers (running predictive models). They would start with statistics over lots of similar examples in the training data. It’s still remarkable LLMs do so well on these problems, while at the same time doing somewhat poorly because they can’t do arithmetic!
It’s like scratching a design onto the bottom of an audio CD, playing it, and if it works on your CD player, shipping it. “Works for me”
I have a positive impression of fly.io overall, I mean they admitted they are still figuring out how to build tech at the level of their ambition and have it be reliable enough, but I might like to be a customer one day.
It's more like a way to counteract lots of social conditioning.
He could just get coffee with someone who uses AirBnB a lot and they could probably tell him exactly what sucks about it. Maybe even one of his own employees. Lots of engineers use AirBnB, and the AirBnB app apparently has a thousand engineers working on it.
What sucks is, for example, being good at technical design but that not being valued. And management being disconnected from employees, which is not unique to software companies.
I'm not entirely convinced I have some special thing that is different from what you would expect from a system that can reflect on itself in lots of different ways at different levels.
There could be other uses.
Do not underestimate the sheer amount of walking around and aiming your phone that this technique requires. A 5m range is not much, and in some cases the effective range seems to be less than that. If you just want a to-scale image of an area, you can stop after "Capturing the model" and take a screenshot of your captured scan, but it's still a lot of work.
These companies are selecting and amplifying certain speech. The fact that the process is somewhat automated doesn't change this. If any legal argument relies on "the algorithm" being likened to simple searching and sorting, maybe we should require some transparency there so anyone can know if that's actually the case.
On another note, Section 230 isn't a free pass to host illegal content. It doesn't make a website immune to DMCA takedown requests, for example. We could have terrorism takedown requests, and the burden wouldn't harm large, irresponsible companies or small, moderated forums.
I’m working on a language that will be sort of a cleaner TypeScript, more simple and sound, with powerful static types, but not ML.
I think functions taking not just read-only views of data structures but deeply “const,” unchanging views is really important. Code that runs in GCed runtimes like JS engines or the JVM can be quite performant, but not with the level of copying and allocation that immutable data structures typically require (when you need to model them changing over time), and I want to solve that at the type level.
It’s like confusing a photo of a person with a person, and a photo generator with a human cloning machine, and saying, “But when I look in the mirror, that’s me! So I am equal to my visual reflection.” Language is the I/O of a cognition process that is itself only a small part of being human and alive.
A lot of activities and jobs do use humans as “cogs” in a machine, and in some cases an AI might make a better cog, but I think some commenters do underestimate the amount of context humans bring to various tasks. You don’t have to analyze the humans to see how complex the jobs are, just try to do the jobs by AI, or watch as others try. Like try to make a self-driving car; it’s hard. Try to replace a cook at Denny’s with a robot. Or a lawyer. See how long the tail of edge cases is.