Me1000

This is the real value here. Keeping a secure environment to run untrusted code along side user data is a real liability for them. It's not their core competency either, so they can just lean on browser sandboxing and not worry about it.

They explain how it's calculated, you just have to trust their calculations are correct.

Ah, thank you!

Relatedly, what does "parallel" function calling mean in this context?

Or it means that the existing models are too large to be profitable even at scale.

I think there is valid criticism of google for inventing a cool technology only to have the rest of the industry discover its usefulness before them. But to say Gemini 1.0 or OG Gemma aren't first generation models because BERT and flan existed before is like saying the iPad wasn't a first generation device because Apple made the Newton. Like sure, they're the same in that they're transformers trained on language and text, but these are new families of models. The training mechanisms are different, their architectures are different, the data sets are different, the intended purpose of the models are completely different, etc. At some point I guess it's a semantic difference, maybe.

>long time ago

This is an incredible statement to make about a field that no one was talking about 24 months ago, a family of SOTA models that didn't exist until 8 months ago, and a family of small local models that didn't exist 6 months ago. But sure, give up hope after the first generation of a model family doesn't impress you.

People seem to forget how incredibly early we are in this whole thing. The fact that so much progress has been made in such a short amount of time should make everyone super excited!

I also use BART most of the time, but I often end up heading back to SF pretty late in the night so that’s not always an option.

Unfortunately Waymo still wont operate on freeways or at SFO. But I long for the day when I can take a Waymo between SF and Berkley or to SFO, those are my most frequent reasons to call a Lyft.

> Rides have usually 10% mark up over Lyft and Uber.

I found that to be true as well, but when you factor in the tip to the driver they come out to more or less the same price.

“Just” is doing a lot of heavy lifting here. It’s so frustrating watching people think it’s just because a bunch of smart and hard working people are either lazy or stubborn.

Many social media platforms exacerbate and amplify some of the worst qualities of humans. Bullying is taken to an extreme, people are often engaging with others in bad faith, and other times you're not even engaging with real people at all.

People are much more likely to say mean things to another when they're behind a keyboard and talking to a screen name or avatar. And real life conversations are much harder to manipulate as compared to social media. Add to that that these platforms monetize by keeping your attention, so there is an addictive component to them. Quitting social media is often very hard, psychologically.

You're correct that many of the problems with social media are not unique to the technology, but like a lot of technology it often times makes existing problems worse. There are strong correlations between certain kinds of social media content and the development of eating disorders and self harm, especially among young people. And we have the ability to start mitigating those problems today, but "fixing" human behavior is going to take a little longer.

Exactly. People forget (or simply don't know) that orbital plane changes are incredibly expensive from an energy perspective. It's not like you're walking down the road picking up trash in a row, you have to adjust your orbit, canceling out movement in one direction while adding movement in another. And it takes an incredible amount of energy even for minor plane changes.

They prompt you before they send your data to OpenAI, but it's clear that they prompt you before they send it to Apple's servers (maybe they do and I missed it?). And their promise that their servers are secure because it's all written in Swift is laughable.

Edit:

This line from the keynote is also suspect: "And just like your iPhone , independent experts can inspect the code that runs on the servers to verify this privacy promise.".

First off, do "independent experts" actually have access to closed source iOS code? If so we already have evidence that this is sufficient (https://www.macrumors.com/2024/05/15/ios-17-5-bug-deleted-ph...).

The actual standard for privacy and security is open source software, anything short of that is just marketing buzz. Every company has an incentive to not leak data, but data leaks still happen.

This arguments is made every singe time a new LLM article gets posted and I'm not sure it's really adding anything to the conversation. Everyone understands language models are not human, there's no need to add a philosophical argument because you don't like the word choice.

"Characters" in any other context are also by definition not curious. They're not open-minded or thoughtful either. Characters in a book are not people, they don't have thoughts, they are whatever the author put on paper. Yet we still use these words to describe them, as if the characters' consciousness and motivations existed beyond the paper on which they're described. It becomes really hard to talk about these traits without using words like curiosity and thoughtfulness. No one thinks the characters in a fictional book are real people.

Completely unrelated.

He actually was a joy to work with. George is an awesome human.

This is an opt-in feature, so by defaut all AI-related features are disabled. You have to go out of your way to generate an API key and add it, it's not even a thing you could accidentally turn on. And if you'd like to use the features without sending your prompts to OpenAI, under the advanced settings you can supply any OpenAI compatible API URL (if you're running an LLM locally for privacy reasons for example).

Thanks to most of the world being "GPU poor", there is a lot of research and engineering effort going into making models much more compute efficient. Another way that OpenAI gets to benefit from the world of open source/weight models.

LLMs are weird, they hype cycle has caused a lot of people to want to hate on them. A tool like any other can be misused or abused, and smart people like to demonstrate their ability to break the tool and show the world so that people can see how smart they are.

I find it quite strange, but I've found myself mostly ignoring the naysayers because they're not contributing anything new to the conversation. There are plenty of people out there posting novel ways of using LLMs and finding novel ways where they don't work. I'd rather focus on the people doing interesting work rather than the people parroting thing that have already been said dozens of times.

And Gemini.

Ollama doesn't have their own inference engine, they just wrap llama.cpp. But yes, it will be awesome when it's more generally available.

WizardLM-2 8x22b (which was a fine tune of the Mixtral 8x22b base model) at 4bit was only 80GB.

You need to update ollama to 0.1.32.

I ended up having to download the latest version directly from GitHub, and that fixed it. Looks like the 0.1.32 mac release hasn't been posted to your website yet.

The 4bit quant doesn't seem to work for me, I keep getting: Error: exception create_tensor: tensor 'blk.0.ffn_gate.0.weight' not found

I've tried downloading it twice now.

For the RAG example, I don’t think it’s the prompt so much. Or if it is, I’ve yet to find a way to get GPT4 to ever extrapolate well beyond the original source text. In other words, I think GPT4 was likely trained to ground the outputs on a provided input.

But yeah, you’re right, it’s hard to know for sure. And of course all of these tests are just “vibes”.

Another example of where Claude seems better than GPT4 is code generation. In particular GPT4 has a tendency to get “lazy” and do a lot of “… the rest of the implementation here” whereas Claude I’ve found is fine writing longer code responses.

I know the parent comment suggest it likes to make up packages that don’t exist, but I can’t speak to that. I usually like to ask LLMs to generate self contained functions/classes. I can also say that anecdotally I’ve seen other people online comment that they think Claude “works harder” (as in writes longer code blocks). Take that for what it’s worth.

But overall you’re right, if you get used to the way one LLM works well for you, it can often be frustrating when a different LLM responds differently.

Interesting, Claude 3 Opus has been better than GPT4 for me. Mostly in that I find it does a better (and more importantly, more thorough) job of explaining things to me. For coding tasks (I'm not asking it to write code, but instead to explain topics/code/etc to me) I've found it tends to give much more nuanced answers. When I give it long text to converse about, I find Claude Opus tends to have a much deeper understanding of the content it's given, where GPT4 tends to just summarize the text at hand, whereas Claude tends to be able to extrapolate better.

Mixtral 7x8b was way better than llama2 70b and used less RAM and compute at the same time. This model is way better than llama.

In fact I would go as far as saying llama2 isn’t that good compared to some of the most recent models.

It’s absolutely beneficial when training because the forward pass and back propagation is still only on the neurons that were activated.

The Mistral guys specifically mention that training speed (due to not needing as much compute) was one of the reasons Mixtral was released so soon after Mistal 7b.