Hacker News Reader

Comment by wokwokwok | Hacker News Reader

There is literally no alternative.

You’re stuck with openai, and you’re stuck with whatever rules, limitations or changes they give you.

There are other models, but specifically if you’re actively using gpt-4 and find gpt-3.5 to be below the quality you require…

Too bad. You’re out of luck.

Wait for better open source models or wait patiently for someone to release a meaningful competitor, or wait for openai to release a better version.

That’s it. Right now, there’s no one else letting people have access to their models which are equivalent to gpt-4.

View on HN

This point is understated. So many people are going around like "I'm building an AI app!" when the reality is, OpenAI built an AI app, you're just designing a front end for it.

OpenAI builds an LLM and an api-interface to that model.

The design of abstractions, prompt engineering, custom fine-tunes and software engineering required to ship a valuable application on top of that interface counts as "building an app" in my book.

The LLM _is_ the product. Everything else in the stack is window dressing.

I don’t think OpenAI’s contribution should be so understated- they built a technology that was considered science fiction just a few years ago. They deserve all the credit for the “AI”.

Good luck building a moat when 95% of your app is just calling the API everyone else has access to.

Paul English (founder of Kayak, exited at $2 billion) said this in response to the exact comment you are making: "Kayak was just a thin layer above ITA, and ITA sold for $700m and Kayak sold for $2b. So don’t dismiss thin layers. The ultimate power is the app and UX."[0]

I would also add "distribution" to the "app and UX" part, but you can certainly build a valuable business upon an API "everyone else has access to" - plenty of companies out there that do that.

[0]https://twitter.com/englishpaulm/status/1623701758781558784

I can appreciate this, but there's a reason I said 95%. The stuff I see being built so far is really an extremely thin veneer on top of the API, but that could change with time.

Kayak would never have succeeded today. Competition is far too hungry

100% of all existing code is just calling an OS-API that everyone else has access to. That's just nonsense

Anyone can build an OS without needing billions of dollars, and in fact there are state of the art open source OSes. Not quite the same.

Yes, but also, come on now.

I mean, it refutes the argument. Everyone has access to the internet, to conv nets, to C/python/pandas/TCP, you name it. Yet, nobody would seriously argue that one can not build a moat with products based on those stacks for the reason that everyone else has access to the same stack. Its just not an argument, at all.

We can now write software that interprets language under the hood (to some degree). The value propositions enabled by this change in the world are so vast, and partly so complex - to make absolute statements like "yeah but you don't control the model, so anyone can copy your solution" seems out of touch to me. What subset of technology doesn't get reverse engineered? Either this applies almost nowhere (because every piece of tech that an engineer can get their hands on is effectively open), or everywhere.

Moat does not come from compile time but runtime. The company, the operation, and the accumulated data, the brand, the trust and the reputation.

There are technology moats as well, but for the most part, you're correct. Everything else usually matters much more.

by this logic, you can just copy facebook, twitter and pretty all software as they are just calling some high level api (http protocol , ios/android api..) etc .

there are plenty of FB, twitter and reddit alternative, they just don't have the data of the original. but openAI provide the API and the data

more like utilizing prompts that anyone can figure out

what does this moat even mean in this context ? I see it throw around but cannot infer the meaning

I think that people who believe this don't realise how complex and nuanced code surrounding LLMs have gotten in the last half a year.

I wasn't trying to understate OpenAI's contribution, far from it. I can hardly express my appreciation for their work.

That doesn't mean that everything else in the stack is window dressing though - custom, domain specific wrangling with the different api endpoints, finding a satisfying prompt, temperature param etc. for specific tasks - the entire process of designing systems around an LLM-api has many intricacies to it, lots of which are completely uncharted territory.

I can assure you: very smart people are knees deep in this process, and they deserve the credit for their share of the value that is being created.

I am not convinced it won’t all be obviated with improvements in the foundation model. Seems to be carving out a very very temporary space for yourself.

Re: smart people avoiding wrapper work - sure! Not just because the result might be short lived, but also because there is a lot more prestige in building something from scratch rather than wrapping.

However, the "very very temporary space" might as well lead to momentum and a moat in a subdomain, and anyone who met a sufficiently large number of smart people knows that lots of them are very pragmatic, don't chase prestige, and enjoy laying ground work for future iterations.

> The engine _is_ the product. Everything else in the car is window dressing.

Interesting analogy, since the engine manufacturer also manufactures the car.

> required to ship a valuable application on top of that interface counts as "building an app" in my book.

It's certainly building an app. It's not building an AI app, though. It's building a front-end to an existing AI application.

how can you finetune gpt-4 or 3.5? The shell/wrapper around a request to open AI is not any worthy piece of software, It'd be more impressive to vend those services. Otherwise it's basically just a few scripts.

Google has recently announced infrastructure for fine tuning and RLHF on their pre trained foundation models. I imagine all cloud providers will follow suit.

https://cloud.google.com/vertex-ai/docs/generative-ai/models...

As your wording implies, finetuning is restricted to the smaller models, i.e. babbage, curie etc.

You can generate the training data for this with 3.5 and 4 and tune smaller models with the resulting data. For lots of tasks, this results in robust results, which btw are also faster than 3.5 turbo.

It's just like building a Facebook app back in the day

Replace social media / graph APIs with the ones from OpenAI

while this is technically correct, difference is that your product won't survive without openai at this point. If you need the model quality openai provides you are stuck and your product can just disappear. Because llm is core building block, irreplaceable one.

Building a product that relies 100% on a single external vendor is taking a huge risk. So many companies have been burned by this in the past that it's amazing anyone doesn't see it as a risky thing.

So I have to believe that the people making these products are intending to make as much cash as possible up front and aren't aiming for a long-term thing.

Use the facade pattern so it’s easier to swap out if needed.

Of course it is an app, but ultimately it is just a frontend.

There is a lot of engineering work and reasoning to make sufficiently complex prompts.

Depending on your business case, the 4096 tokens given to you have to go quite far. Vector embeddings are not "easy" to work with. Trying to splat together a range of techniques to craft a good prompt is hard™.

Adding in Actions (e.g. using headless browsers to open pages etc) is also pioneering territory.

Sucks that OpenAI currently has the market, but there is still plenty of reasons to develop on top of it.

Wait so when I'm using postgres I'm just designing a front end for it? Maybe I'm just designing a front end for my CPU?

So, ChatGPT isn't a technology, it is a service.

That was never in question. ChatGPT is openAIs web frontend for several of their GPT-Models.

The technology is just GPT and transformers, the open source alternatives are just not as advanced as the GPT4 model yet. It might change with current trajectory, mainly because OpenAI keeps nerfing it.

The technology is in the hardware/software synergy.

Anthropic's Claude is on par with or better than GPT-4 for many tasks. It's not self-hosted, but it is competitive with OpenAI's best model.

What tasks do you use Claude for?

I tend to use Claude primary for creative writing (at which it's better than GPT4 and even Claude+, in my experience), and for explanations, which are more thorough than GPT4 or ChatGPT (GPT 3.5) without any special prompting.

Claude should be more well known, imo. ChatGPT/GPT4 gets all the hype, but Claude is really good too... sometimes even better.

I thought Anthropic has Claude and Claude Instant. When you say Claude sometimes feels better than Claude+, do you mean that you feel Claude Instant sometimes gives better result than Claude?

This is what I referred to as Claude: https://poe.com/Claude-instant

And this is Claude+: https://poe.com/Claude%2B

it would be more known if they would provide an interface as easy to use as chatgpt.

Guess there really is a moat.

Just to nitpick that a bit, they have a lead. To me, a moat is some other axis of advantage that makes it hard for a rival to compete directly.

So for example Tesla has/had a lead in EV tech, but their supercharger network is a moat because it’s very difficult for a rival to compete with, even if they have equivalent EV tech.

I know there isn’t established terminology for this. It is a nitpick, but I think a lead is already a term we have, and moat has connotations of being in some way ‘unfair’. To me, just creating a better product isn’t unfair in any meaningful way, while moats in some circumstances can represent anti trust issues.

So like, would that not suggest that Google has a moat with AI, as it is already intertwined with our email, calendars, etc... (if they ever make their AI public)

It makes it all the more interesting that Google is currently losing to an upstart. They have a MASSIVE advantage in data and moats, but cannot execute on novel products to take advantage of it, despite also hiring a significant chunk of the talent in the market as well.

But perhaps at this stage Google's not meant to / need to build cutting edge products, but rather focus on commoditizing the ones that prove profitable.

> t makes it all the more interesting that Google is currently losing to an upstart

How is Google "losing" when they are not even in the same market? Are you expecting Google to develop and sell access to an LLM API or use AI internally to enhance it's other products (which it jas been doing to great effect)

or maybe they like their chinese wall between their internal AI and the public at large!!!

Yes, no one can build a car from scratch in their garage, but there are plenty of companies with enough capital to build their own cars and sell them to people.

OpenAI has a lead on Bard, LLaMA and friends, but I would expect that to close in the next few months or years.

It's not that the isn't a moat, but rather that it isn't looking particularly durable

Compute moat. aka cash moat. It's a deep one, too.

Company moats can grow and later shrink.

There is no AI, there are LLM models, that people hardly understand. Modern solutions are to AI what Alchemy was to Chemistry. Sometimes they get some results but nobody really sure why. The little that OpenAI mentioned about how they achieved GPT-4 performance, says it was by trying a bunch of things and dismissing others. They probably also themselves not sure how they managed to pull it of, meaning that, if for example an increase in parameter size or other settings, kick in negative effects that actually reduce the model performance, they will also be at a loss where to go next.

Is this opinion based on some benchmarking you (or someone else) did?

Nothing that you can self host seems to come close to gpt 3.5, let alone gpt-4. r/LocalLlama is good subreddit to lurk to get a pulse on the local llms. Current leader seems to be Guanaco-65B.

I believe there are benchmarks, but I can informally second that opinion. I'm building a writing app (chiseleditor.com) and there is nothing as good as the ChatGPT models right now.

Since you have your hands in the mess, let me ask you this, and I ask, because I think this is what is meant by people who ask what's a.. bla..bla alternative.. to bla..bla..bla. How can an industry specific or company specific AI be created? meaning you take the LLM engine and you ingress company data.. or if you want to be bold, industry datasets. CHATGPT is marketted as being doctor/architect/lawyer/professor/etc. But what if all you want to do is build an ask jeeve's type of ai lawyer??

I would distrust the currently available benchmarks, as recent research (gah, can't remember the paper title) indicates that for many benchmarks at least some of the data splits have leaked into model training data; and there's some experience with the open source models which match an OpenAI model on the benchmark scores but subjectively feel much worse than that model on random questions.

I’m telling you from looking at this closely that there is substantial evidence solely from new, never seen before prompts that GPT4 is by far the best, ChatGPT/Claude is second, with other anthropic, Vicuña, etc bringing up the rear

Have you tried Anthropic, specifically Claude? I have no doubt GPT-4 is still king, I'm just curious how much of a lead it has.

I've played around a lot with Claude, and find it much better than GPT4 and even Claude+ at creative writing.

I also generally prefer Claude or Claude+ over GPT4 or ChatGPT (GPT 3.5) for explanations too, which tend to be more thorough without any special prompting.

No, I would love to try it out but unfortunately I don't have early access yet.

It seems reasonable for what many who are using openai and self hosting are finding.

There’s a gap, it’s closing, likely faster than anticipated.

Huggingface awaits :)

The model is the spice then basically?

Fitting analogy. The Kenyan data transcribers that OpenAI used are the Fremen, and transcripts of erotic Harry Potter fanfiction are the Shai-Hulud.

Are the kenyan's really going to rise up like the Freman? Most likely not. If anything, they'll be used for cheap labor when the next new thing comes along.

haha was thinking the same. The other day openAI's api hiccup gave me a small panic attack

Techies repeat what they chastise artists for about Adobe

Reply on news.ycombinator.com