Hacker News Reader

Comment by tkiolp4 | Hacker News Reader

I don’t understand these new saas that rely on GPT. It surely is my lack of understanding, but since the interface to GPT is mainly text: why any company would pay for N different GPT saas (e.g., one for searching for code, another for searching for documentation on confluence, another for allowing your business analysts with their queries, etc.)?

Wouldn’t it make more sense for any company to have only ONE interface to use GPT that is wired with some (or all) parts of the company digital assets? Text is already flexible enough to allow such a universal interface.

View on HN

To me, your question seems to boil down to the economics of vertical integration.

Two thoughts:

First, GPT is all marginal cost, no fixed cost. So the marginal economics of 1 super app vs 10 specialized mini apps is roughly the same.

Second, imagine the same argument applied to a utility like electricity. "Why would any company pay for lightbulbs from one company, HVAC from a second company, and appliances from a third company?"

Early on, electricity was complicated and you might actually purchase all of these from the Edison Illuminating Company. But there are returns to specialization, and today the winning formula is different companies that specialize in each. The company best at manufacturing lightbulbs is not necessarily the company that is best at manufacturing washing machines.

Similarly, you could ask: "Why would any company pay for a laptop from one company, an operating system from a second company, and office apps from a third company?"

In some cases it may make sense to vertically integrate (e.g., Apple is happy to sell you a combined laptop + OS + Pages/Numbers), but in many cases the specialized players still do fine (e.g., you might buy a Dell laptop, a Microsoft OS, and Notion/Sheets).

So I think it's very much an open question as to whether the winning approach will be a Swiss army knife (single product) or a toolkit (multiple products).

> Second, imagine the same argument applied to a utility like electricity.

Isn’t that how electricity works? You purchase electricity directly from the producer, and buy additional plugins (literally) that work with the single source of electricity you’re purchasing.

That would be akin to giving each of the 'plugins' your GPT API key and pay the usage bill directly to OpenAI. The plugin developers would still want $$$ for the app even if the GPT usage was excluded though.

Wouldn't it be a one-time purchase then? "Buy my prompt for $10 that enables you to search your codebase"

And we're back to Prompt Engineers :)

I think an IDE plugin or standalone app takes slightly more work than writing a prompt, but sure

Yes, prompt engineering is the future.

I think op means that a more general tool would cover all use cases. If there was a way to easily upload my code to the LLM and query it. At that point the service is providing a way to get your code to the LLM and a prompt, neither of which will be especially difficult in the future.

pretty quickly llvm/ai support is going to be more on the level of electricity itself. like a utility.

Have you tried just asking gpt4 to write a script that then implements the SAAS as a script that then calls gpt4? You should be careful about analogies, intelligence is special.

Slight nitpicking:

Edison used lots of different companies for their own business cases and I think the Edison Illuminating Company only built electric generators. So indeed in the beginning of electricity, when the field was most untested/starting up, you did shop at special companies for a generator or lamps or whatever instead of using an integrated whole-in-one solution of a mega corp! Wikipedia:

> Edison Lamp Company, a lamp manufacturer in East Newark, New Jersey; Edison Machine Works, a manufacturer of dynamos and large electric motors in Schenectady, New York; Bergmann & Company, a manufacturer of electric lighting fixtures, sockets, and other electric lighting devices; and Edison Electric Light Company, the patent-holding company and financial arm for Edison's lighting experiments

These were only later consolidated into General Electric.

The point is that the interface to Bloop (or whatever 3rd party SaaS product) is just plain text. In programming parlance I would call this a facade [1] on top of GPT-4. The benefit is that you, the end user, can just continue typing things like "Where do we create our database pool?" in a search box, but under the covers Bloop can use GPT-4, but then eventually upgrade to GPT-5, GPT-6, or some other much better currently-super secret competitor version.

1. https://en.wikipedia.org/wiki/Facade_pattern

Agreed that may be the case but all OpenAI needs to do is release a nice UI for GPT and they've suddenly pushed these SaaS out of the market. Why fund miners over the companies making pickaxes?

That takes time and money on OpenAI's behalf. Alternatively they can aquire Bloop after Bloop does all the leg work of integrating their niche with chat GPT. But I agree its a tenuous place to be in as an investor / employee / cofounder of a startup that is at OpenAI's mercy. This model of relying on another vendor's product to base your own saas product on has worked in the past, and it has also failed in the past.

That will probably happen. But it will take some time. Twitter did that. At first they were super open let everyone build apps, then they started blocking and copying the most popular ones, or acquiring them.

However, the companies will probably soon be able to switch out the LLM they are using, so potentially they could use Google’s PaLM, or Facebook’s Llama, or some open source LLM, who knows.

They can also fine-tune the models for just-your-data, so that the responses for you are better than just using a general LLM.

I think the key is that OpenAI wants to focus on building AGI and not waste time appeasing users requests for UX shortcuts for GPT powered code search. Sure, they can hire an extra team to manage the user's requests for UX, but at that point they're moving away from their target goal.

or a few iteration down ChatGPT will create that UI too. In the future it will not be a "boring" text chat but an interface that always morphs into the specific task you are asking it about.

Except in the case of GPT-x, the facade builder must take extra pains to sanitize the inputs, so it doesn't become just a free GPT account for the user to interact with on unrelated topics. That's much more challenging due to the open nature of the prompts (the selling feature). One really lame solution is to ask the user to use their own openai API key.

I don't think that is a lame solution, it would work pretty well if OpenAI had better key management, especially spending limits per key.

If you're forcing your user to get their own OpenAI account, you're pushing them away from your product, at least somewhat. IMO the integration needs to be seamless for the illusion of "we're providing value here, not just wrapping GPT" to hold.

Investors want AI. Do you see how crappy their landing pages look? This one is pretty decent compared to what we've seen showcased here in the past months. It's as if someone at YC told them to ship it asap, with no consideration for anything other than "slap AI on it".

MVP, be embarrassed of your first release, etc.

I would like also to understand the amount of startups funded in 2021 (or around) that relies on GPT-{n} now. They pivoted or found out that they solution was much easier with GPT-{n}?

Also, why as a customer would care if you use GPT or something else? It is because it is a buzzword?

People care if you use GPT because it has capabilities that other AIs don’t have.

> why any company would pay for N different GPT saas

Simple: You do it because you pay for what SaaS does, not what GPT does. A user is not interested in postgres or nginx, even though they are the tools used to build the tool they care about and pay for. If what the new tool uniquely does is not adding enough value for enough users, it's going to fail.

Some of these app are too niche, but maybe not all of them. There have been many attempts at a "unified enterprise search app" for example, but they all have generally failed. Maybe customer indexing + GPT + form fact (slackbot, gui, etc ..) is enough for a company. Vanilla ChatGPT probably isn't going do that.

Or consider codex low-code type use cases. Yes you could generate generic code, but integrated into some kind of platform that knows what APIs your company uses (internal cataglog) and some kind of IAM/Auth platform, PAAS to host the results integrated into a single tool might make sense, especially if the prompts are already injecting the boilerplate about how to interact with the ecosystem.

Or consider where just adding some kind of generative feature is just a _feature_ of broader product.

You are not wrong on this one.

The moat they are going to build is going to incredibly ... low.

The main intelligence is going to be provided by GPT models.

Well, one week after the API launch everyone's moat will be low.

It will be interesting to see how fast these tools grow, and how they can add value beyond a purposeful UI.

Agree. The most useful moat you build as an AI-as-a-service company is detailed data on how good each of your past outputs were. E.g. Github presumably keeps track of which copilot completions end up being used and which don't. If you are just using the OpenAI API, you have basically no way of making use of that data for finetuning. You could use the data in order to finetune your own model in the future though, maybe that's the plan.

There's a lot of work you could theoretically put into automating many of the queries you would want to do with a GPT chat interface.

For example, I'm no a medical diet and need a meal plan. This diet has quite a few restrictions.

I could set up automated queries that produce a weekly meal plan, and grocery list, and also checks all of the ingredients against the allowable / not allowable food lists. This could include multiple queries, one to get an initial response, another to validate it, etc. Everything could be organized nicely, so I wouldn't have to input any text to get what I need -- I'd just click buttons. There's lots of ways this could be further customized to be more useful, such as saving recipes, generating new recipes based on old favorites, generating recipes based on foods already at home, etc. (Heck, you could wire it up to a camera in the fridge.)

Anyways -- text input to get text output is the most basic form of this technology. There's a lot you can do with such text to make it easier to interact with for specific use cases.

Most of bloop's interface is search results, which are driven by the LLM and have highlighted line ranges and go-to-ref/def. Down the line, many other LLM powered tasks will need specialised interfaces too, like showing a diff or a git history tree.

This is quite different from a GPT tool for most other jobs, and I think having granular control of the interface layer certainly helps us ship a better product.

That's not to say everything needs to be an app. If the output is just conversational text, it can and probably will be some kind of 'Alexa skill' like plugin.

I think you could say similar things about cloud computing, but we pay for different apps that are all ultimately startups re-selling AWS resources.

If you can give it proprietary training that takes nontrivial work then I don’t see why it couldn’t be worthwhile. Collecting, curating, cleaning up, manipulating data to make the best model of X would easily be worthwhile to make and to then purchase for hundreds of companies.

If it’s just appending a prompt to chatgpt then it’s certainly useless

It seems you're doing the opposite of "missing the forest for the trees". You see the forest (the app) but you're not seeing the individual trees.

The answer to why companies don't have one interface with GPT is that they would still need to invest time and money into building the application that uses GPT. Checkout the bloop source code to get an idea of what's needed https://github.com/bloopAI/bloop there are also many complex things to deal with. You can't just shove an entire codebase into GPT-4, it has limited ability to track context. The codebase needs to be indexed and stored in a way that sematic search can be done on it very quickly. Every app has their own unique challenges towards getting the data into GPT and making it fast.

TL;DR companies are buying apps that use GPT-4 API, not just access to GPT-4. They'd rather buy the apps instead of building and maintaining them.

I think it's because variations are in themselves valuable new stand-alone applications.

What you're asking is like "Why should i pay for N different internet saas" because they share a common characteristic (e.g.they use a database) paired with the naivety where you don't understand the underlying complexity of a business application and think you can do this all by yourself

you're advocating for byok. saas choose not to use byok because they want to charge rent on your usage instead of making money in other ways.

Reply on news.ycombinator.com