Hacker News Reader

Comment by a2128 | Hacker News Reader

Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of dollars next year once the subsidies dry up? With a local model running with open source software, at least I can know that as long as my computer can still compute, the model will still run just as well and just as fast as it did on day 1, and cost the same amount of electricity

View on HN

Facts. Google did the same thing you describe with Maps a few years ago.

It's not just Google, literally every new service always does this. Prices will always go up once the have enough customers and bean counters start pointing at spreadsheets. Ergo, local is the only option if you don't want to be held for ransom afterwards. As goes for web servers, scraper bots, and whatever, so goes for llms.

I think there's a few things to consider:

They make a ton of money on large enterprise package deals through Google Cloud. That includes API access but also support and professional services. Most orgs that pay for this stuff don't really need it, but they buy it anyways, as is consistent with most enterprise sales. That can give Google a significant margin to make up the cost elsewhere.

Gemini Flash is probably super cheap to run compared to other models. The cost of inference for many tasks has gone down tremendously over the past 1.5 years, and it's still going down. Every economic incentive aligns with running these models more efficiently.

Aren't API calls essentially swappable now between vendors now?

If you wanted to switch from Gemini to Chatgpt you could copy/paste your code into Chatgpt and ask it to switch to their API.

Disclaimer I work at Google but not on Gemini

Not tokens allowed per user. Google has the largest token windows .

Different APIs and models are going to come with different capabilities and restrictions.

It's Google. You know the answer ;)

I mean, there’s no need to dry up subsidies when the underlying product can just be deprecated without warning.

Run test queries on all platforms using something like litellm [1] and langsmith [2] .

You may not be able to match large queries but, testing will help you transition to other services.

[1] https://github.com/BerriAI/litellm

[2] https://langtrace.ai/

Google has deep pockets and SOTA hardware for training and interference

It's "free cloud picture video storage" rush all over again

Are you asking whether giving away $5/day/user (what OpenAI charges) in compute is sustainable?

Reply on news.ycombinator.com