Hacker News Reader

Comment by adam_arthur | Hacker News Reader

Much harder to switch cloud providers than to switch LLM models. How much time would it take most companies to move their product from AWS to GCP, for example? What if you use a cloud specific tool like DynamoDB?

Margin is a function of stickiness/cost of switching (among other things).

I suspect eventually we will enter a world where migrating cloud providers is mostly a click of a button, but we're a long ways off from that. Requires vendor agnostic and portable apis/containers/WASM runtimes everywhere.

Swapping an LLM, at least in the current state, is about as close to updating a pointer as you can get.

View on HN

If you swap an LLM, at the very least, you have to run your entire Eval set again.

This will almost certainly lead to a prompt rebuild, to better accommodate new model idiosyncrasies.

If you are unlucky- your use case may be one where Evals require human review.

Unless you are YOLOing it without evals. In that case this is relevant.

You need to rerun eval after each LLM update. GPTs have a new version every couple of months and their capabilities can change quite drastically pretty much randomly. Maybe they will make it more robust in time, but I think this is the feature of the technology and people will have to adapt to these quirks

Yup. Constant Eval.

Well I'd have to change my Terraform provider and the managed Kubernetes resource... Other than that, it'd be the same. So half an hour of coding + half an hour of reconfiguring CI secrets?

Pretty obvious to most that switching cloud providers is not quick or painless for the majority of orgs. There's not really an argument in good faith to suggest otherwise.

Especially given that many orgs use managed or cloud specific solutions that have no 1:1 mapping between vendors

I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more.

> I said I'd have to change that one managed resource. Actually it's two - the managed database. But no more.

As someone who worked on a similar project recently, I'm getting that idea that you obviously don't know what you're talking about.

Didn't you know? It's only one DB ENV variable and you're swapped over from GCP to AWS. /s

I'm running the tech for a global startup with 100M EUR turnover. It's still pretty small, but it's something.

You need to plan ahead, sure. Portability was one of my main concerns (including the possibility to go self-hosted). But it's definitely not impossible, nor too hard to do.

You lost me at Terraform and Kubernetes..

What's a serious cloud provider-portable replacement for Terraform?

And Kubernetes? It goes way beyond containers, you can replace many cloud provider-specific resources with Kubernetes resources. What alternative gives you that?

Reply on news.ycombinator.com