Hacker News Reader

Comment by diggan | Hacker News Reader

Are you saying that it's faster for them to get the hardware to run the weights themselves? Otherwise I'm not sure what the relevancy is.

View on HN

Unless they are already in the possession of such hardware (like an M3 mac, for example).

Yes some have existing infra

I'm having a somewhat hard time believing a corporation where getting a API key for a LLM service is very difficult, somehow has the (GPU) infrastructure already running for doing the same thing themselves, unless they happen to be a ML corporation, but I don't think we're talking about those in this context.

No, this is very real. One reason why this can happen: a company has elaborate processes for protecting their internal data from leaking, but otherwise lets engineers do what they want with resources allocated to them.

Nah this is definitely a real scenario. Getting access to public models requires a lot of security review, but proving through Bedrock is much more simple. I may be spoiled in having worked for companies that have ML departments and developer XP departments though.

Not sure Bedrock counts as self-hosting though, isn't it a managed service Amazon provides?

> I may be spoiled in having worked for companies that have ML

Sounds likely, yeah, how many companies have ML departments today? DS departments seem common, but ML i'm not too sure about

A lot of companies think they do

Reply on news.ycombinator.com