ChatGPT works seamlessly with photos and files, enabling numerous new use cases | Hacker News Reader

ChatGPT works seamlessly with photos and files, enabling numerous new use cases

twitter.com

Observational hypothesis based on 5 minutes with the app (and asking it to repeat the prompt it was given):

This does not use any OpenAI service to analyze the image. Rather, it’s using some other image, recognition model to generate a description of the image, feeding that description into chatGPT, and having chatGPT hallucinate other likely details based on that description.

I wish this could be self hosted or at least e2e encrypted in a way where our private data couldn't be leaked somehow.

Agree. I wonder how well it would handle mechanical design drawings. If it could, it might be very helpful in my industry, but our drawings have strict privacy restrictions.

Eventually I hope we’ll be able to able get a single high end server appliance to run an instance in house for such use cases.

Even better, run locally. That should be the goal.

Is this realistic though anytime soon? How long until this can be self hosted and how long after that until it'll run on a laptop or phone? Granted I know nothing about the ML hardware that's in Apple hardware and even Google phones so maybe we're closer than I imagine.

Do you know if that hardware is capable of running LLM or is that optimized for something else?

> Do you know if that hardware is capable of running LLM or is that optimized for something else?

I do! Checkout llama.cpp [0] and related projects. The game changer with this was Gerganov's ggml format that makes inference on CPUs realistic. I'm currently running alpaca.cpp and llama.cpp, using 7B models 4-bit quantized (so they're the lowest powered, lightest weight models) on an M1 Pro with 16gig RAM. You could run same on an M1 Macbook Air with 8gig RAM.

Gerganov shows the Whisper model running on an Android phone.

[0]https://github.com/ggerganov/llama.cpp

Free, open source alternatives are in development. Or so my alpaca tells me.

Reply on news.ycombinator.com