HNHacker News
TopNewBestAskShowJobs

rshemet

43 karma · joined August 6, 2022

submissionscomments
rshemet··on Show HN: Cactus – Ollama for Smartphones
The models themselves live in an isolated sandbox. On top of that, each mobile app has its own sandbox - isolated from the phone's data or tools.

Both the model and the app only have access to the tools or data that you choose to give it. If you choose to give the model access to web search - sure, it'll have (read-only) access to internet data.

rshemet··on Show HN: Cactus – Ollama for Smartphones
you can plug in a vector DB and run Cactus embeddings for retrieval. Assuming you're using React Native, here's an example:

https://github.com/cactus-compute/cactus/tree/main/react#emb...

(Flutter works the same way)

What are you building?

rshemet··on Show HN: Cactus – Ollama for Smartphones
great stuff. (good timing for a post given all the comet news too :) )

DM me on BF - let's talk!

rshemet··on Show HN: Cactus – Ollama for Smartphones
appreciate the feedback! Made the demo links more prominent on the README.

Some Android models won't support GPU hardware; we'll be addressing that as we move to our own kernels.

The app itself is just a demonstration of Cactus performance. The underlying framework gives you the tools to build any local mobile AI experience you'd like.

rshemet··on Show HN: Cactus – Ollama for Smartphones
Thank you! it goes without saying that the field is rapidly developing, so the use cases range from private AI assistant/companion apps to internet connectivity-independent copilots to powering private wearables, etc.

We're currently working with a few projects in the space.

For a demo of a familiar chat interface, download https://apps.apple.com/gb/app/cactus-chat/id6744444212 or https://play.google.com/store/apps/details?id=com.rshemetsub...

For other applications, join the discord and stay tuned! :)

rshemet··on Show HN: Cactus – Ollama for Smartphones
Cactus is a framework - not the app itself. If you're looking for an Android demo, you can go to

https://play.google.com/store/apps/details?id=com.rshemetsub...

Otherwise, it's easy to build any of the example apps from the repo:

cd react/example && yarn && npx expo run:android

or

cd flutter/example && flutter pub get && flutter run

rshemet··on Show HN: Cactus – Ollama for Smartphones
didn't Ollama vendor Llama cpp too?

Most projects typically start with llama.cpp and then move away to proprietary kernels

rshemet··on Local-first software (2019)
Interesting to think about the concept of local-first in the age of AI.

Wanting to be able to run AI fully privately, and offline is the reason we created Cactus:

https://github.com/cactus-compute/cactus

Fully open-source, cross-platform & blazing-fast; lets you plug in private AI into any app on your phone.

rshemet··on LM Studio is now an MCP Host
hey! we're building Cactus (https://github.com/cactus-compute), effectively Ollama for smartphones.

I'd love to learn more about your MCP implementation. Wanna chat?

← PreviousPage 2 of 2