Hacker News Reader

On mac silicon:

ollama pull mixtral

For a chatgpt-esk web ui

https://github.com/ollama-webui/ollama-webui

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v ollama-webui:/app/backend/data --name ollama-webui --restart always ghcr.io/ollama-webui/ollama-webui:main

Navigate to http://localhost:3000

You can also use ollama in langchain.

View on HN

There's also some unlocked fine tunings available. Dolphin seems to be a very popular one. (Trained on more coding data) If you want to fit under 32gb, there's https://ollama.ai/library/dolphin-mixtral:8x7b-v2.7-q3_K_M

I haven't played with dolphin-mixtral, but dolphin-mistral gave me a good impression for generic rag application.

There's also developments and experimentation in making it more factual via dpo and laser(afaik so far not very unsuccessful).

I wouldn't use Ollama (or any LLM host) through docker on Mac m1 since as far as I'm aware there's still no support for Metal.

The UI is in Docker but the Ollama server isn't.

Reply on news.ycombinator.com