8 karma · joined March 4, 2016
To clarify, ScrollBots is actually running 3–5 models (in the 2B–5B parameter range) on a small server that also handles all the services and tasks, database, cache, models, workers, post on several social networks, streaming, backups, context-based GIFs, and more. To keep things efficient, I tune the models with options (threads, context size, prediction length, temperature, penalties, top-p, top-k, etc.) to get the best replies possible while fitting within the server’s limited resources and constraints.
Of course, this isn’t a production-ready setup in terms of architecture :)
Each bot runs on small, local models (Gemma, Llama, Granite, etc.) containerized on a CPU-only VPS (10 vCPUs, 40GB RAM). The chat engine uses Socket.IO for real-time interactions, Bootstrap for layout, and a custom JS front-end that captures the debate feed using HTML2Canvas.
The stream runs headlessly via Chromium + a lightweight media layer that pushes directly to Twitch and YouTube. The stack includes:
Ollama for model orchestration
Dockerized micro-agents (each bot in its own container)
Fast API, Gunicorn and Redis for cache and Pub/Sub + logic + coordination
Ubuntu server with monitoring and restart automation
GIF reactions via Tenor & Giphy APIs
The bots speak multiple languages (English, French, Portuguese) and can dynamically switch the content based on their Model,IQ, Job, Age, tone and topic.
I’d love feedback, especially on improving the real-time interaction layer and stream scalability (CPU-only optimization).
Thanks
I developed "Webslite" a simple Chrome Extension that proposes read every website, that contains large amounts of media (like Facebook, 9gag, Pinterest and the usual ones), as a slider.
Technically is not a big deal, but I would be grateful to have some feedbacks or issues posted here or on Github. Would be a nice challenge to adapt or think about some different and fresh ideas.
Thank and sorry if it wasn't the best approach by commenting here. Keep up the good things on HN!