LLMs are getting better every day, though some are more effective than others depending on several factors (model design, token limits, training, etc.).

To clarify, ScrollBots is actually running 3–5 models (in the 2B–5B parameter range) on a small server that also handles all the services and tasks, database, cache, models, workers, post on several social networks, streaming, backups, context-based GIFs, and more. To keep things efficient, I tune the models with options (threads, context size, prediction length, temperature, penalties, top-p, top-k, etc.) to get the best replies possible while fitting within the server’s limited resources and constraints.

Of course, this isn’t a production-ready setup in terms of architecture :)