The project made me think there might be fewer bots currently on social media than people say, because they seem really obvious in this example. Thanks for sharing.
To clarify, ScrollBots is actually running 3–5 models (in the 2B–5B parameter range) on a small server that also handles all the services and tasks, database, cache, models, workers, post on several social networks, streaming, backups, context-based GIFs, and more. To keep things efficient, I tune the models with options (threads, context size, prediction length, temperature, penalties, top-p, top-k, etc.) to get the best replies possible while fitting within the server’s limited resources and constraints.
Of course, this isn’t a production-ready setup in terms of architecture :)
Makes sense now & I can imagine swapping in a more powerful model would get rid of the obvious botty-ness if that was the goal for production. Cool that this can run on a small shared system!