"Based on prototypes I [spent a limited amount of time on and didn't research better methods], I'm confident..."
> Nowadays, my rule of thumb for pandas is that you should have 5 to 10 times as much RAM as the size of your dataset
[1] https://wesmckinney.com/blog/apache-arrow-pandas-internals/
Web application using Pandas and "highly optimized web application" would seem to be nearly disjoint sets...