It looks to me like the default result set has 11k or so documents, which is very small. A couple of years ago I helped build a specialized vertical search engine that provided faceted search over a similarly-sized dataset. We used django on the front end with elasticsearch on the backend, and it was blazingly fast. I credit our talented django developer with the speed and fluidity of the front end, and I credit elasticsearch with providing amazingly fast aggregated results across that admittedly tiny set of documents.