Give us some details about your setup. Are you running JH on the cloud somewhere? Or is this running on a single box at VT? We’ve got JH running on a Kube cluster with persistent storage on the NSF Jetstream cloud [1] (which you could qualify for free resources being from VT) . This set up should theoretically address the scalability concerns you are having. See the work Andrea Zonca and I have been doing [2, 3, 4, 5]. Contact me for additional details.

[1] https://jetstream-cloud.org/

[2] https://zonca.github.io/2018/09/kubernetes-jetstream-kubespr...

[3] https://zonca.github.io/2018/09/kubernetes-jetstream-kubespr...

[4] https://zonca.github.io/2018/09/kubernetes-jetstream-kubespr...

[5] https://github.com/Unidata/xsede-jetstream/blob/master/vms/j...