This is something we are painfully aware of and have started to work on. For a start, see Jake's work on https://pypi.org/project/altair-data-server/.