We've had similar experiences with BI consumers in the past. In our experience, it's really hard to make self-serve work for business users. Initially, we're starting with ways to make it easy for end users to get information out of Evidence (things like CSV download, smooth copy-paste into docs/email/PPT/etc).
Over time we'll be putting more effort into features that will let people explore data from within Evidence (which we call "best of BI" internally). We're thinking very carefully about it so we can hit a few of our key principles: maintainable/version controllable by data team, very fast - no waiting time for users, intuitive for end users - limited if any training required.