I've been pretty happy with datadog's distribution type [1] that uses their own approximate histogram data structure [2]. I haven't evaluated their error bounds deeply in production yet, but I haven't had to tune any bucketing. The linked paper [3] claims a fixed percentage of relative error per percentile.
[1] https://docs.datadoghq.com/metrics/distributions/
[2] https://www.datadoghq.com/blog/engineering/computing-accurat...