I've had similar thoughts. Any interest in sharing notes? Generally I've been thinking there should be a standard declarative format for data vis. This should be inspired by prior art like ggplot, Grammar of Graphics, Vega, VegaLite, etc.

From there it could be compiled to targets declarative or procedural (eg to generate VegaLite specs, echart specs or Observable Plot configs in JS).

How have you been thinking about this problem space?