FWIW I did some things in this area a while back and ended up needing to stream canvas state back to my Python scripts for image processing.
https://gist.github.com/iandanforth/0ed987bfddf8205b8a23
I hope that could be a part of this framework in the future! (If it isn't already)