Brings to mind Smalltalk's concept of images (which comes from lisp, I believe) and Forth.
Of course in Smalltalk, you do work with code-as-text, but then it becomes code-as-program (technically byte-code, with among other things a text representation). Does seem more reasonable than the anachronistic insistence on text->parse-to-AST->(whatever magic, and however many transforms the actual compile-part is).
Then again, we all know what a rich editor is. It's MS Word. And MS Word eats documents and depricates file formats.
On the other hand, I think Emacs Org-mode, Gimp images and the literate editor in Python, Leo[l] are examples of more pleasant "rich format" editors. It is a bit odd that what we take for granted for image files (edit history, binary format++) we fear in our IDEs.
[l] http://leoeditor.com/