I use FFmpeg for many years, and I did wonder many times why isn't there a CLI that would be simpler to use but still useful for most tasks.

To my knowledge, no project succeeded in doing that. It's not difficult to code a simple reader and a basic encoder, but then you have many cases not supported correctly, as you wrote.