Reading this it sounds like you are discouraging using it due to the implementation being big and hard to maintain. Did you mean it this way?
This reads to me as it being a PITA to maintain. Cross platform code is usually a pain, cross platform with assembly optimizations is more of a pain. Optimization nearly always makes things harder to maintain, and this sounds like it was optimized to hell and back...
Not more than any other language: when it's well done, it's manageable. When it's spaghetti code, it's not manageable.
People don't really get how complex video codecs are.