The firewall of patents exist precisely because digital video is a way to shakedown the route media would have to travel to get to the end user.
Codecs are not, "harder than" compilers, yet the field of compilers was blown completely open by GCC. Capital didn't see the market opportunity because there wasn't the same possibility of being a gatekeeper for so much attention and money.
The patents aren't because it is difficult, the patents are there because they can extract money from the revenue streams.