The whole legal process surrounding IP, the laws, the patent offices, layers of locals laws, international agreements, weird precedences. The enormous legal costs involved in anything IP (especially patents). Having to relate everything to printing copies in a press or inventing something mechanical. It's all broken.
I don't think we can tweak our way into a better system. Copyright isn't the same thing in a world where "copy" is no longer a real thing. The public interest when it comes to patents is not the same in a world where inventions are organisms and software as it is in a world where inventions are mechanical machines. The rate of patentable innovation is completely different. The line between invention and discovery (crucial to the concept of patentable invention) is much blurrier than it was. The moral implications are not the same. The economic implications are not the same.
The legal system governing patents has several big features which suggest it is completely broken. Patent trolls using the ungodly cost of litigation together with single use limited liability entities (another concept that is now broken) to use the legal system without being bound by it. Patent wars between huge companies and the subsequent ceasefires. There is no way any sane person would have purposely have designed a system this way.
It's like having a police force that just shoots everyone when they arrive at a scene. Then some criminals find out that they can walk into a bank wearing armor and threaten to call the police. Obviously something's not working right.
Even in the domain where patents are have the strongest case: drugs & medical procedures, the patent system is very warped. The problem is that it is expensive to test a new drug in a ways that proves it is safe to the authorities. So, a guaranteed monopoly is necessary to justify the expense. But, that research is not invention and it isn't the thing which is patentable. If we want to reward companies for demonstrating the effectiveness and safety of a drug and getting it approved for use, lets do that directly.
I really think we need to blank slate intellectual property laws. It's one of those things that seem both impossible and inevitable.