However, if you want to build really high performance cores, there are plenty of challenging techniques you have to employ that add a lot of complexity that is hidden below the ISA abstraction layer (speculation for example).
So if you want to make RISC-V go fast, you have to employ more design tricks like "macro-op fusion". For example, scan for two load instructions in the fetch stream and fuse them into a single "load-pair" micro-op if they access adjacent addresses. There are a whole bag of tricks like this that are irrespective of the ISA and add a fairly high "skill-ceiling" to processor design.