I think OS/400 has some really cool technology at the lower levels of its stack, but the applications running on top rarely really took advantage of them.
You start with an advanced machine-independent single-level store capability-oriented operating system, and then you use it to run RPG code???
OS/400 was based on an object-oriented idea. And yet, only IBM could ever define new object types. But IBM went ahead and defined all kinds of random obscure object types [1][2] for all kinds of obscure requirements, many of which are now quite legacy. It would have been far more powerful if IBM had opened it up and allowed end-users and ISVs to define object types of their own.
One thing always holding OS/400 back, is that it has never been a self-hosting operating system. OS/400 has never been able to compile itself, and the OS/400 development team didn't run OS/400. The code for the core of OS/400 was maintained on a mainframe running VM/CMS, which hosted cross-compilers (for PL/MP, PL/MI, etc); OS/400 developers would transfer the compiled code to an actual AS/400 for testing. Around the time of the RISC transition, the development OS changed from VM/CMS to AIX.
[1] https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/api...
[2] https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_74/rba...