I'm not sure how drastic these limits really are. The display is tiny, the graphical memory small (32K), and code is limited to 8192 Lua tokens.
On the other hand, I haven't seen any computational model. It seems the "machine language" actually is Lua, and processing power is not limited by the specs, but by the host computer. Thus, there are quite a few ways we could trade speed for size.
My dream of a fantasy console is one that we could actually instantiate in actual hardware, should we want to. Ideally a cheap FPGA for which we have a Free toolchain, like the iCE40.