RAM is always the bane of machines like this. You can build a working CPU with a relatively few discrete elements, but a usable amount of RAM always needs many times more. Core memory is probably the only sane option in projects like this, save for Williams tubes.