we started at the transistor level, building AND/OR gates. Then XOR/NAND etc, then a simple ALU (using real nand/and/or etc chips) then a simple register, etc all the way up, each step getting an integrated chip to replace what we had built with smaller components in the previous step.
For anyone whose interested in this sort of thing I highly recommend The Elements of Computing Systems (http://www1.idc.ac.il/tecs/).