It is fun: but I like this kind Bonzai tree thing which is not pactical but a relaxing but challenging experience. And it feel really nice to actually really understand everything for a change.
That said, it is a good start... If you can find a cheap one, find the schematic online and start checking out the components and measuring the signals. I am lucky to have many systems that have 74series logic + memory + cpu in them which is really easy to follow for any programmer. Most machines from that era won't have that unfortunately.
If you cannot or will not risk that then simply buy a power supply, some packs of analog electronics, digi scope (few $100 one is enough), a box (2nd hand you can get them cheap) of different 74series, breadboard and a computer kit like [0] or [1] or [2] and start experimenting.
[0] http://www.ebay.com/itm/Zilog-Z80-Colecovision-Kit-8MHz-Z80-...
[1] http://www.ebay.com/itm/Zilog-Z80-ZX-Spectrum-Homebrew-IC-Ki...
[2] http://www.ebay.com/itm/Mixed-Kit-6502-or-63C09-or-Z80-CPU-S...