You can start with a breadboard, oscilloscope and a set of standard parts (LEDs, resistors, capacitors, inductors, diodes). Combine this with a microcontroller you can flash via USB dongle or an RaspberryPi. Should all fit on a desk.
The lab is really only necessary for soldering and creating custom PCBs.