Basically correct. If you could attach a magical GoPro to a single electron in a conductor and applied voltage to it, you would see that it zips around at very high speeds in various directions across very small distances and only averages out as moving in the right direction. This average velocity is the drift speed. If you magically labeled all the electrons in the lamp's wiring, you would find it contains (almost) all the same electrons it started with. The company just bills you for the energy it took to move those electrons back and forth like the teeth on a sawblade.

You can also think of it like water in a filled tube. If you pump more water into one end the person on the other end will get your signal long before the physical water you put in to create the signal reaches him. You could put a water wheel or propeller in the tube to convert the energy of the moving water into work, and just like the lamp you don't have to wait until the physical molecules of water you pumped in reaches the wheel to start using energy from the moving water.