You can still hold on to references to textures and meshes you uploaded to the GPU without using a full-blown scene graph. Some state is necessary no doubt, but this seems more like unnecessary state that could be replaced by something more direct. But i don't know, i'm not familiar with three.js; the click handlers seem useful.