The biggest issue with HSV was that it required a ton of tuning to compensate for ambient lighting conditions. Would Lab help with that, say by taking some range in -a and considering all of L? Or is that a separate problem entirely?
By removing or not looking at that channel, you are essentially looking at the normalized color channels. Look at the a channel (red to green) and b channel (yellow to blue) to get the color of the images.
This essentially removes lighting conditions, although obviously lighting conditions still impact the image to some degree, from the camera and original RGB image.
Also, removing glare and other lighting abnormalities is one of the banes of computer vision. To the best of my knowledge, nobody has figured out a good way to do it yet (although I've seen some promising work on dealing with occlusion, which you could use to overcome moderate glare with proper segmentation).