> probably because most cameras, except DSLRs, don't support storing raw images
This may be true for consumer cameras but not for machine vision cameras. Most color machine vision cameras do support the Bayer format. Basler, FLIR (Point Grey), IDS and so on all support the Bayer format. If you roll your own camera you definitely have to deal with the Bayer format from the sensor you are using.
And if you do not want any compression on your data-set images the Bayer format also takes less space than the RGB format.
I'm not arguing it is a good idea, I'm just interested if it works :)