Really for me just the mantissa would be fine; no need for exponent bc so much of what I worked on is between 0..1

There was an interesting paper from the Allen Institute a few years ago describing a system with 1 bit weights that worked pretty well! Since I read it I've been musing on trying that, though it seems unlikely I will be able to any time soon.