You absolutely should! It really is true. Plus you actually have more dexterity—you can't really hold anything with gloved fingers anyway. Mittens > gloves!