Make sure you're using an application-specific password for any application that doesn't support two factor authentication - eg, Mail.app on Lion.

Thats what they're for.