I haven't seen this problem on recent versions of Ubuntu (10.04, 12.04). fail2ban bans will preempt regular ufw rules (as they should).