People might not share my sentiment but I think doing this is fine. Yes, all your points are true, but they require the attacker to single you out specifically, and try and figure out your process. Frankly, unless you're a very important person to warrant someone singling you out, an attacker is going to go after the countless easier targets that do not require personal attention.