2pm might not be early enough. Depending on genetics the half life of caffeine in your system can vary by a lot. Even if you're in the normal range, a half life of 4-6 hours means that ~25% of the caffeine would still be in your system by midnight...
If you're that worried about sleep quality, it's baffling you have not tried cutting out caffeine altogether