Indeed, if apathy is the absence of caring, and caring is part of both love and hate, then I don't think any of these are opposites of each other, they are merely things that can not co-exist.
I often wonder if apathy is just the belief that one is without love, not actually being without love. I think we die pretty fast if we don't have love and attachment to things, even at the most basic level of air, water, food, and shelter.
not caring as in, not caring if you get rejected. Not caring what others think when you peruse what makes you happy.