Well yeah, everything should be in moderation. Figure out what macros your body needs, and get as much variety in filling those out as is reasonable.

I drink mostly water, but I probably have ~1 cup of milk on average each day (e.g. no milk one day, 2-3 cups another day, etc). I go for whole milk because the higher fat content means I'm having less sugar per calorie, and it makes me feel more full than other kinds of milk. If you like milk, consider increasing the fat content and decreasing the total amount you drink so you're more satisfied when you do have milk.

If you're having multiple coffees everyday with tons of sugar, switching from dairy to non-dairy creamer isn't the right optimization...