I'm not sure I understand, are you basically brushing twice? Once w/o toothpaste and once with?
I rarely ate sweets as a child and avoid sugar. I rarely eat processed foods. Drink is another matter. I most likely have harmed my teeth from drinking beer/wine/smoothies and from being too rough when cleaning. I really miss my younger healthy teeth.
My teeth have felt horrible ever since they failed and were drilled and filled. And now bring me a lot of misery. Look after your teeth! Healthy teeth are incredibly attractive. IMHO.