If Facebook tracks me outside of Facebook's website because that other website has tracking code or a like button (which I didn't click) and I don't have an account on Facebook, I'm not ok with that.
Then your beef is not with FB but the website which _chose_ to embed FB like button. You should stop using that website if you are not ok.