You're not obliged to hand this data over when you don't use their services, and ideally, they wouldn't track you if you didn't want to be tracked, but facebook is pretty close to being ethically void, so they'll do whatever scummy thing they want, so you should by all means make it as difficult as possible for them to get data on you.
Unless you run an ad blocker and/or EFF Privacy Badger, that's exactly what you're doing. Your consent is not required. There are literally _dozens_ of companies that do this, Facebook and Google are just two of them. There were some initiatives in some states to apply a stiff tax to companies that trade in user data, but I don't know if they went anywhere. That's one tax I'd be in favor of, even though I'm not in favor of more taxes in general.
The point is that regardless of whether or not consent is required, it's a question of principle to not give consent (and personally I do use LibreJS and the other various tools to improve the shitty state of affairs).
Companies acting unethically shouldn't be given extra leeway to act even more unethically.