Oh, what you might mean is, "If I can't fly because I will disturb you on the ground, you are stealing the enjoyment of the skies from me." I can see that. I think it also relates to the number of people affected. If one pilot during one flight across a forest disturbs 100 people who are trying to enjoy nature, that imbalance is where the difficulty is.
That makes more sense. I'm not convinced that most people are bothered by aircraft noise to the extent you are, but I understand your argument.
It would also be theft if you were making huge unnecessary noise playing music or whatnot. It's all about the amount of noise and how far it travels.
This does not make sense because enjoying a forest does not cause noise that would prevent the pilots from enjoying the sky.