> if it is indeed his property he should have the right to do what he wants with it.
The general principle is that you can do what you want with your property, but you must pay for any damages you cause in doing so.
The important question is whether you did cause those damages (maybe it was someone else's fault, or maybe the damages were inevitable and would have happened regardless of your action or inaction, or maybe you acted in good faith like any other person would have) which gives judges plenty to think about.