How do you intend to prevent locals from "visiting" your island when you are away? Once someone realizes there's an awesome free island with lots of cool stuff to play with you might find it prone to vandalizing.
I've had similar romantic notions about owning an island (we kayak a lot). But the reality is that every vacant island IS your island. There's plenty of islands < 1 hour away that are beautiful and completely uninhabited. Hopefully someday I will come upon yours... sounds nice.
<pessimism>I hope it works out for you but taxes go up, structures get vandalized, local permits are annoying and 10 hours is a long friggen drive. And the fastest way to end a friendship is to joint own something.</pessimism>