"I'm out of town this weekend, you can use my apartment" does not grant somebody unlimited permission to use your apartment. The supreme court is working backwards from their conclusion here.
And not even that people's right to own and use guns would be revoked if it turned out guns were not good for hunting.