If the game that is using the NFTs I bought is shutdown, what the hell is the value of these in-game items that I bought if the game for them doesn't exist anymore?

Yeah, someone else can create a new game and say "we accept the NFT items from Defunct Game X", you are still completely under the control of whomever is providing you the platform for these in-game items to actually exist. If that platform is taken away you just have some bits of data pointing to other bits of data with no meaning and no value at all...