I always thought Tom was sort of "outside reality". Maybe Tom was the physical representation of Eru itself. Or maybe some aspect of Eru.
You don't remember right. :)
I think what you are remembering is when the Council discusses leaving the Ring with Bombadil, at the end of which they conclude that it wouldn't work; even if you could make him accept, he would lose it, or forget the necessity and importance of it, and even if he didn't, then all of Middle-Earth would fall to Sauron (who can torture even the hills), last of all Tom: Last as he was First. There is no suggestion that the Ring would ever afflict Tom.
> "Could [Sauron's] power be defied by Bombadil alone? I think not. I think that in the end, if all else is conquered, Bombadil will fall, Last as he was First; and then Night will come."