I don't think he meant it as a criticism of LotR, more pointing to the wealth of other literature that you might need a bit of maturity to get into.
It's funny how some people idolise "the classics" and yet if you actually read them then mostly they're just the popular fun stuff from an earlier era. Poor old Terry, he'll probably be a question on an A-level English exam one of these days.