The point of the Chinese room is to show you that the guy who has memorized the standard responses to various questions in Chinese does not in fact know Chinese. He's mindlessly parroting things.
An AI cannot be removed from its box, because it doesn't have one. It really does have enough information inside of its essence to reply. In fact, that information makes up its essence.
I agree that in some sense their knowledge is distinct and of a different character to human knowledge. But what that means conceptually or morally exactly is very complicated, and cannot be dismissed easily