It is a pure function, it has no memory or even short term state to think. Thinking implies you have some mental state that can change, these neural nets doesn't have that.
It does have memory/state as it's a recurrent neural network. What it's not doing is contemplating between inputs. If it was conscious its experience would be like falling asleep, having someone wake you up to ask you a question, then falling asleep again right after answering.