You did not answer the question they asked. You answered what would have been a reasonable interpretation of half of their sentence, if there hadn't been more text directly around it.
> You replied to me.
I did.
But if you were purposefully isolating half their sentence, there was no reason for you to argue with that reply. The way you argued implies you weren't doing that on purpose, and thought you were answering their actual question.