Words and language refer to sensations (P.I. §244: "How do words refer to sensations?").
Sensations can exist independently of language to refer to them (P.I. §256: "—But suppose I didn’t have any natural expression for the sensation, but only had the sensation?").
Thus it can be possible for one to experience the cogito, the mere act of awareness, independently of language. The point of the cogito is its self-evidence, prior to language even entering the picture as a sign standing for or referring to the self-evident sensation of conscious awareness.