Within historical context: This was said after Minkowski introduced the geometry for special relativity, which can be visualized on the `spacetime diagram`, now recognized as the vastly simpler and more efficient approach than the wall-of-text approach. Einstein initially had some difficulty, that was when he said this, but he caught up quickly.

>Strangely enough no personal contacts resulted between his teacher of mathematics, Hermann Minkowski, and Einstein. When, later on, Minkowski built up the special theory of relativity into his 'world-geometry', Einstein said on one occasion: 'Since the mathematicians have invaded the theory of relativity, I do not understand it myself any more'. But soon thereafter, at the time of the conception of the general theory of relativity, he readily acknowledged the indispensability of the four-dimensional scheme of Minkowski Einstein, philosopher-scientist, Schilpp

And later Einstein expressed admiration for geometry again and again

>This problem remained insoluble to me until 1912, when I suddenly realized that Gauss's theory of surfaces holds the key for unlocking this mystery. I realized that Gauss's surface coordinates had a profound significance. However, I did not know at that time that Riemann had studied the foundations of geometry in an even more profound way. Einstein, Kyoto Lecture, 1922

>I admire the elegance of your method of computation; it must be nice to ride through these fields upon the horse of true mathematics while the like of us have to make our way laboriously on foot Hentschel (1998). The Collected Papers of Albert Einstein, Vol. 8 (English): The Berlin Years: Correspondence, 1914-1918