If you s/graphics/3d graphics does that still hold true?
Per-pixel matmul (which is what you really need for anything resembling GPGPU) came with Shader Model 2.0, circa 2002; Radeon 9700, the GeForce FX series and the likes. CUDA didn't exist (nor really any other form of compute shaders), but you could wrangle it with pixel shaders, and some of us did.
Granted, if you didn't have the “squared blend” extension, it would be an approximation, but still a pretty convincing one.