Last time I profiled something graphically intensive on the pi 4, I found that touching any of the draw state or uniforms between draw calls would trigger an expensive re-validation on the whole pipeline. 50% of my cpu time was spent in the driver.
Of course, the app I was profiling was pushing quite a lot of uniforms per draw call. Your workload might be different, so profile before doing anything drastic.
I recommend building a version of mesa with symbols so you can see exactly where CPU time is spent.
BTW, try prepending `mesa_glthread=true` to your command to see if that gives you a free speed boost.
I don't know if these extensions are available on Pi driver, though.
The problem with graphics programming is that really you need to learn two things simultaneously: how 3d graphics are rendered, and how to use a GPU api. They can both be a significant thing to learn alone, and the complexity of cramming the two into your brain together makes things hard.