The flicker was due to oscillation, due to a pole too close to the right half plane. I was about to stick a bigger capacitor in the feedback compensation circuit, but a bit of maths told me that that would make the problem worse (and probably blowing up my LED's) - so instead I used a smaller resistor value, and yay - perfect flicker free light!