If you do the phases wrong you will get pulses instead of noise. Obviously if you do the phases right you can do anything, because the Fourier transform is one to one.
Sure for any specific signal x you can craft the corresponding signal X so that ifft(X)==x, but the idea here is to generate a random signal with certain properties (blue spectrum and uniformly-distributed time-domain values). This is the tricky bit.