If you host SVG file on the same domain as the main site, I can give a direct link to the SVG file and your browser will execute arbitrary JavaScript in that context.
I think in that case setting the CSP sandbox header to "allow-scripts" would have the same effect as the iframe sandbox attribute and allow script execution in the SVG but prevent same-origin access and various other things.