It's a "well defined" term in the crackpot side of AI, represented by lesswrong et al, who are self appointed AI experts who have never published any actual peer reviewed research in AI.
Open AI published a number of papers, essentially and implicitly supporting the position of the AI inexperienced founders. They try to highlight AI safety aspects quite frequently. But I would not take it too seriously, just like you wouldn't take a Facebook employees endorsement of Zucks vision.
If you are familiar with ML, you would know that L2 norms are a key optimization metric in ML. L2 norms are basically Pythagorean hypotenuse computations in high dimensions. I will take AI safety research seriously when they start discussing the risk factors associated with Pythagoras theorem and how we as a society must deal with it before we become a slave to L2 norm driven super intelligence.
If you want to really talk about human risk from AI, Tesla's self driving killing machines are already here along with ML guided drone bombers. It's no different from Therac 25. If we are not discussing these and creating a course about super intelligence, it's just mental masturbation.
What I have seen in LessWrong is lots extrapolations upon extrapolations with worse case scenarios just to say there is nothing we can do. That seems to take away from any serious research that might exist since they are probably the best known community associated with AI Alignment.
And no peer-reviewed research, apart from the dozens of widely-cited academic papers on this? All of these are just from one lab, not even including groups like CHAI: https://www.anthropic.com/#papers
Which of these papers track AGI Safety and are also published in peer reviewed journals like SIGKDD, NeurIPS, AAAI etc?