This is a fallacious line of reasoning that is incredibly vulnerable to slippery slope and moving goalposts.

While the saying is true that "your right to swing your arm ends where my nose begins," that argument falls apart when you try to apply it beyond one-on-one interactions with direct consequences into group generalities and indirect/indeterminable consequences, such as "creating a more dangerous environment."

It does not allow you to prohibit things, nor compel others, out of a feared or perceived potential for harm. That reasoning leads straight to 1984's "freedom is slavery."