It's also dangerous, and not in obvious ways. It's like the opposite of a cauldron of 400f peanut oil --- if that oil could also quickly asphyxiate you. And your intuition for how to mitigate the risk is bad; for instance: gloves would be a mistake. And that's just for handling it; freeze something the wrong way and give it to a friend to ingest and you could perforate their alimentary canal.
Should people be able to play with LN? Absolutely.
Should someone start a company and sell super-fun mini-dewars to people as a novelty item, like the Soda Stream of chilling? If they did, a lot of people would get hurt.
I am fine with it being a little bit of a hassle to get LN, and I am fine with it being a little bit of a hassle to buy a pretty boring desk toy. The CPSC appeared to have been fine with that too; its problem wasn't with you owning little magnets, but with companies selling them as novelty stress relieving desk devices to rekindle your sense of childlike wonder and also make excellent refrigerator art (all things on Zen Magnets current page).
I don't think HN's take on magnets and the CPSC is very sophisticated or interesting or really even all that well informed.