You could imagine a technically advanced society using it as a sort of penal death sentence. "You were not safe around our people, so we are sending you to the end of time."
The one time it was used as a punishment it was not sending them to the end of time, but leaving them in the now while everyone else jumped ahead.
Going back to your original point it's a little different than sending him to the end of the universe, they sent him forward in time, but they actually had no idea how far or where there were sending him.
If something can be "killed" by a dude with an axe, it ain't a world-ending superpower for me. I get that Alduin could harass the known world by eating people or burn down villages if left unchecked, but "end of the world" for me is kinda different scale.