The whole point of the thought experiment is that not everyone agrees that pulling the lever to kill the one man is murder and thus immoral.