While time travel is scientifically possible, the Fermi paradox ends any civilization long before they are able to make it a reality