The public is thinking "Why did he have an affair with a business partner? (who also happened to be a spy). That is a massive conflict of interest. Was he putting Nvidia's interests first, or his own?
I am not convinced that he needs to resign, but describing this as a speed bump is utterly delusional.