Similar devices like those from Ledger or Trezor have trusted inputs (buttons) and/or trusted outputs (screen) which make the whole thing work.