Disease was important but there was a large technological and cultural gap too (e.g. the Incan didn't fight at night!).
It seems like the Incans were overconfident and didn't expect a surprise attack (didn't have their weapons, only a small retinue around the rule in ceremonial garb instead of armor), and then the 8000 warriors were outside and didn't even attempt to fight the Spaniards because they were so demoralized.
Spain conquered and held the whole Incan empire with 168 men, also fomenting smaller factions and internal feuds etc.
The scale of this is absolutely insane.
To be fair to the Inca, I didn't expect the night–vision–equipped Spanish Inquisition, either.