> but most landings failed higher above ground
AFAIK there have only been two attempts by private companies.
SpaceIL's crash in 2019 seems to have been caused by a premature engine shutdown due to a faulty IMU [1].
iSpace's attempt seemed to go well until the very end, when contact with the lander was lost. Their status update from April 26 suggests that also their engine may have shut down prematurely: "engineers monitored the estimated remaining propellant reached at the lower threshold and shortly afterward the descent speed rapidly increased" [2].
If this turns out to be the case, the ultimate culprit may be faulty altimetry.
> These days we even have lasers to instantly detect distance to the ground that could be incorporated real-time to thruster output.
Yes, but lasers don't work well through dust clouds, and throttling rocket engines is hard. You can't smoothly reduce thrust all the way to 0; even with a sophisticated design, there is a cutoff at something like 20% [3]. So you need to plan ahead, shut down the engine before touchdown and hope that you don't end up hitting a stray boulder - which is all it might take to tip over and bury your antennas in regolith.
[1] https://spacenews.com/spaceil-says-chain-of-events-led-to-cr...
[2] https://ispace-inc.com/news-en/?p=4655
[3] https://en.wikipedia.org/wiki/Rocket_engine#Throttling