It reminds me of ML where the bottleneck is often the bandwith between the GPUs.
And as expected working side by side / living in the same house dramatically increased productivity. I couldn't imagine a remote employee being as motivated.
TL;DR -- startups need a certain workaholic mentality / intensity. In-person work can provide this.
In defense of remote: We didn't have a commute, which saved us at least 1.5 hours a day.