In the demo video, they mention that they used a lot of footage from self-driving cars to produce that.

One thing I noticed is there are no pedestrians and cars in those scenes. So they must do a lot of work to filter them out by combining a lot of footage. Therefore, it likely can't be used (as-is) on the street view dataset...

Yes!!!! I always get frustrated when ever it does the weird stretch thing as soon as you move around in street view. Just jump to the next frame of you have to.

Can be done yes. Is it the best dataset to do what you describe, no.