This occurs because in "urban canyons", meaning streets with tall high-rises or sky scrapers, there is little or no line of sight to GNSS satellites (GNSS being the generic term for all satellite positioning systems, not just the American GPS). Consequently, what your phone picks up are signals reflected off of buildings, which exaggerates the distance between you and the satellites, and causes the positioning solution to be pushed away- onto the other side of the street or another city block.
One way Google/Android is tackling this is by using Google's trove of 3D building data, the same that is rendered in Google Earth when you use it in 3D. Your phone uses the building data to correct for reflections. Read on here (and note the authour!): https://android-developers.googleblog.com/2020/12/improving-...
And, the device can use various filters and smoothers to minimize sudden jumps, and normally does, but there are edge-cases (for example, an app may be requesting pure "unfiltered" GNSS location returned directly from the GNSS chipset, hence the jumps). But rest assured, we are working on this issue.
Anyway, thanks for the feedback. We're always doing our best to improve location accuracy and reliability for billions of users in all scenarios and environments, and it's no trivial feat!