That used to happen occasionally to me as well in earlier macOS versions. Didn't have to do any rearranging since Mojave, I think, definitely not on Catalina.
I'm thinking of flipping my monitor upside down so I'll never accidentally brush that area while picking up something on the table.