Google is the absolute worst with this. I have to remember to put ?hl=en to the end of every URL for every product they own.
If the Accept-Language header matches "en-*", ignore that. Instead use geoip to try to guess a language.
If the Accept-Language header is something else, use that.
There's probably some cookie/Google-ID-based heuristics going on too.
Please just give us a UI language selector. Don't leave us with something half-baked that works great for 90% of the users and leaves the remaining people perplexed. People do know multiple languages, and want to search for stuff in other languages too, sometimes. (Yes, it does seem like the UI language is influencing the search results, too.)