Maybe a factor in the model spitting out these absurdities, is that it has consumed a ton of regular old SEO absurdities already and does it best to match the genre.

For instance I found out recently, if I search for Karigasniemi, Google recommends a site with the "top 10 hotels in Karigasniemi". It's a tiny, tiny border village in the middle of nowhere. Needless to say it doesn't have 10 hotels.

Are we assuming that the current models aren't already engaging in this practice? In a world where ads dominate the internet and revenue flows from commercial interests, it's reasonable to assume that these models might already be skewed towards those who have contributed the most financially. After all, advertisements essentially govern the online landscape, so why would this scenario be any different?

Well, yes. This is what was alluded to about the absolute breaking of shared reality that genAI allows. That's not even touching on the fact that the party that has the most control over a popular used model could filter "undesirable people" out of places they don't want to see such folks.

Coming up next: free datasets, with sponsored preferences.