> What alternatives or solutions do you envision for search? How would you build a content heavy site in a way that gives a user confidence when searching for something?
I think a big difference is the move away from folder-based organisational structures. Once upon a time, web sites were collections of HTML pages in a filesystem, and so a hierarchical organisation was natural; people started moving away from just a filesystem (or FTP server) into a CMS, but many older CMSes had a similar focus on folders as an organisational structure. It worked quite well in the hands of competent users, although there were always those people who didn't quite get it (or maybe just had poor attention to detail) and if left unsupervised they'd turn it into a big mess.
But then along came blogs, and suddenly there was a shift to posting content chronologically, with non-hierarchical tags; and as blog software (such as WordPress) came increasingly to be used as a CMS even for non-blog sites, that approach spread to CMSes as well. And I think it makes it harder to logically organise content, and it results in less focus and time spent on content organisation, and the end result is content becomes much harder to find. Blogging also encouraged an approach where people would create a new page rather than editing an existing one, resulting in an intermittent stream of disorganised piecemeal updates in response to new developments, rather than presenting all the information as a cohesive whole.
I think software (especially documentation) is one of the areas where we have this problem the least, because a lot of software people use tools (Git, Jekyll, etc) which still encourage hierarchical organisation. It is really non-software stuff, for example websites of government agencies, university administrations, corporations, etc, which have gone down-hill the most.
> Maybe I'm mentally blocked in thinking of the era of flash intros and marquee tags, but do you have any examples of sites that achieved what you're describing?
I don't remember many websites like that, but certainly a lot of multimedia CD-ROMs in the 1990s had that kind of design, and while there is no reason from a technological perspective why you couldn't replicate that style on the modern web, it just doesn't seem to be very popular.