Back then, the conventional wisdom was that eventually all passenger travel would be supersonic. Boeing put their best folks on the SST project, and the 747 was a stopgap afterthought.
Turned out that fuel efficiency was way more important than speed.
The 777 had already made the 747 largely obsolete, and the kings of long haul flights going forward will be Boeing 787-9 and Airbus A350-900. With probably a mix of both in most long haul fleets.
It's surprising that Airbus did hit the jackpot with its A330, seemingly by accident, and yet seems to have been unable to foresee that it really was the future. Boeing built a better A330, called 787, and even though the A350 is a success, simply by virtue of being a larger plane that goes further it will probably not sell as much as the 787.
Now we have twin engines that can at least do the Atlantic trip just as well, allowing all manner of smaller airlines to offer trips from a multitude of airports.
Technically they could but regulations didn't allow it at the time: 1953 FAA regulations required that twin-engined aircrafts always be within 60mn of an airport, and ICAO had slightly less restrictive limits of 90mn. This meant some routes were entirely unfeasible, or the flight path required to keep within 60~90mn of an airport at all time made it uneconomically unfeasible.
3- and 4-engined aircrafts were not affected by these rules (starting in '64 for the 3-engined ones).
The rules started being relaxed in the mid-80s with the introduction of ETOPS ratings (initially 120 and 180, now up to 370).
The A380 was a bet that the hub and spoke model (where passengers travel to an airline's hub in small planes and onwards from there in larger ones like the A380) would continue to rule the industry.
Instead, what happened was low cost carriers flying passenger directly to the destinations. It appears that passengers prefers that, which is why the B797 and A350 exists.
There could have been, but Airbus shied-away from a really radical design like a blended-wing-body and stuck to traditional tube-and-wings design, just on a larger scale.
There's not much margin for efficiency improvement in that case other than begging the engine manufacturers to work more magic.