What causes the restriction of /24, rather than being able to route individual addresses?
Routers can only handle so many routes, so their operators set floor values for size of route they'll generally accept.
I'm not counting gold-plated routers because I see no reason to force ISPs to buy those.
A top end router today can handle ~million routes. You’re talking a four orders of magnitude increase.
Do we really want IP space carved up into smaller and smaller bites? Do we really want even more and smaller entities advertising routes into BGP? The big players have enough troubles with it - we don't need small businesses with a /28 playing in those waters.
There's a lot of value in owning your user-facing IPs and being able to move them around as needed, but a lot of services don't really have that many user facing IPs, so utilization is low if they get a /24 per location.
On the converse - I think of all the smaller businesses I worked around that really wanted to get their own portable IP block and ASN because they thought it made them serious ... but just had no business playing in that space.
Otherwise, just move to IPv6 or something and fix the problem in a saner way.
So if an ip isn't in the cache it'll go through a slow path instead and get cached.
Why this wouldn't be possible in hardware is beyond me.
Routing on commodity hardware is becoming a thing too with DPDK and OVS, you can now do 200gbps on a single x86 box.
Huge numbers of routes are simply not an issue these days, unless you have truly ancient crappy equipment (maybe it was second hand for example).