While I mostly agree, there still should be a way in Swift to make a collection of virtual dispatch objects, including hash-based collections. But this can be done in a separate library since this isn't a language restriction.
Might try to write one tomorrow, really.