As an example of how this might work, suppose you want a data structure that provides an ordered collection of items, with the ability to insert, randomly access, remove, and iterate over them. The compiler/library/runtime might be able to supply any of a binary tree, a linked list, or an array. You, the programmer, have some idea of the distribution of the data that will be stored in the collection (e.g. the collection will contain integers between a specific range) and the operations your program will perform on it (e.g. the collection will never be iterated over, insertions will be twice as common as removals, only the first/last elements will ever be accessed at any given time). The compiler knows the performance characteristics of each of the operations over various distributions of data (e.g. linked list insertion has a cost of O(1), random access has a cost of O(n), removal has a cost of O(1)). The language would allow the programmer to provide the compiler with hints about the expected distributions of data and operations, and the compiler would use this to pick the optimal data structure for the use case. Another example might be the compiler picking a sorting algorithm automatically, based on the expected distribution of collection sizes and data (e.g. automatically pick insertion sort of the collections are typically small and already partially sorted, but pick merge sort if they are typically large and unsorted).
It would also be interesting if this language/runtime could enable sampling of statistics on the actual observed distributions of data/operations under production workloads, and make it easy to update the programmer's specified distributions to get better performance with minimal effort. Or if the compiler could be given the performance characteristics of the target hardware as well, which could impact its choices. Maybe the program will be reading from an SSD, making the overhead of caching something in memory no longer worth it?
More broadly, it would be very cool if program semantics and performance could be decoupled entirely, and the compiler could have the ultimate freedom to pick the optimal data structures and algorithms so long as the program's observed behavior remained the same.