FFTW [1] does something like this, not just for parallelism but also for selecting optimal algorithms for a given data set and architecture. It uses "wisdom" to store known-good combinations, falling back on quick heuristics and brief benchmarking when needed. If you're going to do a lot of something you can have it update the wisdom from full benchmarks.