Clearly, doesn't work in all cases but often enough, it's not only simpler to set up but runs much faster besides.
We offer a new metric for big data platforms, COST, or the Configuration that Outperforms a Single Thread. The COST of a given platform for a given problem is the hardware configuration required before the platform outperforms a competent single-threaded implementation. COST weighs a system’s scalability against the overheads introduced by the system, and indicates the actual performance gains of the system, without rewarding systems that bring substantial but parallelizable overheads.For example:
* A single GPU machine is often more efficient than multiple cards due to bus i/o bandwidth constraints, lack of software support, and obscure driver failure modes
* A single CPU machine is often more efficient than multi-chip solutions due to memory access latency and caching issues
* A sequential drive data access machine is often more efficient than arrays due to pipelined memory cache layout and baked access latency
* A single CPU core bound process is often more efficient due to avoiding threading and or mailbox overheads
Thus, if a problem is truly separable, than it is sometimes wiser to bind n-many slow jobs to n-many cores rather than parallelize 1 job to try to get it done more quickly.
I found this rather surprising in processing large video media data sets.
Some may disagree, but they are mostly 15% to 30% more wrong for truly separable tasks. YMMV =3
Examples Google BigQuery, AWS Athena or AWS serverless EMR.
These cloud providers always have a bunch of CPUs idle, so they even provide the CPU cost for free, it is just the loading of data that costs.
AWS (still) costs per data loaded though.