I'm not sure why the performance benefits hit a limit with 10 workers on a 64-core machine. It doesn't look like there should be too much communication complexity given largely disjoint workloads.
Are we hitting memory/IO bottlenecks?
Are we hitting memory/IO bottlenecks?
A particularly nasty behavior occurs when you get into a situation where given a set of concurrent transactions, all of them must be aborted. While 2PL is not very popular these days, it does have the advantage that you can often entirely eliminate deadlock via carefully written queries. Then you just have to deal with Amdahl’s law and you won’t run into quite as catastrophic performance given high concurrency.
Andy Pavlo wrote a paper looking at this comparing different concurrency control strategies under very high levels of concurrency.
The commit phase of optimistic locking is cheap except when it's a do-over, and by necessity it causes the second action to use the resource twice, the second time sequentially to the first action.
I think you mean his paper "This is the Best Paper Ever on In-Memory Multi-Version Concurrency Control" which is available here: http://www.cs.cmu.edu/~pavlo/papers/p781-wu.pdf
By the way, his courses are really excellent. I highly recommend subscribing to the CMU Database Group channel on YouTube.
Back in the AMD Q6600 days, in my early experimentation in multi-threading I found that my MT quicksort routine hit this limit at about 50k elements. At that point further subdivision just made it go slower. The exact limit depends on how expensive the comparison operator is and the synchronization overhead for the thread pool/work queue, but there will be a limit.
Not sure if this is what's going on here, but is one aspect at least.
I don’t know what the blog post was trying to prove, to be honest. If I wanted to display the benefits of parallelism, i’d have at least tried to make the test heavily CPU constrained, given that that’s the entire point of being able to go parallel. I’d also have shown I/O performance and waits, along with CPU utilisation.
One theoretical solution is to have millions of parallel queries, and rather than doing random reads to satisfy them, just read the whole table sequentially.
Query latency is high, but throughout is then unconstrained.
OK, good point, I believe this is called a carousel read (spent 5 mins looking for a link, sorry no luck. I know MSSQL does a slightly fancier version of it).
I think the original article was unclear in what it was trying for, actual performance or exposition.
Edit: could have utterly sworn it was a carousel read, but no cigar. Here it is https://www.mssqltips.com/sqlservertip/4472/sql-server-enter...