Normally you only want concurrency control within certain boundaries.
By figuring out the minimum amount transaction and concurrency boundaries you can inch out quite a bit of performance.
It's doable to repartition to 100 partitions or more, but you basically need to replay the work kept in the log based on 10 partitions onto the new 100 partitions, and that operation gets more expensive over time. Then of course you're basically stuck again once your traffic increases to a high enough level that the original problem returns. If the unit of horizontal scaling is the partition, but the partition count can't be easily changed, consumers eventually lose their horizontal scalability in Kafka, from my perspective.
Logical partitions are always handled by the same physical partition. But physical partitions can handle multiple logical partitions.