The other argument I’m familiar with is longer block propagation times lead to more orphan blocks, which is a centralization pressure.
EDIT: responding to your edits:
> layer 2 payment solution which essentially will force transactions to route through centralized middlemen
1. Layer 2 solutions aren’t necessarily centralized
2. Even if they are all centralized I don’t believe it’s worth sacrificing decentralization in the base layer to support small payments