Block sizes will supposedly be constrained by bandwidth / propagation times. SegWit will allow more transactions to occur off chain, reducing the need for larger blocks.
They seem complementary to me.
Unlimited block size? There are many reasons that's a very bad idea but here are a few:
1. It would put immense pressure on the network in terms of latency between miners, leading to less stable mining, a higher rate of reorganizations, and a massive advantage for the larger miners, resulting in increased mining centralization.
2. It would result in a substantial increase in the amount of bandwidth that each node has to handle. Modest increases are OK but unlimited blocks means the vast majority of nodes will die off. Remember that nodes have to check blocks as they come in, so this would be an insane DDoS vector. Imagine putting a funnel in someone's mouth instead of a straw and being able to force whatever you want in there.
3. It would result in a substantial increase in the amount of data that each node has to store. Remember that in Bitcoin 100% of data must be stored by 100% of nodes. Imagine "unlimited emails" with Gmail meant that you had to store all of the emails in the world for EVERYONE. Not a good idea.
I.e., if you were planning out how much disk space you needed years in advance, you would have to increase that figure non-negligibly if SegWit activates.
Propagation times have been addressed by multiple technologies, including FIBRE (http://bitcoinfibre.org/).
They explicitly want prices to be high/number of transactions to be limited (think, supply and demand) .
This is because transaction "price" is equivalent to "amount of money spent on securing the blockchain".
On-chain, actually. :-)