First, as always you get better by practicing. Second, you don't need to be witty or spontaneous at all. You can think of a nice thing beforehand ( * ) and just use it when there is an opportunity. You can also make sure that the possible downside is low by e.g. talking when it is clear that you don't have to spend half an hour sitting next to another in, then, awkward silence. Some sort of reason for initiating a conversation also helps, in my opinion, so that talking about something doesn't come as much as a surprise (e.g. talking about some thing in view or an event that just happened).

Simple example: "Your groceries look a lot healthier than mine" in the queue at the super market started nice conversations for me.

It was clear that we'd just have stand next to each other for two minutes if the other didn't want to engage in conversation. It was a sort of compliment (you are shopping better things that I do) and it had a at least some sort of reason (there are real goods on the conveyor belt and they differ in healthiness).

Oh and really: limiting the risk by choosing temporary situations really helps to avoid the fear of having to spend time in awkward silence if the attempt at chatting fails.

( * ) Just anything can work here: "And I wondered whether I should bring an umbrella today" to someone actually having one; "Looks interesting" to your neighbour at a street performance; ...