The main downside of Python compared to, Ruby, for example, is that Guido made some serious mistakes in the initial design and in the process of fixing them added a lot of extra complexity to the language. Python doesn't cleanly "fit in the head" the way Ruby or JavaScript do, at least for me.
But instead of wasting time micro-optimizing in such a narrow language space, pick Python or Ruby or whatever and spend your extra cycles on something different enough to be worthwhile like Go or Haskell.