The REPL in Clojure (and other Lisps) is far more tightly integrated with your editor environment. When I wrote python, I would do what you're describing: test something in (i)python, write some code, rinse, repeat. With Clojure, I Write said experiments in the same place my code already is (and will end up). Making that feedback loop 10x tighter makes a world of difference in UX: I don't think it helps insight to equate "typing things in irb" with "mashing C-x C-e until the expr does what you want".
(I have no idea if an equivalent workflow exists in Elxir, but I've never seen it been done that way: just like people use ipython, they use iex in my experience.)