I would say, just start! It's indispensable for exploratory data analysis for sure.
Usually I am looking less at analyzing reams of data, though, and more at the relations between objects. ("What kind of object is User.things? It's a query set, but do we ever have more than one? Do we have any users with no Thing associated?") While notebooks can surely express that, I rarely feel like I need to re-create the asking of the question. Perhaps it is that I have not yet graduated far enough in my explorations of data that a Notebook is the best tool.
I'm going to look more closely at them this week, though. Thank you for the encouragement to do so.
I keep these things in a dated "Notebook" file, like "Notebook/2013-07", just to hold all my one-offs.
Why do I keep them? Well, often it takes a while for me to set up the particular environment I need. If there's a problem with the calculation for X, I need to load the right data, set up the context for X, run through the individual bits until I see what the issue is.
I save the ipython notebook where I go through all that, and then if I need to go back to the calculation, for example to demonstrate to someone else my example that something has gone wrong, then I can just load up the notebook. If something is supposed to have been fixed, I can load up the notebook and repeat the calculation and see if it works now.
Unlike emacs org-mode, I can easily save the graphs as part of this. ("Look, if I plot the distribution, there's this big extra lump over there. Where's that coming from?")
If I go back to a file, it typically takes me a bit of thought to remember exactly what I was doing there. Once I figure it out again, I add sentence or two of markdown.
It's a useful thing to have around.
I might be misunderstanding what you're wanting here, but are you looking for something like (R example):
#+begin_src R :exports both :results graphics :file graph.png
...
#+end_src