The main things I've learned is that indexing is incredibly important. Number your pages, and keep at least one page in the front of the journal for referencing what pages belong to what category. Don't overthink the categories, sometimes broad is good. Most often I just use "Random thoughts/writing" or "programming".
It might seem odd to keep an actual, physical paper journal in this day and age. But for some reason, the tactile nature of it just feels right. I have more motivation to pop open my paper journal and physically write some things than I do to pop open VIM, or Evernote, or Simplenote, or whatever. Even on a phone or a tablet. (especially so, in fact) On top of that, there's less of a chance of a paper journal being stolen, you never have to worry about internet connectivity (Evernote, non-premium), or battery life, or having to fight a multitasking system... the list goes on. I've had times where I just want to take a quick note and I end up fighting something auto-opening, or already being open, or waiting for a new note, that I just get frustrated and end up back with real paper. I haven't found anything that is as flexible or works as well to date. (I also HEARTILY recommend you get either blank paper, or graph paper. particularly graph paper. I have a huge notebook I carry around of graph paper, and it has proven invaluable for so many things. from diagramming software code flow, to diagramming an interface, or even just plain writing notes. this is what I use: http://www.jetpens.com/Maruman-Mnemosyne-Imagination-Noteboo... It's good for carrying in a bag/backpack for serious needs, not as great for a journal. I tend to have both with me)