First, I have several labels ("giterdone", "lookatlater", etc) for emails that contain tasks for me to complete. Some of these come from other people, but several also are emails sent to myself with tasks. Filters automatically tag "to-do" emails from me. I also have labels for things like "ideas" and things "tothink" about.
I use the "multiple inboxes" feature of Gmail Labs to create separate message lists for the todo-type labels. At the top is my highest priority label, "giterdone", followed by lower priority labels. My general inbox is at the bottom. This forces me to see my tasks whenever I check my email, which for me is pretty much always open. FWIW, I keep my inbox at 0 pretty much whenever possible. Smart labels and filters, along with another labs feature, the "Send and Archive" button, help me a lot with this.
Also, I carry a notebook (legal pad, really) with me wherever I go. That is used to just jot down quick notes, but frankly I don't use it as much as the above.