I'd like to learn programming but I have trouble finding the right starting place.There is no right starting place. There are lots of good starting places though. What have you looked at so far? Have you tried anything yet?
What language should I pick up if I am interested in creating a dynamic website?
Expect at least one "use X" answer from fanbois of every language "X" in existence. Ultimately you can use Java, Python, Groovy, Scala, Ruby, Clojure, Haskell, Erlang, Fortran, or COBOL; whatever floats your boat. But if you're looking for a recommendation for a language that's particularly good for "Getting started," I'll throw out Groovy, Python and Ruby as good choices, especially if your interest is in building web-apps. You'll also want to learn some Javascript in either case.
How can one get started learning something like Python or JQuery?
Buy a book titled something like "Learning Python" or "JQuery in Action." Google "Python tutorial" or "JQuery tutorial." Go to the python.org website and/or the jquery.org website and browse the docs... look for links like "tutorial" or "Getting started." Start writing code. Get stuck, Google for answers... get really stuck, post questions on Stackoverflow or on relevant mailing lists. Pick a project to build and start building it... iterate over those last few steps. Find a user's group for the language you pick in your area. Attend the meetings and "hack nights" (if they do something like that.) Write more code. Look for interesting videos on youtube, watch those. The Google Tech Talks series offers all sorts of stuff. Write up a blog post when you learn something cool or implement something interesting. Fork interesting projects on Github and play around with them. Make some minor improvement or add some new feature. Release an open-source project on Github.
If you get that far, you should be good to go. Never stop learning though.
Edit:
Also, read this: http://www.catb.org/~esr/faqs/hacker-howto.html
You might also want to browse through some of pg's essays, if you haven't already:
http://www.paulgraham.com/articles.html