A utility patent covers a new invented item or process. They are the hardest and most expensive to get. A design patent protects the way something looks. It's easier and cheaper to get. If you wanted bang/buck you might consider a design patent on your UX if there is a strong connection between UX and utility.
A DIY "provisional patent application" can be filed for a utility patent. This is basically creating a description of your invention, and submitting to the USPTO who will file it without looking at it. You then have 12 months to create a legitimate patent application referencing this provisional application and you get retroactive protection. Of course the provisional application is only as good as you make it. And if you rely on it when filing a patent, and it has some flaws, you get the flaws. You can shoot yourself in the foot this way.
When/If you create a manual, it is automatically protected by copyright law. You can go further and file a copyright registration for the document. This can serve as prior art and make it more challenging for others to patent your work.