If you find something that just captures your imagination then go with the flow.
By coincidence it was a video linked to here on HackerNews that gave me the impetus for the tech project on my last career break. It was the video of the guy who fixed a ten year old bug in Guitar Hero without the source code. It turned out to be a memory allocation failure. I found that so interesting I spent quite a bit of my 6 month career break investigating memory allocators, memory debug tools, and even wrote a couple of my own experimental allocators - absolutely fascinating stuff. Really miss those days!
How do you find code you want to write?
If you have a day job and need an escape, then don't bother with making it robust, polished and feature complete - just have fun doing it regardless of what the world may think of it, only thing that matters is what you think and what you get out of it.
If you don't have a day job and your next project is going to be your source of income, then I think my original statement applies.
What many do is turn the fun side project into a day job by focusing all their attention on it once they see potential in it and then start polishing, making it robust and feature complete.