Thanks for asking! I have several different but interconnected processes:
1. I have a regular writing practice where I just open up something (Evernote, Byword, notepad, whatever) and start writing about whatever comes to mind. This is primarily just to develop the habit of writing. I mainly do this with my 1,000,000 word writing project [1].
2. I do content marketing for work, which is much more systematic and planned out– I work backwards from common queries and search terms. The most successful post I've ever written, though, was something that I had written out of curiosity [2].
3. Sometimes 'inspiration' hits: somebody asks the right question, or something happens and I feel a need to respond– and everything just comes right out of my subconscious, almost as if it were already written. But when I examine this, it's clear that I was already ruminating about it for a long time before.
> It seems like there is a paradox, you can't force great things to happen with objectives and planning.
Not exactly a paradox. The objectives and planning aren't about making great things happen; they're about creating a stable context in which great things happen. It's about putting in the hours of practice so that you can improvise freely when the moment calls for it.
> So are you personally concerned about whether a specific piece of writing is objectively great, or are you more interested in your ability to write well in general?
I think I cycle between the two, back and forth, depending on what I'm doing. If I'm working on something with a sense of purpose, I want it to be great. Otherwise I'm happy just working on my craft on a day to day basis.
_____
I agree...the best ideas "come in on little cat feet"...
I bet the people who are constantly pumping out works that, on their own, would be considered achievements—the Picassos and Frank Zappas and Stephen Kings of the world—have a pretty different view on how much of creating something good is inspiration vs. a schlepp.
12 months from now you won't be able to tell what you wrote while inspired and what while schlepping.
Also, how will you ever improve in something if you only do it while inspired? And how will your inspired work be any good if you have no practice and experience to build upon?
Yes and no. Once you succeeded in creating a masterpiece it can become incredibly hard to ever create something worthwhile again because success (especially when rewarded by the markets) can screw up your mind for good. You loose your sense of wonder, you feel the pressure to have success again and so on. It could very well be that you just create more of the same or get worse due to this. I think bands that are considered one hit wonders might provide a good example.
It's a bit like an optimization algorithm that found a local optimum but can't escape its current neighborhood.
I've been writing almost every day for probably about a decade now. The reality of it is a little more nuanced that that.
It's more like... 20% of the time you were inspired, you'll realize that what you produced was still crap.
Similarly, about 20% of the time you were schlepping, you'll realize it was actually pretty good.
And about maybe 20-30% of the time, you'll find yourself getting 'inspired' midway through a schlep. And as you do it more and more often, you start subconsciously preparing things in anticipation of the work.
I'll take those chances. Especially if it increases my chance of getting started at all.
Remember, the only option with 0% chance of great is not doing anything.
You're just playing around with stuff, create a lot of drafts and play with them if you feel that they're fun. And then, suddenly, one of the drafts doesn't get dull; surprisingly, it gets more and more interesting the more things you add to it.
And then you realize that this one is actually done.
A software's features could always be improved, a book can always be revised further, a song could always be tweaked or rearranged, you can always knock the sand castle down and try to make the parapets straighter this time.
Most of my projects have a wishlist of features, 80-90% of them eventually get cut when I decided I was finally, finally going to release the damn thing.