Granted, I write microcontroller firmware for a living these days, but I still use RPis and Arduinos for proof-of-concept work because of the simplicity of the toolchains.
Not to say that your original comment didn't have merit; the beauty of hobby projects is that you can completely skip the last 20%.
A full OS brings a whole lot of software complexity and places greater demands on the processor, which means greater power draw, greater hardware expense (because you need a beefier system just to run the OS) and reduced reliability.
There is certainly a role for systems like a R-Pi that have a complete OS on them, but there is also a real place for lighter systems.
Unlikely. The marginal cost is so low that a "beefier" but standard processor that's widely used is likely to be cheaper than a lower-powered but less-popular processor. And using a standard system that everyone else does is more likely to have the bugs ironed out.
I don't like it, but realistically unless you really need low power it's cheaper and simpler to get an extremely standard off-the-shelf RPi like system and slap whatever on it. The savings you can shave off by going smaller just aren't worth the effort.
And having a full OS running means that more code is running. More code means more potential points of failure.
Of course, if your project is something that needs a larger processor or more advanced software, then what you say can be very true.
Also, if your skillset is more in line with programming larger systems than smaller ones, the development time will be shorter for you by going with a full OS.
Additionally, Linux has available a multitude of very rich in-band debugging and logging capabilities that help you understand and mitigate common problems— whereas a bad interrupt or peripheral configuration on some microcontroller could have you banging your head again a GDB frontend or shelling out thousands for a tool like Tracealyzer to tell you what on earth is going on with it.