While this sounds fine in principle, it soon gets messy. A configuration change may require everything to be rebuilt. But if you rename or delete a source file it didn't track how that ended up in the output directory so you end up with a slow accumulation of crud. It turns out to be fairly easy to confuse all the incremental logic and end up with messy builds. Add poor decisions like using the datestamp of items in the output directory to build the sitemap (they don't set output directory datestamp to that of the input items) and you get important failings.
In my opinion, incremental builds should give the same results as full clean builds or you have a fragile non-repeatable build - something that is very undesirable. I wrote a bit more at http://www.rogerbinns.com/blog/on-nikola.html