And if your user visits more than one page on your site, you've successfully killed asset caching. More data for your server to send _and_ a slower experience for the user, losses all round
True, there is no caching. But that's still a technique worthy to use when they are very small (hundreds of bytes), or when the images should never be cached (eg. low-resolution real time status graphs). In these cases, they reduce the amount of data on the wire (no extra HTTP requests) and reduce page load time.
Not if you put the data URIs in a stylesheet shared across all pages: that gets cached nicely.