Another class of randomness is that which is used to randomize the layout of shared libraries, stacks, etc. --- address space layout randomization (ALSR). If someone is able to guess the randomness used by ASLR, then they will be able to more easily exploit stack overrun attacks, since they won't need to guess where the stack is, and where various bits of executable segments might end up in the address space's layout.
Another case of randomness is to create crypto keys; either long-term keys such as RSA/DSA keys, or symmetric session keys. If someone screws this up, that's when the "bad guy" (in this case, people are worried about the NSA being the bad guy) can get access to encrypted information.
It is only the first two use cases where we use RDRAND without doing any further post-processing. These are cases where the failure of the RNG is not catastrophic, and/or performance is extremely critical.
We do not use RDRAND without first mixing it with other bits of randomness gather in the system for anything that is emitted via /dev/random or /dev/urandom, because we know that this is used for session keys and for long-term RSA/DSA keys.
The bigger problem, and it's one that we worry a huge amount about it, is the embedded ARM use cases which do not have RDRAND, and for which there is precious little randomness available when the system is first initialized --- and oh, did I mention that this is when long-term secrets such as SSH and x.509 keys tend to be generated in things like printers and embedded/mobile devices and when they are first unwrapped and plugged in, when the amount of entropy gathered by the entropy pool is usually close to zero? What we desperately need to do is to require that all such devices have a hardware random number generator --- but the problem is that there are product managers who are trying to shave fractions of a penny off of the BOM cost, and those folks are clueless about the difference between cost and value as far as high-quality random number generators are concerned.
What if the RNG has been compromised by the NSA? Well, that's where you need to mix in other sources of randomness into the entropy pool. The password used by the user when he or she first logs into an android device, for example. Screen digitizer input from the user while they are first going through the setup process. In the case of a consumer grade wireless router, it could sniff the network for a while and use packet inter-arrival times and mix that into the entropy pool. Yes, someone who is on the home network at that time will know those numbers, but hopefully someone who is in a position to spy on those numbers, isn't also going to have access at the same time the super-secret NSA key used to gimmick the RDRAND instruction (assuming that is gimmicked, which is effectively impossible for us to prove or disprove.) But then again, your wireless router isn't going to have access to unencrypted plaintext which is critical --- if you're sending anything out your wireless network without first encrypting it first, I would hope that you would consider it completely bare and exposed!
If you are super paranoid, you'll need to find a hardware random generator which you've built yourself --- and hopefully you are competent enough to actually build a real HWRNG, and not something which is sampling 60 Hz hum (or 50 Hz hum if you are in Europe :-), and mix that into the entropy pool as well. In that case, even if the Intel RDRAND is compromised six ways from Sunday, the NSA won't have access to the output from the HWRNG --- and if it turns out you were incompetent and your HWRNG is bogus, at least RDRAND is also getting mixed into the entropy pool.
And if I were in China, I'd use a hardware chip built in China for the RNG, and combine that with an Intel chip. That way even if the HWRNG chip is compromised by the MSS, and even if RDRAND is compromised by the NSA, the combination is hopefully stronger since presumably (hopefully!) it's unlikely that the MSS and the NSA are collaborating with each other at that deep a level. Ultimately, of course, if you don't trust Intel, you don't trust the silicon fab, etc., then you'll have to build your own computer from scratch, write your own compiler from scratch, etc.
(MIT CS undergrads used to have all of that knowledge, starting with building a computer out of TTL chips and how to build a Scheme interpreter from machine code, etc. But not any more, alas. Now they learn Python and it's assumed that it's impossible to understand the entire software stack, let alone the entire hardware stack, so you don't even try to teach it. But that's another rant....)