Yes. Raw entropy values need to be first mixed in an entropy pool first in some way that's not easily predictable. Fortuna's entropy pool is a high-quality algorithm / system that happens to be part of a cryptographically secure psueorandom number generator (CSPRNG). The two can be seperated, just as AES-NI instructions have RdSeed and RdRand.

ASCII:

    entropy sources -> entropy pool -> CSPRNG -> ...

Beyond the randomness thing (which you're right about, although NIST is claiming that they're actually random), the problem is that they're not secret.

Given that you'd probably want to avoid using for any crypto use (not just secret parameters) unless you can be sure that using this randomness for a public parameter doesn't break the cryptosystem (or USG isn't in your threat model). Given that few of us are experts in cryptology it would seem like the safe thing to do is to avoid for crypto use at all.