I'm not sure I entirely follow your comment, however I was merely pointing out that reckoning with parameter uncertainty by "computing or approximating the posterior...", as you said, is not always applicable in probabilistic ML.
An estimator (of a parameter) is a random variable, as it is a function of random variables, however this depends only on the data distribution, there is no other implicit distribution on which it depends.
For instance, the distribution for the maximum likelihood estimator of the mean of a normal distribution is normally distributed, however this does not imply that the mean parameter has a normal prior, it has no prior, as it is a fixed quantity.
Yes
> you also make assumptions about the parameter distribution
No
> your variance estimate only holds asymptotically
Don't follow