Although this is technically true, in practice probabilistic machine learning makes use of "un-priored" parameters all the time.
An estimator (of a parameter) is a random variable, as it is a function of random variables, however this depends only on the data distribution, there is no other implicit distribution on which it depends.
For instance, the distribution for the maximum likelihood estimator of the mean of a normal distribution is normally distributed, however this does not imply that the mean parameter has a normal prior, it has no prior, as it is a fixed quantity.
Yes
> you also make assumptions about the parameter distribution
No
> your variance estimate only holds asymptotically
Don't follow