> But you make the assumption that the data can be generated by your model

Yes

> you also make assumptions about the parameter distribution

No

> your variance estimate only holds asymptotically

Don't follow