Same intuition. We run ~all our back on firebase cloud functions since Beta in 2017 and cold start has never been such a problem.
We're not doing anything crazy, it's just a basic CRUD application with minimal data (entire DB is less than 100MB at this point). And yet we're seeing constant, constant lags of several seconds for almost every single request. I can't explain it.