Lambda only has default support for the LTS releases of .NET, which is 2.1, 3.1 and the upcoming (November 2021) .NET 6.
The only way to run .NET 5 in a Lambda that I know of would be custom runtimes or containers. Is that what do you mean by “switched to using Docker images”?
This article describes the containerized performance of .NET in Lambda and the cold starts are dramatically (~4x) worse.
https://www.kloia.com/blog/aws-lambda-container-image-.net-b...