This isn't a cause of performance problems you're seeing. What's most likely happening is that the overall size of your code has increased a lot over time, causing performance issues due to issues that have existed for a long time, but weren't being felt yet.
The issue that's most directly related to this is here: https://github.com/dotnet/roslyn/issues/40300
However, immutable vs. mutable isn't related to the issue I linked. It's just about keeping more data around than is (perhaps) necessary. You'd see the same issue with a mutable AST. If you're curious about specific work that's being tracked you can use this label: https://github.com/dotnet/roslyn/issues?q=is%3Aopen+is%3Aiss...
And if you submit reports via the VS Report a Problem tool, with the option to collect a diagnostic trace, you'll generate exactly the data needed to fix issues that you're facing. The team is very keen on addressing performance problems, especially if there's diagnostic data that can pinpoint the source of a problem.