I do not use Chrome and my windowserver is using 3GB after a month of uptime. I feel this is not a M1/Apple issue to be honest.