Two things.
1. The decision making process here was unsound. You need to follow the one, some, all approach of rolling out changes. Jumping in on multiple thousands of users was irresponsible.
2. Slack should provide some guidance earlier in the process about user limits. They are known for their friendly UI/UX. An email to the admin saying: "Hey, we've noticed you reached 50% of our maximum users for your instance, are you sure you are on the right path?" would have gone a long way.