Comment by realdavidops | Hacker News Reader

Comment by realdavidops | Hacker News Reader

Parent Full thread

The biggest thing we encountered was related actually to our initial deployment with active directory. This made logins slow, but actually found we could remove the requirement for Active Directory.

It is super heavily based on Wildfly, and if you're not using a tool like docker, it can be kind-of a burden. It runs decently well in standalone mode, but we ended up using the docker container's clustering with Kubernetes service discovery helping to find the other nodes to achieve a clustered deployment.

Outside of that is has been extremely stable, we use Kubernetes deployment mechanism along with a correctly defined readiness check to allow us to seamlessly upgrade, and we've gone from 4.3.0.Final to 7.0.1 in production without any problems. We haven't upgraded to 8 or 9 yet as we're actually working on some new frontend UI changes we wanted to get out the door with the release.

I'm curious how you manage upgrades. I am in the process of rolling Keycloak out to production now and the only thing I don't quite grok is how to do zero-downtime upgrades. It seems like the upgrade may make backwards-incompatible changes the DB schema. Do you replicate the entire database for the upgraded environment?

When using a K8s cluster with the helm chart [1], it's actually the stateful set that takes care to the update.

When the first replica restart, Keycloak makes the updates to the database itself. Sometimes rolling back to a previous version can break. They do not hold the reverse of the database version [2].

I believe the reason behind the STS (StatefulSet) is so the cache have the time to spread among the replicas as it get upgraded.

[1]: https://github.com/codecentric/helm-charts/tree/master/chart... [2]: https://www.keycloak.org/docs/9.0/upgrading/

We schedule a downtime window during upgrade, but typically see no frontend impact to the core service, SSO for end users. We snapshot the DB for rollback if needed as the migrations are not reversible.

Our actual DB size is pretty small so these are very non-intensive tasks.

May I know how many replicas and CACHE_OWNERS do you have?

Reply on news.ycombinator.com