While a lot of users avoid hot load and restart/replace nodes instead, I don't think many restart/replace the whole cluster.
While a lot of users avoid hot load and restart/replace nodes instead, I don't think many restart/replace the whole cluster.
Exactly. Blue-green deploys could get seriously messed up if the static typing system doesn't strongly think about this.
I think the open question is whether strong, _static_ type system can handle it at all.
On the server side: what happens if a process geta a message it doesn't handle? Either it will crash (and be restarted by supervision, hopefully), it will receive and discard it (possibly logging it), or it will ignore it and leave it in the message queue (not great for long term memory use).
On the client side: what happens if the process sends a message that is ignored or crashes the receiver? The client will timeout, and won't know if the work was done or not; best to bubble up the error, perhaps by crashing (and maybe restarting by supervision).
If you're ok with these 'worst cases', you don't have to proactively deep think. The proper deployment strategy is clear: deploy servers that handle new messages first, then deploy clients that send new messages, but if you mess the order, you'll be within your failure model. It gets more tricky when there's a difference in data types and a piece of data could be used by old code, then new code, then old code, of course.
Of course, the actual worst cases are more fun, but less likely to happen. You could find a bug that crashes BEAM or something; I don't think that would be easily found by sending a unexpected message, but maybe (depends on what you do with unexpected messages).
But if your static typing just says yeah I dunno, it could crash, I don't think you've gained much from the exercise. I'm probably biased against static typing, but if it can model the system and provide useful insights that are accurate on the system, that sounds helpful; if it can't model the system though...