Pg_upgrade [0] is an official part of postgres and does the binary inplace upgrade for you. You should obviously test before running in production, but it has worked perfectly for us when upgrading a 10+TB cluster from pg11 to pg13
Also it looks like this is not new https://github.com/docker-library/postgres/issues/37
Notsomuch for production. I know... folks do it. But I wouldn't run anything on it that I wouldn't do w/ sqlite.
Also, in general, that persistent storage is (generally) not locally attached, which means that (some) file system interactions may have unexpected semantics. Now, with a DB, you're probably only going to have the PV attached on a single worker node, so most of that will actually just work.
Debian even has special custom helper scripts for doing it. So just build your container with both postgres versions?