Are there disadvantages in allowing users arbitrarily long passwords (or with a very high limit, e.g. 256 bytes)? No.

Are there advantages? Yes: the longer the password is, the stronger it is. It doesn't matter if there's a specific point after which cracking takes time till the death of the universe. So there's no point in imposing a limit.

Besides, too many websites limit password length to ridiculous lengths like 12.