Reasons are that office people want to be around people, which would be missing half the time, leading to them leaving to a more office friendly company eventually.
At the other end, the remote workers will be treated to a business as usual environment where things happen more face to face and synchronously, leaving them out of touch with the work environment, which will inevitably lead to them leaving for a more remote friendly company.
Pick one and stick with it.