Certainly, if we took the primary clause of the sentence and substite in any number of pronouns, you'd agree that the objective forms are correct:
The book belongs to whomever. Not "whoever".
The book belongs to her. Not "she".
The book belongs to us. Not "we".
I don't know the English grammatical rule for this situation, but it certainly seems reasonable to say that the dependent clause does not get to dictate the form of an independent clause.