Not irrelevant at all. The case of the relative pronoun is determined by its role in the relative clause, not by the role of the relative clause in the sentence as a whole.
See:
- H. W. Fowler, A Dictionary of Modern English Usage
- Eric Partridge, Usage and Abusage
- Ernest Gowers, The Complete Plain Words
who are unanimous on this point.