Of course there are different shades of white, what I am saying is that the color of the sun during the day is what people most would classify as "white" and not as "yellow" if it the color was printed on a sheet of paper.
If the sun color was actually yellow, people wouldn't be able to tell the difference between a white object and yellow object under the sun light (they would both appear yellow). And clearly they are.
On the other hand a white object doesn't get any "whiter" under another kind of light than it appears under direct sun light, this again is because the sun light is white, and not yellow.
Of course there is some subjectivity here, but it is a misconception to call the sun light yellow. For example, this "bright sun" RGB [1] color, is clearly not the color of direct sun light. Hopefully, you will agree with this.
[1] https://www.colorcombos.com/colors/FCD440