Same for tattoos or car modifications. Yes, there's a lot of signaling within your social strata, but if you're in that strata it's valid to say that some tattoos are better than others.
Also it's okay not be in the loop. Not everyone has to have an opinion on why Leonardo Da Vinci is a better artist than Botticelli.
But the key point you're making - that good art only exists in a specific context - I totally agree with. Good taste/art exist only in relation to a specific culture, and depends on what that culture values (technique, creativity, bold ideas, traditional ideas (for example: pre-romantic period, good composers were seen as empty vessels "pure music" divinely flowed through rather than as geniuses with bold new ideas)). Shostakovich wouldn't have had a chance 100 years prior.
Let's say you are in a museum alone, some of the works presented can resonate and excite you more than the others, but you can't share your excitement with anyone. Where is the social signaling element gone then?
Lets say: Two months later I'll be at a dinner party and over a bowl of Caldo Verde I'll recount an anecdote about the Paul Manship sculpture that I saw that time in the museum on my own, and how it was placed in relation to the other works.
Also, kids are exactly the demographics super susceptible to claim liking or disliking things based on what their friends say about them.
Show a kid 2 paintings they’ve never seen before. Show it to them in isolation. Guess which one they pick?