I just watched the first half of the talk, and it's pretty interesting. I don't think Adam Neely is actually saying they're the same; he's saying it's interesting to think about their similarity. And what you actually get out of the talk is that his point is that pitch is our perception of frequency over a certain limit, and that rhythm is our perception of frequency under that limit. They're both perceptions of frequency and the ratios of those frequencies are the defining property of both chords and polyrhythms.