The Turtles thing predates James (and was probably originally rocks): https://en.wikipedia.org/wiki/Turtles_all_the_way_down
I've also heard that the anecdote (mentioned in TPR) didn't involve William James and that it was Bert Russell talking to the lady.