I suspect many people conflate scholarship with clergy because that was the way things worked in the middle ages in Europe: if you wanted to have enough time for scholarly pursuits the clergy was probably the best place to be. Once it became possible to make a career as a scientist this practice changed but the association between clergy and scholarship still lives in the public perception.