Prof. Harvey's claim rings completely true to me. Students understood the syntax quickly, and spent little time on it. It was not a point of frequent confusion. There were plenty of difficult concepts in the course, but the details of the programming language were not, for most students, among them.
Students who already had programming experience when they started the course often had more trouble than inexperienced students, but mostly because they had to unlearn imperative habits since the imperative features of the language, except for I/O, weren't used until late in the course.
SICP covers a huge breadth of material, from basic computational ideas to algorithms and data structures to interpreters and compilers to query languages to concurrency, and does it in an entertaining and challenging way. Even decades later, I find myself pulling ideas from it in my daily programming work.
I worked at Google for almost twelve years, and I can't count the times I found myself muttering, when reading a design document, "I wish this person had read SICP."
I'm certainly biased, but I would encourage anyone who would like to become a better software engineer to read SICP and study it carefully. Take your time with it, but do read it.