Arabic is a hell of language to learn. Even as an Egyptian born in a semi-Arabic speaking house...I'm finding Arabic really difficult to learn.
You're not alone. The US Defense Language Institute teaches many languages, mostly to native English speakers. Of the languages they teach, Arabic takes the longest.
It's a shame the article didn't list the number of weeks the DLI spends on each of the languages. It "only" covers a few dozen languages, but course length would be an excellent proxy for language difficulty for English speakers.