Sinhala does not have the structure of Dravidian languages. However, the alphabet used to write Sinhala has a shared origin with the alphabets used to write Dravidian languages. Also due to long coexistence with Tamil in the island, Sinhala burrows lot of loanwords from Tamil.

I am Sinhalese who grew up in Sri Lanka. As such I am exposed to Tamil, but when I watch Tamil TV, at best I can pick few words here and there. But when I watch Hindi show, I understand a lot more due to the structure is very similar to that of Sinhala.

I don't understand sinhala but if it is classified as Indo Aryan language and not Dravidian, then that is the reason.

There will be always be some influence and loanwords from neighbouring languages but comparative linguistics can tell origin / family of the language pretty accurately.