It's definitely worth comparing, if you just want the cheapest bare flight I've found booking through the cheapest agency on comparison sites can be cheaper. However you will usually end up with Ryanair level of service where you can't change anything and are the lowest priority if you the flight is full.

A couple of years ago a friend booked an intercontinental flight, they saved €20 by booking through an agency, but later wanted to cancel the flight due to illness. If they'd booked even the cheapest option direct, they could have paid to do so, but as it was through an agency they couldn't.

On the other hand, if something goes wrong, or you wish to cancel or change your flights, you'll have to deal with the online travel agency, and won't be able to deal with the airline.