There's a better separation of concerns that makes it easier to customize specific pieces of the project.

For example, OpenLayers divides fetching data (Source), parsing data (Format) and interacting with data (Layer) into individual pieces. So it's relatively straightforward to handle a case where someone decided to go invent their own data format without having to subclass half the system and patch it.

For smaller projects leaflet is significantly smaller and simpler, and definitely is the right approach. But if you need to do "a lot" with a mapping system OL3 probably has it built in and supported instead of maybe available as an extension.