The populating objects part doesn't seem to be the hardest part. The impedance mismatch with ORM is trying to merge the concepts of "result sets" that are not "table rows" to objects in such a way that you can interact with them on a per-table row basis. It's easy to say that each row in the users table represents a user so each row is a distinct object. It's harder when you want to get data out of the database using a single query that hits multiple tables and returns a joined result set that doesn't map to the already established object hierarchy, an object hierarchy that is based on the normalized schema. Denormalizing only helps with some parts of this.

Writing the naive ORM implementation that tries to model a relational schema as a sets of nested objects is doable, but it becomes difficult to get decent performance out of it, since the kinds of queries that map rows to objects are not usually the kinds of queries that are performant in SQL.