Hacker News Reader

Comment by rowanseymour | Hacker News Reader

I adore the Django ORM but as listed under cons... it makes it very hard to avoid accidental N+1 queries, and they don't seem interested in addressing this (https://code.djangoproject.com/ticket/30874). Yes lazy loading is neat when you're messing around on the shell, but you should absolutely not be leaning on it in production at any kind of scale. So instead you have to use unit tests to hopefully catch any N+1 queries.

View on HN

Just to say there are libraries to help you find n+1 queries too (when your code is running).

I use https://pypi.org/project/django-nplusone/ for instance. Sentry also warns of these by the way.

> they don't seem interested in addressing this (https://code.djangoproject.com/ticket/30874).

Actually it seems they are: https://code.djangoproject.com/ticket/22492#comment:11

It is a huge red flag to me when people recommend Django ORM without admitting the pitfalls of the Active Record pattern. This is a problem which simlpy doesn't need to exist and it's wasting computing and dev resources time and time again.

What would be the best way?

If it were up to me there'd be a way to completely disable it globally for all models. Let me explicitly enable it when I'm just shelling around or checking results in unit tests. It is not a feature for production environments.

Would auto-joins be better in production envs?

Laravel does this (or at least has a config for it). You can disable lazy loading in non-production environments which will throw fatal errors whenever a model is lazy loaded.

The nice part is, in prod, that code _allows_ lazy loading since application stability is more important. Hopefully at that point you have a good performance monitoring tool that will alert you to that problem. Laravel also has hooks to more granularly fire events about lazy loading if you want to roll your own notification solution.

So I disagree with your assertion. Lazy loading is ONLY for production. While it is possible that such a feature could potentially bring down your DB server and therefor your app in general, if the feature is turned off locally, then hopefully you’re catching it well ahead of time. And if you’re running small app with no users, meh. For large apps, hopefully your team’s standards are enough to keep the monsters at bay.

Lazy loading is a sharp knife that requires skill to use appropriately.

Hmm I would call lazy loading the opposite of a sharp knife that requires skill.. you're not being explicit about database work. You're relying on the framework to bail you out where you've forgotten a fetch. And by default Django will do that silently so missing prefetches easily go undiscovered.

Static lint wouldn't be a bad way

There's a Django app (that I forget the name of) that disables lazy loading in templates, thus requiring you to load everything explicitly in the view.

I think there's a few 3rd party solutions like https://github.com/charettes/django-seal but I don't love the idea of using something that I assume is monkey patching Django code.

There is an ongoing effort to include back the equivalent in core [here](https://github.com/django/django/pull/17554)

Not really.

.prefetch_related (for whole models) and annotate/Subquery (for fields or aggregates) have existed for many years, alongside a pile of aggregate functions which have existed forever and have improved.

Whether or not you use the tools given to you is a sign of developer quality and experience. You can easily avoid n+1 99% of the time but you have to appreciate the problem exists.

I think the Django project demanding some competence is okay.

Not sure you understand my complaint because forcing the user to track down N+1 queries to know where they're missing .prefetch_related is the problem. I don't want to fix something 99% of the time. I want my ORM to enforce correctness 100% of the time.

Reply on news.ycombinator.com