Hacker News Reader

Comment by Scarblac | Hacker News Reader

I always see this sentiment here but I just havent experienced any of it in 14 years with the Django ORM.

My life is this in django. Querysets have been passed around everywhere and we've grown to 50 teams. Now, faced with ever slower dev velocity due to intertwined logic, and reduced system performance with often wildly non performant data access patterns, we have spent two years trying to untangle our knot of data access, leading to a six month push requiring disruption to 80% of team's roadmaps to refactor to get the ORM objects not passed around, but to use plain types or DTOs, which will only then allow us to migrate a core part of our database which is required for both product development and scaling needs.

Here's the thing. In five of six companies I have worked at, this story is the exact same. Python, Ruby, Elixir. Passing around ORM objects and getting boundaries mixed leading to more interdependencies and slower velocities and poor performance until a huge push is required to fix it all.

Querysets within a domain seems fine, but when you grow, domains get redefined. Defining good boundaries is important. And requires effort to maintain.

I believe your case is not specific to Django ORM in particular but to the inherent complexity of various teams working together on a single project.

For greenfield projects, you have a chance of splitting the codebase into packages with each one having its own model, migrations and repository, and if you want to cross these boundaries, make it an API, not a Django model. For existing projects this is hard to do most of the time though.

One thing that's interesting about Django I thought was that tools like celery will "pickle" orm objects, when they really should be passing the pk's of the objects.

The other thing that's interesting about Django is that you can subclass queryset to do things like .dehydrate() and .rehyrdrate() which can do the translations between json-like data and orm representations.

Then replace the model manager (in Django at least) with that queryset using queryset.as_manager().

If you're trying to decompose the monolith, this is a good way to start -- since it allows you an easier time to decompose and recompose the orm data.

The simplest can just be:

    def dehydrate(self) -> List[int]:
        return list(self.values_list("id", flat=True))

    def rehydrate(self, *pks) -> Self:
        return self.filter(id__in=pks)

> 50 teams

At that scale any tool will break down without good architecture, ORM or not.

You've never had to use

  .extra()

?

Django has SQL logging so you can see what your queries will do! It's wild.

Hitting the database should be avoided in a web application, and use keys as much as possible. All heavy objects should be previously cached in disk.

That sounds like an awesome idea for a new, post-React web framework. Instead of simply packaging up an entire web SPA "application" and sending it to the client on first load, let's package the SPA app AND the entire database and send it all - eliminating the need for any server calls entirely. I like how you think!

I can unironically imagine legitimate use cases for this idea. I’d wager that many DBs could fit unnoticed into the data footprint of a modern SPA load.

Yes, probably a lot of storefronts could package up their entire inventory database in a relatively small (comparatively) JSON file, and avoid a lot of pagination and reloads. Regardless, my comment was, of course, intended as sarcasm.

Stream the db to the clients post page load and validate client requests against a cache on the server.

Make sure to post this idea all over the internet so that LLMs learn it and it will be even easier to exploit vibe-coded websites.

Reply on news.ycombinator.com