On autoloading | Hacker News Reader

On autoloading

erock.io

55 pointsqudat39 comments

So glad someone wrote this. In PHP - I absolutely hate having to work around autoloaders in third party APIs. I love structuring my imports exactly around the directory structure I design; some project-specific code in /services/ can always call something in ../util/ or a static in ../const/ or a pseudo-global function in ../fwk/ just as I, the designer, intended. And if one of those should break as I structure a new project, any old IDE will notice and highlight it. If something has to be put in some weird directory structure, doesn't matter; I don't consider these imports to be anti-DRY, they're the linkages. You don't want to leave them out of the code.

DRY is for logic, not linkage. Linkage is a necessary part of sharding your logic into intelligible and discreet modules, packages or classes. The linkages between those must be obvious for anyone who comes along and reads your code later.

I'm not sure what you are talking about here.

In PHP, everything can be configured with composer, PHP's defacto package manager, including all your directories and their namespaces.

> I absolutely hate having to work around autoloaders in third party APIs.

If your "third party API" aren't using composer themselves, a defacto standard in PHP, then they are not professionally written.

PHP doesn't have modules or any form of modularity or packages, unlike Ruby. PHP only has direct file references from other files (include,require...) and SPL autoloading which is just a way to automate direct file reference (through namespaces or other techniques).

I'm not sure what PHP autoloading has to do with any of this article at first place.

Ruby doesn't have modules or packages either. The require method just reads and executes a Ruby source file, changing global interpreter state.

> some project-specific code in /services/

> or a static in ../const/

> or a pseudo-global function in ../fwk/

For the love of God or Atheismo, stop making these kinds of ass backwards structures the rest of us have to deal with when a legacy project gets dropped on our desk..

There's an established pattern that works across all projects, psr-4, use it.

I don't really see the practical difference between what I'm saying and doing it with Namespace\Subnamespace\ClassFile. The code I write anyway is PHPDoc'd and easily linkable as long as it's kept in the designed file hierarchy. [edit: Yes, I use folders and not normally namespaces for organization in my PHP projects, in part because many of them have been steadily maintained since PHP 4].

What I'm objecting to are "autoloaders" that allow "packages" to register in a way that makes it difficult to follow code. Things like Square's API which includes at least a dozen PHP files called things like "Autoloader.php", "autoload.php", "bootstrap.php", etc, each with their own unique twist on if (file_exists($file)) requre($file).

Steve Yegge had a series of essays where he discussed two flavours of programmers. "liberal", which e.g. likes lower-friction dynamic languages, prefers magic to boilerplate; and "conservative", which e.g. would prefer to have bugs known at compile time, prefers boilerplate to magic. https://gist.github.com/mrnugget/49ad3ee4043c746e42187e2820d...

Autoloading sounds like a feature which reduces the friction while iterating on the program (when the mental model of the program is fresh), but adds a lot of friction if you need to maintain the program and are unfamiliar with the codebase.

It feels like the points cited in the article in favour of autoloading are "liberal" ("nicer DX to just have all the classes available everywhere"), and the counterpoints are all "conservative" ("code should be readable/maintainable").

Putting it in liberal/conservative terms moves it to a simple matter of opinion. But most code spends most of its life maintenance. Would anyone argue against the importance of maintenance?

I think it inevitably comes down to trade-offs.

If you can write highly maintainable code without paying the cost of slower speed, surely that's a winning strategy.

But I suspect an increase in maintainability means "more friction to change". (e.g. I'd expect adding documentation and unit tests is to take more time in the short term than just writing the code without documentation and unit tests).

this problem is precisely what tooling is for. golang for example solves the explicit import problem quite nicely with almost no friction outside of ambiguous cases, which you'd need to handle explicitly anyways.

I think golang makes for an interesting example:

Your program won't compile if you have unused imports, or even unused variables.

Which is low-friction for maintenance. You're not going to have to worry if deleting something is unsafe.

But this strong-arm implementation of a lint means that iterating on code meets high friction. It slows down developer iterations as you have to comment/uncomment in order to compile. -- "But if you use some tool it's not so bad" is advice I've been given, which strikes me as the same as "automatic loading isn't so bad if you're using an IDE".

Having used autoloading in PHP it feels very similar to working with Java/Kotlin. With PHP there are community standards that are follow almost everywhere, you know your dependency code is in the vendor dir and then you know that code will follow a naming convention so the code is really easy to find. As the author of the post said Jetbrains have solved this problem and you can jump to the definition of any class really easily. Their issue is they know there is a tool that does the job but refuse to use it. This seems to me like complaining that using a screwdriver to hammer in nails is a bad idea.

I used to love Python's dynamic typing, and magic, until I started working on build process critical Python written by other people - and it wasn't even a large codebase, it was about 1500 lines across five files.

My least favourite part - dicts of dicts of dicts parsed from JSON, so good luck knowing what conf["foo"]["bla"]["bob"] was without a lot of code reading, good luck figuring out what other properties conf["foo"]["bla"] had, and good luck figuring out if those properties always existed.

In the end I introduced Pydantic to give the JSON objects a known structure with validation, and that dramatically eased the maintenance friction in one foul swoop (and bringing in Pydantic exposed properties that everyone was setting religiously but never using) - instead of a dict of dicts, it was a now a dict of app name to BuildConfig.

Bringing in mypy and gradually introducing type annotations took more effort, but once it was done, the codebase was far easier to read and understand, large sections of dead code were easily identified, and runtime bugs became far less common.

In other words, I removed all the dynamic bits of Python.

That said, when I'm working with Python libs that do terrible things (like deciding if they're running in a multiprocessing context by checking for an ENV var when the module is imported(!)) I do appreciate how Python's magic lets me reach in and monkey patch terrible things away - although I feel a bit icky doing so.

I spend a lot of time in crufty old C++, and I love how clean Python syntax looks. But then I wonder if it's trying to be too clean, with all those pesky explicit types scrubbed away. Which is odd, because the first time I read "explicit is better than implicit" was in "The Zen of Python", and I always very much agreed.

Python these days is sprouting type annotations all over the show, the benefits are too good to ignore.

And if a given library doesn't use type annotations, people will often create type stubs for it that mypy can consume.

https://github.com/python/typeshed/tree/master/stubs

TBH, Python generics are rather clunky, although I think they're probably still going to have cleaner type signatures than C++

This seems to be the story for many dynamic scripting languages. TypeScript obviously, but also Ruby and PHP. Lisp folks seem to be the most steadfast in preferring dynamic typing. But even that’s not universal.

Liberal and conservative is a terrible analogy, it's about being explicit or implicit. And in programming explicit is always better.

Nope.

with PHP autoloading you can still detect issues at compile/build time

I don't know 'bout y'all, but to me, one of those "flavors" is clearly "right" and the other is clearly "wrong."

I think larger lesson here is that magic may look enticing, especially to the beginner, but it might cost more over long term than a little more involved but non-magical process.

It is irritating to work with a system that from time to time requires you many orders of magnitude more time to do a relatively simple task.

I prefer a little bit of upkeep (which I can usually amortize/automate) of a simpler system because it lets me be more predictable with my estimates. I can plan better because there is very little surprise lurking that could suddenly cost me an additional day of debugging.

And the cost of that upkeep might be very overestimated. For examples, I never felt that adding imports to a file is costing me anything. My IDE usually handles this for me. Or I might just copy something from somewhere else I had gotten something working in the past.

While I agree that OP's complaint is valid, I think they're wrong in blaming the autoloader for it.

If you were to remove the autoloader and to explictly use `require` in your app, when you `require a` if `a` also `require b`, you implicitly get access to `b` at the same time.

So the problem isn't the autoloader, but Ruby's and PHP single global namespace.

If anything the autoloader makes it easier to locate the source, because it forces you to follow a strict naming convention. If you see `Foo::Bar`, you know it's defined in either `foo/bar.rb` or `foo.rb`, and it's easy to locate with your editor fuzzy finder.

Another proof is that Django doesn't have this problem even thought it has an autoloader (IIRC).

I'd be very much in favor of a Node or Python like import system for Ruby, but there's nothing wrong with the autoloader.

There are not a lot of things from JavaScript that I would love to see ported to Ruby, but ES6 imports is one of them.

I’m not a Ruby dev so I can’t comment on that. But I can say that ESM truly is great, although it’s something many don’t appreciate until it finally clicks. I didn’t see the benefit over say CJS for a long time. It was the static analysis potential that finally made it seem so valuable to me. But being URL based (including data URLs!), idempotent, and extensible (eg type assertions) are also huge benefits.

> So the problem isn't the autoloader, but Ruby's and PHP single global namespace.

Interesting point and I agree! The thing I like about python and javascript's import system is you import the object that you end of using, creating a namespace as a variable, defined at the top of every source file. This lends itself to traceability which was my primary argument in the post.

I can see how I conflated the global namespace in ruby with automatically loading classes.

Cheers!

It's crazy to me that this is even controversial. The fact that I have no idea where code is coming from in a rails codebase is by far the most aggravating thing about the whole ecosystem.

It's crazy to me that people think it's crazy to have reasonable opinions on why autoloading/global class hierarchies is awesome. I'll go into open revolt if Ruby ever starts enforcing explicit imports. The Rails Way is what I love so much about being a Rubyist: convention over configuration.

Author here. I agree with you that it's a matter of personal preference. One of the reasons why I decided to write a blog post about this is because my team in general really likes Ruby and everything it brings to the table (autoloading included) -- so venting to them would not be as satisfying.

As a long-time Rubyist, I freaking love autoloading and have gone to great lengths to implement it in new frameworks I've developed after using Rails for many years. I find it fatiguing and ofttimes infuriating whenever I have to wade into a TypeScript project and there's 30 lines of import statements before I can get to any actual code. Any syntactical "noise" in a programming language bothers me to no end…I prefer to write expressive syntax which is as close to thoughts in the English language as possible.

I'm not saying the OP is "wrong"—that is, they're more than welcome to use whatever language they like and write all the import statements they prefer. I'm just glad I can choose to do away with all those distractions.

I'm a bit confused by the terminology here. I thought "autoloading" referred to deferring the loading of a constant until it is first used (as described in this post[0]). But the author seems to be talking about loading code without an "import/use/require" statement.

Are those two different meanings for "autoloading"? Or do the two go together in Ruby? (As is probably obvious, I'm not a Ruby dev)

[0]: http://www.rubyinside.com/ruby-techniques-revealed-autoload-...

Yeah, there's the autoloading vs. eager loading question of when a constant first triggers loading a code file, but I think in a broader sense autoloading is used to refer to the idea that your code files will just automatically get loaded in a project as long as you use the right naming conventions.

While I also quite dislike the autoloading behavior of rails, in practice, I haven't found it to be abused too much in large rails projects. So let's say we take away autoloading, it is likely that mostly models code will need to deal with circular dependencies.

Meanwhile, in golang projects, within-package circular dependencies are happening blatantly everywhere, and somehow I don't see much complain about it.

This is where Python shine I belive. Its import system is amazing. We can import the whole file or just particular funcion/method in that file. We can import relative with `.` and `..` and even `...`

And we can "import as" to avoid name conflict.

Using import together Mock library we can even swap out(Mock) particular imported item during unit test.

If one thing I would love to see is having that import system in Python ported to Ruby.

Granted that the way Ruby currently works also expose a few functionality for doing DSL easiser.

Similar point to this is wildcard imports in Java.

Previously I prefer import bla.* because it's shorter and neater, but as the code grows, more naming conflict and more difficult to find where's where, I prefer explicit import. I can configure my IDE to collapse the import lines anyway.

Yeah, most code styles in JVM code bases I've worked in (Kotlin and Scala included) prohibit wildcard imports, reviewing PRs is a lot harder when you're playing "Where did the function/class come from".

The big problem doesn't exist in Java, as it's being compiled, and all classes used are fully qualified in the bytecode.

In source-based languages, if a library adds some class, you'll get into some trouble. In Java, this will also happen at source level, but results in not being able to compile.

I am not familiar with Ruby autoloading, but PHP autoloading is (in the vast majority of cases) not that magical.

PHP autoloading (with composer, the de facto standard) is basically just a one to one mapping of namespaces to file names, which have matching class names.

Now there can be more magical autoloading, if it is manually defined, but that's not very common compared to directly using `include` or `require`.

Having never used Ruby or even heard of this feature before, is this not a massive supply chain-style attack surface? If I define methods/classes with the same names as common other packages, is there any way the user can tell which one they’re using if I get them to install my package?

A feature of importing is namespace management, if two dependencies provide SomeClass.

>Grep codebase class SomeClass; If nothing was found then

This one line kinda says it all.

If you reach the point where you grep your whole project for SomeClass and nothing comes up, everything else is just going to be a bandaid.

Reply on news.ycombinator.com