"Version" include wreaks havoc on case-insensitive filesystems | Hacker News Reader

"Version" include wreaks havoc on case-insensitive filesystems

bugs.llvm.org

62 pointssirn45 comments

If include <version> in angle brackets finds some Autoconf VERSION detritus, it's because the compiler has been wrongly told to look for <...> headers locally. (Probably by the same Autoconf junk.)

There is a (not so) modern (any more) gcc option -iquote to specify a directory to search for #include "..." without affecting #include <...>.

In the TXR project I have a "signal.h" local file:

http://www.kylheku.com/cgit/txr/tree/signal.h

Yet, <signal.h> is also included. Like in this file:

http://www.kylheku.com/cgit/txr/tree/signal.c

which has both #include <signal.h> and #include "signal.h".

This is fine, if you know what you're doing. #include <signal.h> has no business finding that local file, and will not do so, unless someone screws up.

Why is the directory containing VERSION in the include path? Isn't that normally an include/, src/, or similar?

Because config.h, for whatever reason (possibly to make more easily human-editable), is created in the top-level directory.

It's where autoconf, when told to write out a config.h, typically puts the file.

See https://www.gnu.org/software/autoconf/manual/autoconf-2.69/h... for more info.

So, fix autoconf, if you can't retire it. It is actionably hazardous to require that a directory that contains random files be in the include path. But I'll bet it doesn't; it is probably just the lazy choice.

That said, there is no requirement at all that "#include <version>" must pick up a file called "version" (of any case). The compiler could prefer a file called "version.standard-header", if it exists, or "std/version"; or, things in <> brackets could be looked up on a different list, such as one designated by "--isystem" (which does already exist). There doesn't need to be such a file at all -- the standard just says what names must be found to be defined after that point in the code. The compiler can Just Know. Or, it can recognize just Standard header names, and get them somewhere special. Which is why we don't bother about this in committee. Doing the dumb thing is dumb, no matter what we say. Do the smart thing, and trouble evaporates.

I always thought it was mad that compilers shipped with standard header files that had exactly the name as appears in the standard, and that preprocessors shipped that favor or require such. If you need for the files to have exactly those names, define a cpp option naming a file with a list of them, and exactly where each is.

Standards evolve. There's value in keeping things simple and consistent to allow that. So it'd be a cost to have specifically version treated exceptionally here, and it'd be a cost to have system headers treated very differently from other headers (because it makes it harder to extend the system).

So while it makes sense that system headers needn't be files - in principle - it sounds like a pretty big change too with some downsides as well. All that just for "version"?

Not for <version>, or just for <version>. Rather, for being able to designate header names without worry about what files have been shoveled into random directories.

It really is on the compiler implementers for not providing a better accommodation to this very predictable problem.

That said, fixing the autoconf scripts may be the most immediate route to sanity for the "config.h" / <version> case, eliminating the problem at its source.

> for being able to designate header names

In other words, there's no reason why '#include <foo>' should ever be looking in the translation unit's directory; that's what '#include "foo"' is for.

I bet autoconf actually says "-I $top_level" or some such.

What's mad is that -I controls where <> names are searched.

I concur.

-I is a traditional option from Unix compilers with a very silly behavior; but for years now we have had something called -iquote in gcc (and clang).

Before -iquote, there was -I- which is deprecated.

There is value in not telling your compiler to look for all headers in your project root. There is no telling what clash that will create.

Documented, public <> headers include other headers that are not documented. There is no telling what their names are.

sure, no question - but that's out of the standards body control. And even if you don't include your own top-level dir, using a common file name might clash in other dirs too.

Hoping that compilers would not resolve headers via user-specifiable directories sounds absurd to me - every single compiler I know of allows that, not just on unix, so if you want to retire that, that's going to be huge, multi-year effort. No point in considering that hypothetical reality when it comes to meaningful header names unless there is a reasonable likelihood of that happening. Not objecting to such a push; it just sounds wildly out of scope when picking a header file name.

Case-insensitive filesystems cause so much trouble, one could almost mistakenly conclude that causing trouble is their raison d'être.

What troubles do they cause? I might have been lucky, but during my ~10 years on a a mac, I do not remember having had any problems with case sensitivity on a file system level. I might be a special case though, as I always name my files in lower case and with hyphens instead of spaces.

Actually, I kinda like not having the possibility of having both "Downloads" and "downloads" by mistake in my home folder.

Seems like you never had the joy to port some windows-software to linux where the original developers weren't aware of case-sensitive systems. And you never hat to work with buggy windows-applications that randomly rewrite/rename files to all upper-case or something like that.

If Unix were case-insensitive too, none of this would be an issue.

Some MacOS installations are case sensitive, others are not. Sometimes you find software that was written and tested on the case insensitive version that fails on case-sensitive filesystems.

Case insensitive filesystems made more sense decades ago than they do today. Case insensitivity is fundamentally a UX feature that has been implemented in the wrong layer. Rather than the filesystem itself being case insensitive, the search functionality in GUI interfaces to the filesystem should be case insensitive. And furthermore, the user should never be typing out the complete name of any file that already exists, because everything should provide auto-completions of existing filenames.

You want the rest of us to suffer your nerf-fs too? No.

> What troubles do they cause?

For one somewhat recent example: CVE-2014-9390 git: arbitrary command execution vulnerability on case-insensitive file systems. (Also affected mercurial IIRC.)

> What troubles do they cause?

Once, a colleague working on Linux created two files with same name but different case, and committed them to the Git repo, and then I tried pulling it on my Mac. I don't remember exactly what happened next, but I wasted a lot of time trying to sort that mess out.

(There was no real reason for them doing this, it was just a mistake on their part, and they didn't notice they'd done it before committing the files.)

You could argue that this is a problem with case sensitive file systems, not insensitive ones. In fact, this could never happen on a case insensitive file system, as you could never create a file named the same but with a different case.

It seems to me that most problems with case insensitive file systems are rather problems because there are different file systems out there. Just as there always is with "competing" standards.

The whole idea is hopelessly tied to ascii notions of alphabet, where two functions (storage a retrieval with keys, and searching for matching keys with the UI) have been conflated. What is 'case insensitive, case preserving' in Unicode for Korean or Chinese or Arabic? It doesn't remotely make sense to make some keys illegal in the file system layer for reasons of disambiguation.

The worst part is how it doesn't seem to affecr Linux- and Windows- only users.

Let me put it briefly. Much Linux software is multi platform and vice versa; this is how being the "underdog" (quotes cause servers) works. When you look at the transitive closure of the dependency graph, all sorts of case-insensative assumptions will pop up.

I'm on a shop where all developers run Linux on their laptops, but we work on tools and libraries that are supposed to run on macOS for all those other developers. Supporting them is a huge pain.

I like it when apple iOS-ifies macOS making development harder. Hopefully it brings people out of denial.

Aside of caseness there is another a bit similar problem.

Unicode allows different forms that lead to the same representation for the user.

Gödel.txt may be written as - G-\u00F6-del.txt where \u00F6 is "Latin Small Letter O with Diaeresis" or - G-o\0308-del.txt where \u0308 is "Combining Diaeresis"

From user perspective they are indistinguishable.

Traditional Linux filesystems do not normalize this in any way. MacOS filesystems normalize it to decomposed form with "Combining Diaeresis" (which is mad). NTFS does not normalize, but windows apis behave differently - if you create NFC+NFD Gödel.txts under same directory then you will either see both, see one, or even see both, but be able to select only first from system "open file" selector.

Unicode filenames becomes a true bundle of joy when you're dealing with ideologically mixed systems like a unix client and unix server, but with samba in the middle. Boy oh boy do I love that.

> Case-insensitive filesystems cause so much trouble, one could almost mistakenly conclude that causing trouble is their raison d'être.

I too hate windows because of it's case-insensitivity but that's not the problem here. The problem is a messed up Include-Path and/or the shitty naming/placing of C/++-standard-headers.

The problem is the hubris of compiler maintainers.

> windows

It’s amusing and frustrating that you use a lowercase W here, expect everyone to know what you mean, while arguing in favour of case-sensitive filenames - the user-hostile systems which would stubbornly refuse to know what you mean.

I feel like if you choose to run a case insensitive file system, you are on your own to work around it, sort of like the devs working on porting things like PHP or node to Windows. It's expected that on those platforms, you will have more compatibility issues because what you're running is a hack.

What I've never seen is an easy way to convert a case insensitive file system into a valid one after the fact. It would be great to give people an easy way to fix it after the fact.

Other than Mac OSX, are there any other case-insensitive file systems in common use today?

https://www.quora.com/Why-does-OS-X-choose-to-have-a-case-in...

Perhaps Mac users should just apply a patch and let everyone else use case as a differentiation.

FAT is commonly used for removable media.

And what about NTFS?

NTFS is case-sensitive, Windows applies case-insensitive matching for reads by default.

This is a super academic distinction that is confusing discussion more than helping. NTFS as it is used in practice 99.9% of the time is not a case-sensitive filesystem

And a super-relevant distinction for a low-level OS/DB programmer like me.

But not when talking about whether #include <version> is bad!

About NTFS, the less said the better.

("If you can't say something nice ...")

Even Microsoft is abandoning it as a place to keep source code. At least, their own source code. Windos is not built from a tree on NTFS because it's too slow.

NTFS is also case-insensitive though case-maintaining, similar to HFS+/APFS

Not true.

FAT is not a currently maintained filesystem that a significant base of developers, other than people interested in "retrocomputing" use.

Every single pen drive I've bought or been given came pre-formatted with FAT32. Every single SD card I've bought or been given came pre-formatted with FAT32. Like it or not, it's the standard filesystem for removable media up to 32 GiB, and has the advantage that unlike exFAT, it works everywhere.

exFAT is also case insensitive, at least in some implementations.

Microsoft does maintain FAT, because they license it to consumer products companies.

Point is I'm not going to create a FAT partition on my disk then go work out of it.

Maybe you won't, but other people do. Dropbox and similar services have eaten away at the practice, but a lot of people still walk around with flash drives, partition table untouched since they bought it, with all their personal and work files. Not merely just using it to transfer large photographs or pirated movies, but keeping their de facto 'home directory' in that flashdrive. For a few years in the mid to late 00s, it seemed like almost everybody was doing this.

Before that, I knew people who did it with zip drives. And I even once knew a guy who tried it with DVD-RAM, but gave up pretty quick.

Reply on news.ycombinator.com