I'm “still afraid to use spaces in file names” years old
twitter.com
twitter.com
Pro tip: rename your development directory (or even better: the workspace path in CI) to put a space and/or special characters in it.
Forces you to deal with this properly, and immediately ensures that every automated test checks this case without you having to remember every time. Hasn't been particularly inconvenient, since I'm autocompleting it 99% of the time anyway, and I haven't shipped a single path parsing bug since.
> Microsoft intentionally made programs install to C:\Program Files on Windows 95+ to force programmers to deal with spaces in filenames.
one occurrence is enough to make devs care about it
Not least, it makes writing scripts for various shells and getting the quoting rules right an absolute pain as well...
/Users/{root, user0, user1, ... }...
/System/{Logs, Apps/{opt, container, ...}, Temp, Conf ...}...
/Devices/{Mount, sda, sdb, null ...}...
/Boot/...
> the system kept trying to replace folders in their original places.
$ cat ~/.config/user-dirs.dirs
XDG_DESKTOP_DIR="$HOME/"
XDG_DOWNLOAD_DIR="$HOME/Downloads"
XDG_TEMPLATES_DIR="$HOME/"
XDG_PUBLICSHARE_DIR="$HOME/"
XDG_DOCUMENTS_DIR="$HOME/"
XDG_MUSIC_DIR="$HOME/"
XDG_PICTURES_DIR="$HOME/"
XDG_VIDEOS_DIR="$HOME/"(Anyone who tried to package a program that hardcodes the “usual” binary paths know the pain)
Probably the easiest way to experiment these days is to create a VM and make snapshot, then start knocking down walls, just to see when and where the house collapses. Then revert and try something new.
There could be a competition!
So I ran two simple commands:
cd /tmp
rm -fr .*
> When a pattern is used for pathname expansion, the character ``.'' at the start of a name or immediately following a slash must be matched explicitly, unless the shell option dotglob is set. The filenames ``.'' and ``..'' must always be matched explicitly, even if dotglob is set.
This is definitely interesting though, and an improvement I would say
So either those low hanging fruits are higher than they seem, or we're all just a bunch of dwarves.
My bet is on the second option.
Will upstream accept such patches though? Sounds unlikely to me.
/Users -> /home
/System -> /usr
/Data -> /var
/Config -> /etc
/Boot -> /boot
/Ephemeral Temp -> /run
/Persistent Temp -> /tmp
It's not an oxymoron to have files which are temporary but not limited in scope to a single power cycle. For example, you could have a long-running process which you want to be able to resume if it's interrupted; /var/tmp would be an appropriate place for the state. The data is temporary because it will be deleted once the process is finished, but you wouldn't want it wiped out by a system reset. Generally /tmp is cleared at every reset, and is often a tmpfs mount, while files in /var/tmp are automatically cleaned up only when they reach a certain age.
Nothing will immediately break because you linked /var/tmp to /tmp. Whether it causes issues depends on the programs that you (or your users) run and how they make use of /var/tmp. However, if someone did have to restart a long-running process from the beginning because recent state information in /var/tmp was not preserved across a reset, I would say that is a problem with the administration of the system and not the program that stored its state there.
/tmp and friends are poorly named. They really should be /shared or /dmz or /freeforall or something.
* If you need service-specific tmp space use RuntimeDirectory or PrivateTmp if your app is hardcoded to /tmp.
* If you need service-specific persistent data that goes in /var/lib/your-app.
* If you need temp space for your user it's at /var/run/user/your-uid.
* If you need more than one user/service to share files but not everyone then god have mercy on your soul because all options are bad. There sure are a lot of them but none of them are at all satisfying.
Gentoo does, at least by default: https://wiki.gentoo.org/wiki//etc/portage/make.conf#PORTAGE_...
Caps are annoying to type, and difficult to remember (Do You Caps, or Do you caps, or DO YOU CAPS, etc).
Spaces are nuisances that bring no benefit. At best we should use non-breaking space for filenames, but that would be even more atrocious.
It's frustrating having to spend time to decide whether I should install a program in /var or /opt or /usr. What do they even mean!
So, I disagree with this convention altogether and use /apps or ~/apps now.
You can do `configure --prefix=/Program\ Files/<app>` if you want.
Offering too many similar choices leads to mess. There's nothing fundamentally different between using one or more of these options and using the only option, except that in the second case there isn't any opportunity to make mess.
> You can do `configure --prefix=/Program\ Files/<app>` if you want.
Thanks for the tip! Can't do that with distro repo software though :-/
What does that even mean? When you install something, you put it where you want it.
If you don’t like where your distribution puts files, choose a different one. Not all of them use the same convention.
I think the best model I've seen for consistent binary locations is the 'Applications' folder in Mac OS X, but it fails as well by retaining the /usr/bin elsewhere.
Yes, the Linux/POSIX model is confusing, but the split is to segregate administrative domains:
- / and /usr are the domain of the distribution. As a user, you should never install there. The administrative group is root.
- /usr/local is the domain of the machine admin. If the machine is yours to manage, you can install software there. The administrative group is staff.
- /opt/$vendor is the domain of third-party vendors. Each vendor (like Steam, Eclipse, Arduino Studio) can get its own subdirectory and its own administrative user group.
How would you achieve the same on Windows? How do you make sure the Adobe updater can only install new versions of CS, but not surreptitiously install a new (free!) spyware package under C:\Windows? How would you allow certain power users to share one Google Chrome installation, allow each of them to update it, but not let them install additional software system-wide?
Also, as mentioned by the siblings to thia comment, the 'mess' has a purpose, and is less messy than it appears.
Want to manually install something? Into /usr/local it goes. Done.
The only way to handle this that I've been really impressed with is Mac's "Applications" folder. Unfortunately, I dislike most other things about Mac.
/opt is for standalone packages, so if it’s a single file, no.
/bin is only for stuff needed on single user mode, so probably not (unless that’s what the binary is for.
/usr/bin is going to typically contain files installed by your package manager and should probably be left unaltered by human hands.
The deps I would assume /usr/local/lib but it hasn’t ever come up for me.
This is one of the big advantages of using a pre-made advantages of using a ready-made Linux distribution: beyond the convenience of having an installer or easy to install packages, you get some assurance that the system as a whole has been thoughtfully put together.
Arch Linux for example symlinks /bin and /sbin to /usr/bin and /lib to /usr/lib among other things.
If you expect other users to be able to execute the program, then you should put it in either `/usr/bin` or `/usr/local/bin`, depending on whether the former is already being used by a package manager. `/opt` is generally for self-contained software that doesn't play nicely with the rest of the system, but might still be installable through the default package manager.
Can't remember the last time I had a problem arranging this. If using autotools, which covers 95+% of stuff, it's usually a question of something like "./configure --prefix=$HOME/usr".
(If I want to share stuff between users, /usr/local/ is of course a better place. macOS is a bit more restrictive, so I have a separate user for this, whose /usr folder is readable by everybody.)
On freedesktop systems there's the ~/.local directory which is supposed to be a mirror of the file system hierarchy. Seems like a good place for bin, lib, include directories.
- Things installed in /, if it’s different from /usr, are generally not to be touched;
- Things installed in /usr are under the distro’s purview or otherwise under a package manager, any modifications are on pain of confusing it;
- Things installed in /usr/local are under the admin’s purview and unmanaged one-offs, there are always some but overuse will lead to anarchy;
- Things installed in /opt are for whatever is so foreign and hopeless in not conforming to the usual factoring that you just give up and put it in its own little padded cell (hello, Mathematica);
- Everything is generally configured using files in /etc, possibly with the exception of some of the special snowflakes in /opt; the package manager will put config files meant to be edited there and expect the admin to merge any changes in manually, and sometimes put default settings meant to be overridden by them in /usr/share (see below)—both approaches can be problematic, but the difficulty is with migrating configuration in general, not the FHS as such.
There used to be additional hierarchies like /usr/X11R6, and even a /usr/etc on some (non-Linux?) systems, but AFAIU everyone agrees their existence makes no sense (anymore?), so much that even FHS doesn’t lower itself to permitting them.
The distinction between / and /usr might appear to be pointless as well, and nowadays it might be (some distros symlink them together), but previously (especially before initial ramdisks were widespread) stuff in / was whatever was needed to bring up the system enough that it could netmount a shared /usr.
Inside each of /, /usr and /usr/local there is bin for things that are supposed to be directly executable, whether binary or a script and all in a single place; share and lib for other portable and non-portable (usually but not necessarily text and binary) shared files, respectively, segregated by application or purpose; finally, due to the dominance of C ABIs and APIs on Unices, the top level of lib also hosts C and C++ library files and there’s an additional directory called include for the headers required to use them. Some people also felt that putting auxiliary executables (things like cc1, the first pass of the C compiler) inside lib was awkward so they created libexec for that purpose, but I don’t think the distinction turned out to be particularly useful so not all distros maintain it.
That’s it, basically. There are subtler but logical points (files vs subdiretories in /etc) and things people haven’t found an obviously superior solution for (multilib and cross environments), and I made no attempt to be historically accurate (the original separation of / and /usr happened for intensely silly reasons), but those are the fundamental principles of the system, and I feel it does make sense as a coherent implementation of a particular design. Other designs are possible (separation by application or package not purpose, Plan 9-ish overlays, NixOS’s isolated environments), but that’s a discussion on a different level; the point is that this one is at the very least internally consistent.
Re the unfriendly names ... I honestly don’t know. Newbie-friendliness matters, but it’s not the only thing that does; particularly in a system intended for interactive text-mode use, concise names have a quality of their own. There’s a reason I’m more willing to reach for curl and jq rather than for httpx and lxml, for regular expressions rather than for Parsec, and even for cmd.exe, as miserable as it is, rather than for PowerShell.
I feel weird that no HCI people seem to have seriously considered the tension between interactive and programmatic environments and what the text-mode user’s experience in Unix says about it, but even Tcl, which is in many ways a Bourne shell done right, loses something in casual REPL use when it eliminates (as far as idiomatic libraries are concerned) short switches. Coming up with things like rsync -avz or objdump -Ctsr is not very pleasant initially, but I certainly wouldn’t want to type out the longhand form that would be the only possible one in most programming languages (even if I find their syntax beautiful, e.g. Smalltalk/Self).
As I recall, there were very good reasons for separating / and /usr (as well as /home and /var). The biggest one was that various Unix kernels would panic[0] if / was full. But that issue was almost universally fixed by 1990 or so.
And netmounts of pretty much everything other than / were pretty common for many years, due to the high cost of storage.
So no, the reasons weren't silly, they just don't apply to more modern systems.
(I have an old Unix intro book that describes /usr as user home directories, the rest is a second-hand retelling[1].)
[1] http://lists.busybox.net/pipermail/busybox/2010-December/074...
In your home directory.
See my post to this thread at
And conditions that formed separation are long gone, Arch Linux symlinks most of it:
/bin -> /usr/bin
/sbin -> /usr/bin
/usr/sbin -> /usr/bin
/lib -> /usr/lib
/lib64 -> /usr/lib
/usr/lib64 -> /usr/libToo much choice for things like this is harmful IMO. Deep down I truly couldn't care less where the files end up, as long as that place is the 'right' place. There are too many 'right' places which makes it hard to find random things at a later date or when on a box you're not super familiar with. It's also a complete waste of time to think about it at all.
The “standard” doesn’t tell you what directory structure to use inside /etc to group related config files. The “standard” doesn’t tell you where an HTTP server should serve its files. Everyone just does their own thing which makes upstream docs incorrect and useless for newcomers.
The FHS, does actually answer both of of those questions. Files inside /etc/ should be grouped in subdirectories[0] andd the HTTP server should serve user-specified website files from /srv[1] and normal distro-provided files (such as the apache test page) from /var[2].
[0]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s07.htm...
[1]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s17.htm...
[2]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05.html#p...
I’ve never seen that in my life, but I’m sure someone does that. This is one of those cases where the people who follow the standard are increasing fragmentation
So, given some data, say a file and/or directory, maybe from saving a Web page, that is relevant to subjects A, K, T, and Z, where in the file system directory trees to put that data?
My solution: Put the data in a directory for one of the subjects A, K, T, or Z without thinking very hard about which of these. Then go to a file I call FACTS.DAT (right, an old idea with an old 8.3 file name!). I maintain that file with a few, simple editor macros. So, sure, the file is a catch-all for entries of random short facts. And each entry starts with a time-date stamp and a list of key words. So, in the case of subjects A, K, T, or Z, include the key words appropriate for each of those. Then in the body of the entry, put the tree name of the file/directory where did store the data.
In a few seconds with my favorite text editor I can append an entry or search for an entry.
So far this year I have put 686 entries in the file FACTS.DAT for about 2.1 entries per day. For anything like current personal computers, handling such a file is trivial.
The idea works great!
"While no one sane would put newlines in directory names, such corruption of the results could lead to exploitable vulnerabilities in scripts."
Want to see true craziness? POSIX file names are just a bag of bytes. They don't even have to be text, they can be anything (almost), there's no standard text encoding:
https://lwn.net/Articles/325304/
And in typical Open Source fashion, someone actually claims it's a feature: https://lwn.net/Articles/325398/ because hey, you 99.999% percenters can suffer so that I, 0.001% percenter can implement my wacky system.
$ mkdir hold
$ cd hold
$ cat ../wildname.c
#include <stdio.h>
int main(int argc, char **argv)
{ char n[256]; int i,j=0; FILE *fp;
for(i=1; i<256; i++) if(i!=47) n[j++] = i; n[j] = 0;
if(fp = fopen(n, "w")) { fprintf(fp, "hello world!"); fclose(fp); }
}
$ cc ../wildname.c
$ ./a.out
$ ls -l
total 16
-rw-r--r--. 1 luser lgroup 12 Nov 11 16:32 ??????????????????????????????? !"#$%&'()*+,-.0123456789:;<=>?
@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~?????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
-rwxr-xr-x. 1 luser lgroup 8464 Nov 11 16:32 a.out
So anything not directly blocked by the software is allowed.
Ergo, clear specifications, strict yet flexible types and APIs, etc.
Otherwise, it's just bad design.
/var stands for variable data--like log files, cache directories, spool directories, etc. You shouldn't put executables there. Ideally you should be able to set the noexec flag on it.
`/usr` actually exists because the original UNIX developers ran out of disk space and had to attach another disk. The difference between /bin and /usr/bin is not worth it and even Debian symlinks /usr/bin to bin.
But your distribution's package manager should be putting stuff in /bin or /usr/bin, not you. Anything that follows the regex "{asterisk}/local{asterisk}" is something the system owner can do whatever with. So you should be using /usr/local/bin or $HOME/local/bin. I don't know why there's no /local off of the root. (One thing I do on my own systems is make and use an /etc/local although I think you're supposed to use something like /usr/local/etc).
/opt is for third party programs that aren't installed via your distro's package manager.
If you do this, any customizations you make to a system can be easily backed up by copying all dirs with local in the name.
There's multiple decades of tradition behind these names, but they do date back to the age where actual teletypes were used.
Make the overlay of your dreams!
Of course, that's not perfect either, because a) decades of changes vs. compatibility have made it less clean in certain places, and b) pretty much all the POSIX paths still exist for unix-y compatibility, but overall it's like that.
Restructuring the directories is the easy part. You just delete the old tree and make a new one. You can also mount procfs and sysfs wherever you want.
The hard part is modifying existing software to work with the new tree. So many programs assume you have a "standard" file system tree. So many programs assume procfs is mounted at /proc. So many programs have hardcoded paths. Shared library locationd can become part of the binaries when they're compiled. It's insane and you'd essentially be creating a new Linux distribution.
Works from File-Open dialogs and address bars and even in the command prompt you can even do "explorer documents".
Fortunately the future of the Windows API does look better since Microsoft has now added proper UTF-8 support since Win 10 1904. All you have to do is request it in the application manifest and the A functions will accept and return UTF-8.
Apple OSes use something they call "unichar" inside NSStrings. I'm not 100% sure what it is, but it feels like it's the same 16-bit wide character.
They really should have gone with WTF-8 [0] since the W functions generally accept WTF-16 and not just the valid UTF-16 subset.
The double grave I is used by some obscure orthodox religionious texts
for (int i = 1; i < INT_MAX; i++) {
if (dirExists("C:\\PROGRA~%d\\ProgramName", i)) {https://web.archive.org/web/20100107184218/http://blogs.msdn...
And so, yes, I’m certain someone must have done it, because it’s clearly bad idea jeans and so Murphy’s Law says it must exist.
https://docs.microsoft.com/en-us/dotnet/api/system.environme...
There are several other classes that take care of getting folders, least of which checking system variables.
https://docs.microsoft.com/en-gb/archive/blogs/twistylittlep...
In nix at least you can call execve or other APIs that take a char argv[] and the whole problem is largely solved and you don't need to quote things.
Easier to add a flag to ignore case rather than fix bugs where files only differ by case and are therefore overwritten on a case-insensitive filesystem.
Easy fix!
> C:\P̷̧̽r̸̬͘ŏ̵̮g̷̜͘r̸̦̋a̴͎̒m̶̲̈́ ̷̠̉F̵͇̈ĩ̴̫l̶̨͗ë̵̦s̸͚͆\
So, some things were still being handled not quite right (whether that's because it shouldn't be allowed to be the username, or because programs should handle it being in the path, I'm not sure, but probably one of those.)
thisismyconfig.txt vs this is my config.txt or this_is_my_config.txt
...i've forced myself to stop using spaces, character, and even cap. They are all constructs that provide minimal value for the extra complexity.
Just wondering, what is the readability of this for people who are dyslexic?
Saying, "who can't understand..." is arrogant, selfish, and an example of why normal people hate people in the SV echo chamber.
Exactly how I feel every time Economics is brought up on HN.
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
But Hacker News should do something about all of the anti-bitcoin and anti-anti-nuclear ideologies running around in here.
I don't really mind it that much but it'd be nice, it's really the only 2 extremisms I've experienced here, all other subjects are discussed in a fair manner.
There's no such thing as fair discussion about those topics here.
Otherwise what is your proposition, don't state any opinion unless you find a counter opinion commenter to match with?
Lots of folk here are pro-privacy and lots of folk are anti-bitcoin (and some of them will be the latter because they're the former) so I don't understand how you'd extended your position in a way that leaves HN with any value.
It's the same in any language.
Hugs who hurt you.
I'm also pretty sure most of us in any language use Slack, SMS or other forms of communication where text isn't necessarily presented in a grammatical correct manner and we all figure out what the person is saying.
Switching to Chinese would also help eliminate the spaces issue.
Aside from parsing errors it just seems to lend itself to ambiguity.
Also programs automatically that turn paths into links don't work with spaces.
could you please explain what you mean by that?
Your system is only intended for other programs to interact with? Go nuts, make hex UUIDs. Actual people are supposed to use it? You need separator characters.
I also don't see how those characters add "extra complexity" unless you're doing dumb things like text processing on paths and filenames (as opposed to using OS/library functions that handle paths correctly) - in which case, there's your problem.
A file system has a human interface and a computer interface. Don’t mix them. Let users give file names in whichever way they please.
A former co-worker changed his name in our auth system to include an apostrophe, so that whenever we handled names wrong he'd find it.
My best guess is "change the system time to be a timezone for which, during your work hours, the other-timezone is in a different day than UTC is" - but I'm still not sure what effect that would have on CI failures.
it keeps everybody on their toes lol.
League of legends doesn’t run until I sed files for instance.
When I got a new Mac, I just gave up and acquiesced to the case-retentive world :-(
Sure enough, they had SomeFile and were importing Somefile and it works fine on Mac but not on Linux (which, of course, is what our production servers use). It amazes me that "works fine on my machine" is still a thing when I definitely worked at companies that solved this back in the 2000s. It was solved. It was done. Then devs became enamored with running everything locally. Even dozens of microservices or databases. Even though JS is fairly isolated, you still have NPM packages that need built against the local OS and C/C++ library and compilers, etc. Which also has caused issues in the past.
Docker hasn't supported LXC since 2016, and stopped relying on it in 2014
https://www.docker.com/blog/docker-0-9-introducing-execution...
Actually no. At least on Windows Docker can do native Windows containers too
https://poweruser.blog/lightweight-windows-containers-using-...
Enforce this in LDAP.
Strict convention is better than flexibility and predicting obscure edge cases that can fail.
Non-ASCII paths are extremely common (e.g. the user's home directory on Windows, for the large majority of users outside the English-speaking world) and spaces, punctuation and weirder characters will definitely happen when you least expect it.
Yes if you can avoid it then absolutely that's great, but I don't think most people can.
It's also not usually very difficult to deal with, as long as you actually spot the issue in the first place.
But then your program will crash hard and unexpectedly when a user decides to save under "~/house plans" or ~/Téléchargements.
I think it's better to exercise this in CI, that's what CI is for.
Anyway, all this was fine when we were on LDAP rather than Active Directory. So basically it's all Windows' fault.
...and only hire people from the exact same background as you, who will never have unusual characters or accents in their name. And also make sure not to have any users who aren't exactly like you, and conform to this very narrow requirement. Surely, excluding 90% of the world won't hurt revenue in any way.
Use strict schema for the hardware interface, networking, physical stuff the user never sees. Microservice names don't need to be non-Latin. Database replicas, infrastructures, etc. And you're not going to piss off employees by giving them ASCII ldap/email addresses.
Use utf8mb4 or similar for storing names. Don't state "first" or "last". I've been through this rodeo too many times. You're not surprising anyone.
If you consider spaces “unusual” I would say you haven’t encountered a single average user in your lifetime. Spaces in file-names is the single most common thing people have, outside programming environments.
As a x-plat developer, the only platform where I (still) regularly encounter these kind of bugs are platforms where solving problems through scripting is common, like Linux, where the primary means of operation is through stringly-typed statements getting parsed and processed in a untyped-fashion. It's not very reliable.
On Windows people more often use “real APIs” (because scripting doesn't really work as well), but then these problems just goes away.
Pros and cons, I guess.
This will also break any code in external tools that are called during the builds of your application and do not handle spaces correctly for whatever reason, thus making it so that you won't be able to successfully finish the build.
Then again, you probably shouldn't be relying on technologies like that, but when you're struggling to keep an old enterprise system alive, causing yourself more problems is not necessarily what you should do.
Still a good idea in most cases, though.
Saw a few hacks where malware authors used the RTL feature (which is baked into Windows) to obfuscate file extensions. It looked like .exe.innocuous-document.docx, but was actually .docx.innocuous-document.exe
I changed my username to not contain a space because it was too annoying to deal with all the random dev tools breaking. The worst offender was probably npx on Windows [1] (resolved after four years by deprecating npx), but it was far from the only one (though the JS ecosystem was somehow the worst in this regard of all languages I worked with).
Spaces are very useful for readability.
Mysterious character requirements that do not conform with Microsoft’s OS limits, limits on tbe fully qualified pathname length, etc.
Inject a Turkish 'I'. I don't know how to type or paste it here, but picture an English lower case 'i' that is upper case. It is a splendid way among many to shake out some loc bugs.
The problem with that is that YOUR code may handle it, but your tooling may not. If my code formatter break on spaces, I'm not going to change the formatter.
Last week I opened a ticket for a Firefox bug. Following up on the bug took me 2 hours in total.
FOSS is not free, you pay it with your time. And as with everything you pay for, we all have a budget.
A small extra step but something you get used to if you spend a lot of time in the cli.
I set up symlinks which help navigating around but then the relative paths are wrong for git.
No thanks.
Friends don't let friends put spaces in paths
Naming freedom needs a stdlib module
I'm begging software developers to stop using subprocess APIs that take a string argument (system(), child_process.exec(), Process.Start(string)) and start using subprocess APIs that take an array of arguments (execvp(), child_process.execFile(), Process.Start(string, IEnumerable<string>).)
I never use / on Windows as a result.
These days using CMD instead of PowerShell should be rare enough and PowerShell certainly doesn't mind the slashes.
# Rename all files in a directory
rn() {
rename "s/ /-/g" *
rename "s/_/-/g" *
rename "s/–/-/g" *
rename "s/://g" *
rename "s/\(//g" *
rename "s/\)//g" *
rename "s/\[//g" *
rename "s/\]//g" *
rename 's/"//g' *
rename "s/'//g" *
rename "s/,//g" *
rename "y/A-Z/a-z/" *
rename "s/---/--/g" *
rename "s/---/--/g" *
}
By the way: what's your beef with en dashes? I mean, if it was "everything should be 'HYPHEN-MINUS' (U+002D)", then fine, but why specifically en dashes and not em dashes?
Of all the changes in that list, removing the character that doesn't appear on a standard keyboard seems like the least controversial...
It's up there with those damn angled quotes.
Whether people or processes, something is likely to introduce the character at some point.
(Also—that entire function is super inefficient and could be replaced with a single invocation of "rename".)
# Rename all files in a directory
rn() {
rename \
-e "s/ /-/g" \
-e "s/_/-/g" \
-e "s/–/-/g" \
-e "s/://g" \
-e "s/\(//g" \
-e "s/\)//g" \
-e "s/\[//g" \
-e "s/\]//g" \
-e 's/"//g' \
-e "s/'//g" \
-e "s/,//g" \
-e "y/A-Z/a-z/" \
-e "s/---/--/g" \
-e "s/---/--/g" \
*
}
# Rename all files in a directory
rn() {
rename \
-e 's/[ _—]/-/g' \
-e 's/[:\(\)\[\]",]//g' \
-e "s/'//g" \
-e 'y/A-Z/a-z/' \
-e 's/--+/--/g' \
*
}
In addition to CLI I use it from emacs dired-mode too:
(defun my-dired-detox ()
(interactive)
(dired-do-shell-command "detox" nil (dired-get-marked-files))
(revert-buffer))
find . -depth -name '* *' | while IFS= read -r f ; do mv -i "$f" "$(dirname "$f")/$(basename "$f"|tr ' ' _)" ; doneI have:
Movie Bla (2020)
Movie Bla (2020).mp4
Movie_Bla_(2020)
Movie_Bla_(2020).mp4
Movie_Bla_(2020).srt
normalize_names() {
rename "s/-/_/g" *
detox -s lower *
}
I've thrown some edge cases at it, and it handles it super well. It deals with consecutive "_", remove leading garbage, normalize unicode, and even prevents naming conflicts by opting out early.
Thanks you.
You should better be very afraid of using spaces in filenames.
You should do everything you can to support them but you have to know you'll invariably encounter countless cases where you'll have this or that tool that won't work properly with them.
I still live in a world where I cannot name a song from the french group L'impératrice with an eacute in the filename or my car's media system will display garbage (it's running QNX and I don't know which filesystem).
FWIW, and it should be food for thought, every single Git repository in the world contains a pre-commit hook sample (disabled by default but it's there) that enforces that every committed file in the repo is named using a subset of ASCII characters.
Every Git repository in the world has that example: let that sink in.
I use Git for documents too, not only code. Why shouldn't I use my native language?
It was a great joke for a couple weeks two internets ago.
It's been like three internets since I heard someone using "internet" as a measurement of time.
It's actually interesting to think about "generations" of internet, just like generations of people, and how the culture shifted between them.
There was a time in the early '00s when broadband was catching on, yet YouTube didn't exist. A time when Ebaumsworld and Newgrounds ruled the internet. When Homestar Runner was pop internet culture. Weebls Stuff. The frog blender.
See this answer on stackoverflow:
https://stackoverflow.com/questions/1732348/regex-match-open...
The characters reach up off the screen as I reply to this. They overlay the comment above you. Amazing. How?
[https://stackoverflow.com/questions/6579844/how-does-zalgo-t...]
I have an Android phone and I tell MusicBrainz Picard to save all files with ASCII-only names and Windows-compatible names for the ones that get sent over to the phone. Basically for this reason. Sometimes it's players on Android itself, but even more frequently, whatever bluetooth radio I'm connected to freaking out with non-ASCII characters.
L'imp?ratrice? L'imp�ratrice? L'impératrice? L'imp‚ratrice? L'impÚratrice?
And "path component" is not an arbitrary string either - e.g. appending a path component to the path should first require converting/parsing the string into the path component, and only if that's successful appending it to the path.
For maximum correctness, you want to turn it into a file handle as soon as possible, and do all operations through the variations of the file functions that end in "at", like: https://linux.die.net/man/2/openat
The downside of this approach is that you still technically have to carry the path around with you if you ever want to present it back to the user, because once you have a directory handle, you can get back to the root directory easily enough by following parent links and seeing what directories you end up in, but that may not be what the user "thinks" the path is, and they want to see their path, not a canonicalized one. And they're mostly right. And it's not easy to correctly track changes to their intended path from this basis either.
Basically, I don't know of a really solid, 100% correct way to handle this with any reasonable degree of effort.
But no sooner.
For example, I've run into problems where I'm configuring program A server to talk to file location B... but I don't have access to file location B. But the client-side library for talking to the server tries to convert location B into a file handle and then freaks out because I can't access it. When I don't want to access it. I want that program to serve it.
If it was using simple "path" objects that didn't confirm that I have access to the path, everything would be hunky dory. But because it tried to convert it into a file handle unnecessarily, I get blocked.
Otherwise, use an integer unix timestamp, the number of seconds since 1970-01-01T00:00:00Z: 1636673555. Use an unsigned 32-bit integer or a 64-bit integer to avoid the 2038 problem [1]. JSON's maximum safe integer value is a signed 53-bit integer, so if you're using HTTP JSON RPC, you'll have to check for overflow.
[0] https://en.wikipedia.org/wiki/ISO_8601
[1] https://en.wikipedia.org/wiki/Year_2038_problem
[2] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
This is why I get stressed out when I see paths turned into special objects encoding separators and such.
It tells me the path is living for way too long compared to the file handle.
I only want to see path-specific objects if we're modifying the path, and even then I want that to happen as late as possible.
That's not right. You want to resolve a file/folder path to a file/folder at the exact point it makes sense.
It's a problem if you're using a path when you wanted the file. The file can be switched/modified out from underneath you.
It's also a problem if you've got the file when you only wanted a reference. Now you can't simply switch/modify the file independent of the reference. E.g., maybe you want config file changes to take effect immediately and transparently.
You can also have the hybrid case, e.g., where you want the folder directly, but have a relative path to a file that is resolved late.
If you're unsure, I'd err on the side of late resolution.
I guess that depends on what you mean by "string". `open` and `fopen` need a char* path to open a file. Whatever fancy Path abstraction you use eventually becomes a char* string, because that's what the kernel needs.
I think for any POSIX system, either there is locale support defining the encoding, or it uses the POSIX locale, which defines the encoding (ASCII).
Of course you need to handle cases where filenames cannot be decoded in the system encoding (probably by replacing characters that cannot be decoded), because a filename in a different encoding, or even with no valid encoding, has been used on disk. While systems can say that file names containing bytes that are not valid characters in the system's encoding are not valid file names, that doesn't stop people mounting disks with them, so the problem never goes away if you support opening media from other systems.
What I am saying is that this is no more a Unix problem than it is a problem on any system that supports removable media.
"This is `a
perfectly vali'd.\010! file name\377, despite the weirdness"
Yes, paths have structure, but saying "a path is not a string" is equivalent of saying "C source code is not a string". Both are strings, and both are something else, represented by strings according to rules. Different internal representations have different advantages and disadvantages. I fully agree that for things such as "adding components" an internal sequence/list representation is better, but strings can pass arbitrary IPC or even ABI boundaries much easier for example. (And you wouldn't bat an eye for example when you see FQDNs like "www.google.com" passed as a string instead of as ["www","google","com"] because the string representation works pretty well.)
Because neither are strings, their native representation shouldn't be such - it should be something structured, and only when necessary (IPC, FFI, serdes) be serialized into a string representation. This would save people a lot of time and effort.
Sure, a browser will hopefully quickly parse that URL and break it up, an image viewer will do the same with a JPEG. Will anything that's only interested opening/displaying that URL or JPEG, through a library or external program?
POSIX paths are actually remarkably simple in structure[1]. The only caveat is equality and normalization: Without normalization, a path a might be equal to a path b while their representations differ, e.g. "/etc/foo" and "/etc/bar/../foo". But this is the same whether you have a string or a list of strings, you need to normalize in whatever representation you choose to check for equality.
[1] Almost shocking myself, even Haskell defines its primary FilePath type literally as "String".
text processing is hard if you must support Unicode, and that means every Unix command line tool must implement or employ a text processor to handle input. it would be much easier if objects were passed back and forth. PowerShell got this right.
xset m 0 0
xinput --set-prop 'pointer:Logitech USB Receiver' 'libinput Accel Profile Enabled' 0, 1
Somehow these short letters stick much better for me, and the effort for finding them in the manual is the same, although in case of extra complexity as with xinput, it's even worse with the long names. I don't use either command often, but it's hard to forget xset m. The only thing I remember about xinput is that it's a horribly long lithany of things which I need to look up every time, and the syntax still feels weird.
If you are using cli tools of github written by a random person, then no wonder you will see non-standard approaches to UX.
For example, if a command features a "-ConfigFile" flag, valid short-form variants include "-C", "-Co", "-Con", "-Conf", and so on. But if the command featured an additional flag "-ConfigURL" for example, the aforementioned short-form flags would be ambiguous.
One of the things I like about Fish is that by default it can tab-complete program options and also shows a one-line description of what each of them does. (It grabs that info from the man page).
We computer people are truly an odd bunch.
Yeah I used to have all kinds of simple as well as supposedly sophisticated completion setups with zsh years ago but I've given up on it since then. It's always half-assed and half the time causes more problems than it solves. Same with bash. There are some places where I must resist the urge to try complete a filename because the shell starts trying to figure out which target it can complete from a Makefile in a large build system and just freezes. The only practical way out is to interrupt and type the command again or wait a stupidly long time. There are other issues like completion trying to be smart and filtering out things it thinks you don't want to complete. Nothing is more frustrating than a shell refusing to complete a filename that you know is there.
I could never overcome my repulsion for Java and ObjC because of that. On the other hand, I fell at home with crazy RegEx that look like line noise to most people.
As a beginner, I liked short variable names. When I came back a few months later, I learned my lesson. Years later? easier to just start over.
For everyday command like `ls -l` I don't mind but anything more serious I take a more cautious approach.
The amount of time that has been wasted by Windows using "C:\Program Files" instead of "C:\Program_Files" far outweighs any highly questionable aesthetic benefit IMO.
I’ll be damned if I have to remember or lookup what -n means to some obscure program, when reading someone else’s script. Exception given for super common tools where everybody knows like ls -la.
With the disclaimer that shell scripts, especially ls, aren’t exactly suitable for reliable automation in the first place.
Programs like Powershell eschew ease of use in CLI for readability in scripts.
Nocase (did I break a rule by writing it that way?) seems great when you're enmeshed in the domain and you can see the implicit separators, but then someone looks at your naming from the outside and you're guaranteed to have an 'expertsexchange' in there somewhere.
And you use something else than your pinky finger for the shift key specifically when typing capitalized letters for camelCase?
For reading, CamelCase has 2 significant ambiguity issues: similarity between I and l, and what do you do with acronyms. Acronyms wouldn't actually be a problem if everybody just wrote them would in snake_case (i.e. only capitalize the first letter), but they don't and so it's anyone's guess whether you're going to get "Id" or "ID".
There's also a minor issue where if you're on a case-insensitive file system it can be a little difficult to change casing, but adding/removing underscores is easy.
And if CamelCase is so hard to read, why is it the norm for "high level languages"? Shouldn't those be optimized for ease of use?
That's over-selling it a bit. It's more common, but not dramatically so. Outside of class names, CamelCase isn't the norm for Python, PHP, CSS, HTML. It's also not the norm for shell scripting, but shell scripting has horrible readability for other reasons.
I believe CamelCase is more common for languages like Go, C#, and Java because they grew up in large organizations where having god objects/classes with 400 methods is kinda normal and having aMethodWithAReallyLongName is pretty common. One of the advantages of CamelCase is that it does shorten really long names.
The first one is some magical incantation.
Ergonomics isn't all about making everything self-descriptive for someone seeing the thing for the first time. It's about making things comfortable to actually use. If it's so long and complicated that you can't even remember how to do it, it's not very comfortable to use. Even if I could remember, xset m 0 0 is still far more comfortable.
And fwiw you still don't know what 0, 1 in accel profile do; you need to look that up or take a wild guess, and if you want to use that command, you'll also have to know how to look up the device because chances are yours is not the same as mine. So it's not any less magical in the end, just more verbose.
The "cool" thing about the xinput command is that you don't even find accel profile in the man page. You gotta look elsewhere if you want to understand what it is and what it does and what the parameters are.
xset m? Yes, that is documented in the man page.
I would never even type such a command. I would just copy paste it once.
I don't want to waste time searching for a command to copy-paste when it could just be made short, simple, memorable and ergonomic. I could type xset m 0 0 faster than I could open a browser and ask google how to disable acceleration with libinput. And again: you can't just copy-paste the xinput command unless you're lucky enough that it matches your device. On my new computer, the device has a different name than on my old laptop even though it's the same damn mouse.
At least it would push all the "This switch was added by someone playing with UNIX at a university in 1986 and hasn't been used since" options to the end of the list.
We're talking about `xset`. It doesn't make sense to optimize that for usage of more than once a year.
xset achieves that perfectly. If I somehow didn't remember how to set mouse acceleration with it, a quick glance at the synopsis immediately tells me. Or I can just run the command and it'll tell me:
To set mouse acceleration and threshold:
m [acc_mult[/acc_div] [thr]] m default
This is something I've observed more than once: I easily memorize useful sets of one-letter flags even if I can't remember or know what they all stand for. This just doesn't happen nearly as much with long options. Commands like ls -ctrl or ss -nap quickly become part of my repertoire even if I don't use them very often, but I really couldn't remember ss --numeric --all --processes (if I had written that from memory, it could've ended up as --num --all --pid or --numeric --any --process), and I don't even know what the corresponding long options for ls are. In the rare case when I have to deal with an option that has no short equivalent, I feel like I have to look it up every time if it's been longer than a few weeks.
You talk of optimization but I think this is just a very basic (and reasonably successful) attempt at sane design. It's not like someone had to go far out of their way to make this in a manner that isn't batshit insane.
On the first, you think you know what it does, but you're not sure. So maybe it gets looked up.
On the second, you know you don't know what it does. You so know to look it up.
Personally, I'll take the second. Assumptions during debugging are dangerous things.
They are so easy to remember that you need to configure your shell to remember them for you?
Wait, are you saying that I need to change my shell or config to make up for another tool's poor design?
No, thanks.
(The ability to shorthand connection->c and similar is great, too; obviously not unique to nmcli)
This is a lesson I had to learn the hard way, multiple times.
I suspect the number of people who figured all that out without having to find it by googling / arch wiki / whatever is very very low.
Now I'm not gonna say xset is the easiest interface to figure out, but the syntax for setting mouse acceleration is right there in the synopsis, and if you search down the man page, you'll learn a little more (and also if you just run xset without arguments, it'll tell you how to set mouse acceleration). It might not be the best designed tool but it's something I learned back in the day as a teenager just by looking at the man page.
I think the real issue is that people nowadays are designing these interfaces to be consumed by interactive configuration tools, GUI apps, and desktop environments; they're more dynamic, more complex, more flexible, but not easier to figure out, not for you on the command line. The command line is just a last resort. Second class citizen if you will.
alias mouseoff='xinput set-prop 11 "Device Enabled" 0'
alias mouseon='xinput set-prop 11 "Device Enabled" 1'
$ xinput --disable
Segmentation fault (core dumped)How often do you reach for iptables? If you're like myself, and most home/desktop users, then probably once in a blue moon to set it up and then you leave it alone. But a system admin? Maybe they touch it a few times a week or month. Every time I use iptables I have to relearn how Linux networking works.
Similarly, the xset/xinput thing. When I need those tools I just create a script or throw it in .bashrc. I adjust the settings once and will not touch them again for a couple years. It makes sense to have long parameters that are readable. I can look at my .bashrc and see exactly what device is getting adjusted.
If you had comma-delimiting like in an algol-derived language, you wouldn't need to quote things with spaces.
edit: also, code is read more times than it is written, so optimizing for readability over brevity is generally a good move.
We started out with 'eth0', 'eth1', etc. Which adapter was which could change when adding and removing a network card. That was bad, so that prompted the evolution.
Now we have 'enp1s0', 'enp0s31f6', 'enp13s0' and many similar variations. These are supposedly more stable across device changes. As it turns out, it wasn't.
But wait, there is more! Now we have the "predictable names" scheme that produces interface names that are even longer, and not even slightly easier to remember.
Read about the whole sorry saga here:
https://wiki.debian.org/NetworkInterfaceName
I do get that it is not an easy problem to solve, especially in the face of removable network interfaces (like USB Ethernet / WLAN). But surely this is not the best we can do.
One thing I like about OpenBSD is that buses are scanned and drivers probe in order and there's no race between drivers coming up. Unless your hardware is physically tampered with or broken, all interfaces come up with the same name across reboots. Linux isn't like that (even if you don't touch your hardware, interfaces could swap across reboots), so you need to do something about it.
As is typical on Linux, the default is unergonomic and if you want something nice, you're on your own to make it so.
If you already have userspace daemons responsible for device insertion and naming, it really wouldn't have been so hard for it to e.g. automatically add a config file / database entry for each interface the first time is seen. So the devices that came up as eth0 and eth1 are still eth0 and eth1 on the next boot; if I unplug eth0 and add a new card, the new one would be eth2 because eth0 is still reserved for the first card I had.
Ubuntu did that with their persistent-net.rules udev rule. That was a part of the PITA of the old naming.
No. These are stable across reboots. The old eth? weren't. And yes, that had been a PITA.
Of course, if one doesn't write a CLI to begin with, this trade-off doesn't exist - you can have your cake and eat it too.
Ironically, even NASA doesn't like space.
https://www.nas.nasa.gov/hecc/support/kb/portable-file-names...
2021-01-01-some-important-document.pdf gives me the warm fuzzies. On the off chance that some more differentiation is needed, throw in an underscore and a whole new world opens up
2021-11-11_client_projectName.ext is also OK. But underscore separates fields, hyphens for space replacement.
work/client/project/2021-11-11-file.ext is more or less how I lay stuff out. I’d say client/project is a folder level distinction (arguably dates too).
[EDIT] Realistically most of the stuff under <project> is git repos and I usually make a “home” repo where I keep org files for tracking hours, notes, and resources related to the engagement.
https://www.nayuki.io/page/designing-better-file-organizatio... ; https://news.ycombinator.com/item?id=29141800
But life made me use Emacs org-mode more and more, and I'm now in love with tags. Retrieving information has become so easy, especially with org-mode's tags inheritance, that I hardly think making connections between headings or notes is necessary anymore[1]. And I believe that applying tags to filenames (a la Karl Voit [2]) will create the same effect
[1] A Zettelkasten-like system is still unbeatable imo when it comes to ideas repositories, i.e. a second brain you can talk to and get new insights. It's just not that great for personal knowledge management or project management.
But why not the other way, hyphen-minus for separating fields and underscore for space replacement? That seems to me more consistent with how underscores and dashes are used.
my_project-some_activity-this_document-20210923-v02.txt
[1] https://raku.guide/#_syntax_overview (see section 1.7.1)
In Forth, 'words' (which are roughly analogous to functions and operators) must always be separated by whitespace, as Forth doesn't parse out operators the way most languages do. In exchange, you get the ability to use symbols in identifiers, as Forth has no reason to single out symbols like + as being syntactically special. You can even use a number for the first character. (For that matter, Forth will even let you override the usual interpretation of a numerical literal, but that's always struck me as going a bit far.)
It gives you a + word, analogous to the + operator of most languages [0]. It also gives you a 1+ word, as an (admittedly slight) abbreviation of the sequence 1 +. [1] If you wanted a 2+ word, you could easily define it yourself.
(This property of Forth evidently wasn't enough to get it to take over the world, but it's still neat.)
[0] https://www.complang.tuwien.ac.at/forth/ansforth-cvs/documen...
[1] https://www.complang.tuwien.ac.at/forth/ansforth-cvs/documen...
Notably he didn't promise any, but presumably one'd need a separator... Maybe, per his "K" usage of the month, one'd use the alphabet again. 11am would be "K" again... or lowercase just for giggles?
I don't think it reads very well, but I also think one'd get used to it pretty quickly.
I hadn't heard that before and I love it.
Perennially relevant xkcd: https://xkcd.com/1053/
Yes on the date format.
Saves you so much time.
211111_foobar_v1.txt
I am old enough that I still save before printing. I think it was Lotus 123 that engrained it for me.
R traditionally uses the . as a legal character in identifiers. Once you get it used to not being syntactic, I found I actually prefer them to underscores.
2021-01-01_what-happened_who-did-it_possible-reason
https://web.archive.org/web/20151007005513/http://www.911cd....
P.S. should anyone want to see/run the actual batch, a copy has been uploaded here:
http://reboot.pro/index.php?showtopic=18962&page=29#entry204...
Sadly such set includes loads of Java programs. If only SUN had shipped a standard way to generate isolated exe files in 1998... but they worked under the presumption that you'd have a JVM already there, because distributing that monster was difficult in dialup times, so you could just hand people a jar; and the enterprise market did not care, since they had webapp servers. Sadly it's an "optimization" that became obsolete very quickly but wasn't rectified until it was too late (java 9+).
Actually it isn't, just use double quotes and add a '~'. It's just about the only thing batch files handle better than shell scripts. set "VARIABLE=%~PATH"
https://docs.microsoft.com/en-us/windows/win32/api/processth...
The arguments are a single string. So you want to pass parameters with spaces in them? You've got to add quotes and stuff all of that into a single string. Instead of doing it in a more sane manner, like oh, the arguments to main().
Oh and cmd.exe uses a different escaping scheme than the CRT.
They aren't going to break existing API or bloat the kernel with a bunch of functions that do the same thing.
Giving CreateProcess a new flag so it would to correctly accept "path\\to\\my\\program.exe\0arg_1\0second argument\0argument with literal \" symbol" (with an implicit \0 terminating it) as lpszCmdLine is an easy part; the hard part would be forcing everyone to switch to using it.
Also, I'm pretty certain this processing happens in the user space, and Win32 API is already bloated beyond any belief.
The escape character on PowerShell is backtick, and on cmd it is caret. You don't need to quote everything.
And shells and other programs still have problems with perfectly legal characters in filenames too, like '!' or ':'.
Without asking you to always quote and escape every file name - what alternative is there? If they tried this you'd probably find you didn't like it.
"It's nothing."
"What do you mean?"
"It's nothing... It's empty space. I never taught the computer how to read empty space!"
"I never taught Virgil how to fly."
Build script works fine locally on Windows, but then chokes in *nix test server, as it's effectively a different path.
As of sometime in 2018, fsutil can set specific directory trees to be treated case sensitive in Windows 10 without setting it for the OS. This ability is mentioned here[1]
[0]: https://devblogs.microsoft.com/commandline/per-directory-cas... [1]: https://docs.microsoft.com/en-us/windows/wsl/case-sensitivit...
I also love when you're using bash and you have a file with ! in the name, and you accidentally fail to correctly backslash it, you not only get "bash: !rest_of_filename: event not found", but it also fails to add that command line to the history, so you can't just hit up and fix it. You have to actually go to the mouse and copy and paste.
> Local file names can be made explicit using absolute or relative pathnames to avoid scp treating file names containing ':' as host specifiers.
So `scp foo:bar user@host:~` fails because it tries to find the host foo. But `scp ./foo:bar user@host:~` works just fine. I feel kind of stupid for not guessing as much.
Edit: nope, just tried it and scp still sees the quoted filename as a host + path
"Does foo:bar refer to the local file `foo:bar' (size: 102kB, date: 2021-11-11) or to the file `bar' on host `foo' (FQDN: foo.example.com, IP address: 1.2.3.4)?
1: local file `foo:bar'
2: file `bar' on remote host `foo'
Your selection: "
Also back then Mac file names typically did not include an extension, because the file's type was stored as part of the metadata in its resource fork. I remember one time a friend of mine was visiting and was playing around with a paint program on my Mac. Being used to DOS, when she went to save her file, she typed a very short name, and then asked me what the proper file extension should be. I smirked and said, "That's not how you name files on a Mac. THIS is how you name files on a Mac." And then I named her file "Ailsa's Cool Picture". Her mind was blown. :-)
¹This is because the colon was the path separator. But since the classic Mac OS had no command line interface, the typical user would never type or even see a file path written out.
However, I found the lack of a command-line to be restricting.
Sometimes i think that the command-line is a crutch that keeps programmers from learning how to make good UIs.
In many languages it's a requirement. For example, in Romanian, there are 8 words that collide with „fata“ if you remove the diacritics (fata, fată, fața, față, făta, făță, fâța, fâță).
Given that we have to use diacritics, spaces don't seem like a big deal.
* fată - girl
* fața - the face
* față - face
* făta - was giving birth
* făță - a small fish, or a child who won't sit still
* fâța - was fussing
* fâță - variant of făță
As you might infer from the first 4, Romanian uses postfix "the" and for singular feminine words you can't tell the difference if you use only ASCII.
Use a proper dictionary for that. The very nature of statistical models makes proper translation without context impossible for these systems, especially when uncommon words and diacritics are involved.
That has lead to phantom diacritics: reading letters in unfamiliar words/names based on what you assume they are. For example some pronounce Chirica as Chirică because they assume someone forgot to type the breve in ă.
The resulting text was understandable by the "computer people" but not the general population who did not use the networks back then, perhaps somewhat comparable to when some time ago USA parents encountered the "SMS slang" used by their teenagers.
There is one big difference: CLI utilities don't usually care about diacritics (though encoding issues can throw a wrench in that), but they care a lot about spaces. So putting spaces in filenames requires properly quoting or escaping parameters, whereas diacritics does not. That makes one-off shell snippets and scripts a lot more annoying (though TBH I tend to shy away from those anyway, these days).
That is what context is for.
Edit: can't save, downloading works.
Apparently it's quite easy to reach the 260 chars limit
In the meantime:
$ touch 1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
touch: impossible de faire un touch '1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111': Nom de fichier trop long
Firefox cut off at ~217, httpie at 255.
This can cause user confusion
sh, bash and cmd.exe are shit. The shell needs serious rethinking.
Eg. you'd type `ls -l "Spaced [TAB]` and it would turn it into `ls -l Spaced\ Name`. I remember similar annoyances with other special shell characters (eg. single quotes, dollars, slashes), but that all seems to behave sane now.
You have to remind yourself to do this manually in scripts if you don't want to see lines full of "No such file or directory."
One of the reasons the shell is broken is because the character they use as an argument array member separator is something that regular people use to distinguish between two words, such as in a file name.
$ cd $HOME
$ mkdir my\ dir
$ ls my[tab]
$ cd /
$ ls $HOME/my[tab]
ls: cannot access '$HOME/my dir/': No such file or directory
(edit: this is on GNU bash, version 4.3.48(1)-release but I've seen this behaviour for years)
And I experience the problematic behavior on my Ubuntu VM. However, I can get the above describe expansion behavior if I run: shopt -s direxpand
$ cat proba.sh
#!/bin/sh
echo "Using quotes:"
for i in "$@"; do echo "$i"; done
echo "No quotes:"
for i in $@; do echo "$i"; done
$ ./proba.sh "ho ho ho"
Using quotes:
ho ho ho
No quotes:
ho
ho
hoCmake doesn't support semicolons, because everything in cmake is a string, and ; is the list item separator.
PATH is separated by colons, so you can't add directories containing : to it.
"What do I use a folder for?", they ask, in the same breath that they request "some way to organize things logically".
The no-filesystem movement has worked hard to eradicate this scourge from user experiences, but I fear that this is the devils work. Computer users should know what a file is, and what its for - and they should know what a folder is for, and why they would want to create one to put their files into it ..
But yet: they don't.
It hasn't improved since the 80's. Taking away the users responsibility to understand these things, only makes computing worse. The fact that "special chars in paths" breaks things, also holds this factor into place, imho.
Is that the movement to store all your data as an amorphous pile of crap, and then provide easy-to-use search tools to actually find the content you're looking for?
On one hand, I really like the search tools that come from this. But I still like to actually organize my data, so I can browse it if I want to. Also, these search tools seem to only work well enough on macOS and fall flat on their face in Windows. (and no idea where Linux falls on this)
Meanwhile, I've got a single directory full of PDF files (over 60,000+) which I routinely "ls -alF | grep <search term>" for, and I've also got some PyPDF scripts for doing deeper content search - but yet I yearn for a way to automatically parse the filenames and organize things categorically into a folder tree resembling a word cloud, symbolic links and all .. one of these days ..
As you as you do anything programmatic in/out of these drives it all hits the fan. So I'd add to the original statement - "Avoid 'technical' companies with special characters in their name", it's just not right...
I'm wondering when the first generation of college students will start who have never used a physical keyboard to input text.
Also, emoji.
And even more fun is, when it mostly works, but then it doesn't and you notice too late.
https://github.com/whyboris/Video-Hub-App/issues/667
Any help would be really appreciated!
It broke on the first try on a jr hire's machine, the source checkout location was `C:\source code`.
The linux kernel is one of the offenders, if you check it out on Windows or macOS (which supports CS but remains CI by default) you'll immediately get garbage in netfilter, because it's an habitual user of having different files with names identical but for the casing e.g. xt_TCPMSS.h and xt_tcpmss.h.
It can be enabled on a per-directory basis like so:
> fsutil.exe file setCaseSensitiveInfo C:\folder enable
NTFS had support for this for decades now - it was designed that way to be POSIX-compliant.
It's shoddy software that lacks support for it, not the OS or the file system.
https://stackoverflow.com/questions/1976007/what-characters-...
ETA: not broken in a technical sense, but having to escape them isn't the best experience. So it's just easier for me to avoid spaces.
#!/usr/local/bin/sbcl --script
(load "~/.sbclrc2")
(require 'replace-all)
(in-package :replace-all)
(format t "file is ~s" (second sb-ext:*posix-argv*) (probe-file (second sb-ext:*posix-argv*)))
(let* ((args sb-ext:*posix-argv*)
(orig (second args) )
(newfn (if orig
(replace-all orig "(" "-")
orig))
(newfn1 (replace-all newfn ")" "_"))
(newfn2 (replace-all newfn1 " " "-"))
(newfn3 (replace-all newfn2 "&" "-"))
(newfn4 (replace-all newfn3 ":" "-")))
(when orig
(format t "renaming \"~a\" to \"~a\"~%" orig newfn4)
(multiple-value-bind (new-name old-truename true-newname)
(rename-file orig newfn4)
(format nil "new-name ~a old-true ~a new true ~a" new-name old-truename true-newname))))And it is one of the biggest reason I hate Unix shells as programming languages, it is a minefield. In fact I think that after a dozen lines, Perl is a better option. It has most of what shells are good at (i.e. running commands), but saner and more powerful.
E.g. Here's a random StackOverflow q&a about a Git pre-commit hook where the top-voted answer does not properly handle filenames with spaces : https://stackoverflow.com/questions/2412450/git-pre-commit-h...
However, the 2nd and 3rd most upvoted answers do mention "-z" option to handle spaces.: https://stackoverflow.com/questions/2412450/git-pre-commit-h...
Same goes for capitalisation. All filenames should be lowercase.
Maybe it's not strictly necessary, it can avoid headaches.
I mostly use the shell and navigating in directories with spaces is annoying, you have either to quote it or put a \ before each space. You also have to remember to quote everything, and in bash that can become complex, you start adding quotes everywhere to solve problems caused by spaces (or other special characters like *) in filenames.
So I prefer to not use them, a simple _ is as readable as a space. Only thing is that spaces gets rendered better on graphical file managers, but... that could have been solved (and can still be solved) by simply adding an option to render a _ as a space graphically if there is no ambiguity. I don't care that much since I don't use graphical file managers that much.
I'm old enough to remember working with 8.3 filenames in DOS, and while the length limitation was maddening, the space part never was. Then Windows 95 came out and all restrictions were thrown out.
Why couldn't we just have a file system that robustly supports long filenames, including variable length extensions, while prohibiting certain special characters - namely spaces, slashes or any directory denoting characters in files, and characters that have special meaning in regex context? (brackets, asterisk, etc.)
https://dwheeler.com/essays/fixing-unix-linux-filenames.html
node-gyp doesn't like it when there's a space anywhere in your working path. Stuff I was messing around with was all in ~/Code Projects at the time, and using npm install on some things just broke. Looking back, I definitely could have done a better job parsing the error messages but still...
There's an issue but it was closed in 2018 as "The workaround is to use a path without blanks" https://github.com/nodejs/node-gyp/issues/439
(defun mdy ()
(interactive)
(insert (format-time-string "%04Y-%02m-%02d")))
Then to create a date-stamped file name:
(defun file-mdy (file-name)
(interactive "sbasename: ")
(find-file (format "%s-%s.org" (format-time-string "%04Y-%02m-%02d") file-name))
(save-buffer))
Nobody seems to misunderstand this date format. US folks might find it annoying, but understand what it means.
It's cumbersome-ish, but can be made to work.
Then there's shell injection via files containing a newline character in their name...
The findnl script as part of fslint identifies problematic patterns, and has 4 levels of stringency, with "POSIX" being the most stringent. https://github.com/pixelb/fslint/blob/master/fslint/findnl
Spaces are used to separate parameters in the command line. There's also no real need for filenames to support spaces.
But, of course, if you mix the abstractions of metadata (filename) with location, things won't be trivial.
https://github.com/acmesh-official/acme.sh/issues/1408
This is the sysadmin equivalent of piercing your nose just to make your parents mad.
Let's test something: http://example.com/my silly webpage.html.
Hey look, HackerNews just broke a URL with spaces in it. And it's written in a Lisp dialect and all; it's not some Unix job cobbed together with shell, sed and awk. The language has a string data type, and strings are passed to functions without word-breaking interpolations taking place.
You know what else breaks on spaces? Basic everyday gui text manipulation.
Suppose that in a block of text we have the sentence:
> Please look for the Holiday Schedule 2021 file.
If you double click on any part of the name like Schedule, pretty much every text widget on the planet will just select only that word, and not the entire filename.
However, if you have:
> Please look for the holiday-schedule-2021 file.
There is at least a ghost of a chance that a semi-intelligent GUI can pick that out as a word.
There exist good reasons to keep identifiers as clump beyond just command line shells.
It's why we need encoding like %20 in URLs that never pass through a shell script.
I also don't use :, as I have ran into problems with both Bash and its completion and FAT FS. Unfortunately, I routinely have timestamps in filenames, so I need to use +%F-%H-%M-%S instead of simple +%F-%T.
One thing has improved, though: I have not run into problems with ěščřžýáíé (which my language is full of) for maybe a decade, except on OpenWRT where space seems to be scarce to support non-ascii.
Edit: I now remember one problem, getting images for a website from an OS X user, which used combining characters instead of direct code points (https://en.wikipedia.org/wiki/Unicode_equivalence#Example), but HTTP requests got normalized in some browsers, leading to strange 404s.
It's not fear that keeps me from using spaces in file names, it's habit.
If we're going to play this dangerous game, from now on I'll figure out how to use nulls (\0) in my file names, and make all the C/C++ programmers cry.
I do, however, use Cyrillic (UTF-8) in filenames, and I regularly try out if moving a file into ASCII-path will let some programs open it (half the time it's that when I am having trouble).
This might already exist, but I wonder about a terminal that was really just a multi-line repl to a language. It would be preloaded with libraries that replicated all the features of the gnu core utils, but instead of calling grep like normal, you called a function like grep("args"). The advantage would be that you had access to a full blown programming language at all times. So when you needed to do something more complicated you would still have access to all the standard language features. And when you didn't need that, your canned core utils like functions would work
Edit: thinking about it again, it might not have even been the space but the exclamation mark in my path. Or both.
Edit: replace $@ with quoted version which actually changes the behavior (I was wrong that the difference is between $* and $@).
https://www.cmcrossroads.com/article/gnu-make-meets-file-nam... explains the situation in GNU Make in 2007 (and I don’t think it’s changed since then, though jgrahamc especially could correct me). Not being able to use such features as $^ and $(patsubst) is severely debilitating for all but the simplest of makefiles.
One way to look at it is that people of a certain generation eschew spaces because the tools of their formative years simply couldn't handle spaces - but another is that the olds have learned that generally erring on the side of KISS ("Keep it simple, stupid!") isn't a bad idea.
The main culprit is GNU Make which does not cope with spaces in filenames. As far as it is concerned an array is a string separated by spaces so it gets very confused. Yes there are some partial workarounds, no none of them consistently work. You learn very quickly to check all code out in a file tree with no spaces in it, otherwise builds can randomly break in strange ways. It's not always clear up front whether Make is going to be involved somewhere in the build, so it's just easier to be safe.
I still don't use the special character of my name in my username because that has caused me many hard to fix troubles. Think "cannot recover user password because this user doesn't exist".
How can I not be afraid of spaces if this happens like every other day with every other custom tool ...
Browsers will take http://example.com/some name.pdf and automagically turn it into http://example.com/some%20name.pdf, and deliver the goods without a problem. But having that space in the URL is still out of spec, and will cause your web page to fail validation, even though it works fine.
PS-Microsoft is horrible about stupidly named folders being created and dumped in there.
Last I checked, the standard answer for GNU make is "Spaces are expected to break the tool, that's working as intended, it will never be fixed." And because we build our towering edifices of software on the pillars of the past, I can't guarantee to you that a project of arbitrary complexity won't try to cram a list of filenames through a make script.
- Spaces in filenames get transformed to non-breaking spaces by the filesystem;
- The filesystem treats nbsp as equal to space (just as case-folding treats A=a, B=b, etc.)
Now, argument parsing, mouse double-clicks, etc. all respect filenames as "words", and the output from things like 'ls' just work.
(Yes, I'm well aware that there are case-sensitive filesystems out there. I'd forgotten that iOS was one of those).
That being said, even after all these years I sometimes need to try a few times in order to get the quoting and the escapes right when communicating names of files with spaces through multiple layers of software.
Often they are not even valid UTF8 which, when you uncork the filesystem for the first time in a decade causes the most delightful crashes. The more years the better the aroma.
But also a "use a font that has a proper capital ß" hipster.
At least in my crazy old illogical head anyway.
On Unix/Linux we've grown up with case sensitive by default but everywhere else it still seems to be a problem now and again.
I should qualify this...I'm en-US so I have no idea what the experience is like for anyone else.
[Introduction](./Introduction.md)\\
[Chapter One](./chapter one.md)\\
Maybe we ought have to a different character signify the end of a name? Or signfiy a option section, or the next option section of a command?
Personally I wish console shells had chosen another delimiter than space, but here we are.
I wouldn't say it's always poor engineering though, especially the 'liberal in what you accept' half.
I still don't know how to process this emotionally. Either it is somehow naively really genius, or stupid.
In any case, it scares me, mostly because it is a non-IT person.
Except nowadays I worry more about user names that get fed into collaborating applications (with different edit criteria) and password characters (again for systems with differing, strange edit rules.)
Although lately I have started saving my Logic Pro files with spaces, simply because I prefer it to be the name of the song as-is.
It's not about fear, it's about making good decisions, and avoiding unnecessary complication.
Weirdly, my friend hates underscores. But he's a baseball fan
i still get issues with old one-off scripts, that still work, and I forgot to properly quote stuff... plus the urls are pain in the ass with the %20;s.
An old prof of mine used to send emails where the subject line was always a valid identifier in C.
Hello_dear_students_where_are_your_reports_
MISRA C:2004, 5.1 - Identifiers (internal and external) shall not rely on the significance of more than 31 character.
Yet to the machine (script, shell, program, ...) it matters a lot, since u0020≠u0009≠u00A0≠u2000≠u2001, etc. whereas the aforementioned codepoints render like this: " " (and yes, that's indeed the five codepoint in that order - at least I typed them that way).
(Ab)Using whitespace like that can lead to all sorts of funny business, not just when dealing with shell scripts and variable expansion.
Damn, I feel old now :P
01 - Metallica - Metallica - For Whom the Bell Tolls.mp3
Names like that were common, and had many spaces.