Comment by ggm | Hacker News Reader

Comment by ggm | Hacker News Reader

Parent Full thread

This is only going to get a shedload of "you do you" responses. BSD licencing is quite content to use ZFS, its pretty much ubiquitous now in widescale deployment and the source would be impossible to re-lock. Worst case is fork.

I very much regret the fragmentation of FS design, it has many mothers. "there can only be one" was never going to work, but we seem to have perhaps 4-5 more than we really need. ZFS manages to wrap up a number of behaviours cohesively with good version-dependent signalling so it should always be possible to know you're risking a non-reversible change to your flags. And, it keeps improving.

But, counter "it keeps improving" so do all the other current, maintained, developed FS and if somebody tells me they prefer to use Hammer, or one of the Linux FS models with a discrete volume-management and encryption layer, I don't think thats necessarily wrong.

Mainly I regret Apple walking away. That was about Oracle behaviour. It wasn't helpful. A lot of Apple's FS design ideas persist. I never got resource/data forks, it only ever appeared on my radar as .files in the UNIX AUFS backend model of them. Obviously inside Apples code, it was dealt with somehow. It felt like the wrong lessons about meta data had been learned. Maybe an Ex-VMS person went to Apple? Also Apple has a rather "yea maybe or no, dunno" view about case-independent or case-dependent naming. Time machine is good. Feels like it should fit ZFS well. Oh well.

I wish zfs had a 'tag' type of feature like Apple style fs. I have thought about making an object store to get some of the metadata on the fs level, rather than having to make some cobbles together solution of an SQLite DB of metadata with the files or something, but many of the object store solutions seem to be for fairly large files, rather than a few thousand pictures of family and such. Unsure what a good solution using zfs would be.

I am not sure what kind of "tag" you mean, but in any modern file system which supports extended file attributes you can attach arbitrary metadata to any file.

You just have to define whatever kind of tags you want and choose names for the xattrs in which to store them and then you should use them in your scripts or applications.

I am using all the time xattrs for things like file checksums and content classification, on XFS and on FreeBSD UFS, but almost all modern file systems support xattrs, except Linux tmpfs (which has only a partial support that is useless for those who use custom xattrs, so a file with xattrs cannot be copied to /tmp if tmpfs is used).

> except Linux tmpfs (which has only a partial support that is useless for those who use custom xattrs, so a file with xattrs cannot be copied to /tmp if tmpfs is used).

tmpfs is getting user xattrs in kernel 6.6, together with quotas.

> You just have to define whatever kind of tags you want and choose names for the xattrs in which to store them and then you should use them in your scripts or applications.

It is still just a oob store, you have to traverse the directory hierarchy old-style; the old BeOS did (and Haiku today does) something more: it indexed the selected xattrs and allowed live queries against those indices. This allowed for very interesting uses.

> tmpfs is getting user xattrs in kernel 6.6

I am aware of this, but the so-called user xattrs support most likely remains too bad to be of any use.

The kernel list message:

"Add support for user xattrs. While tmpfs already supports security xattrs (security.*) and POSIX ACLs for a long time it lacked support for user xattrs (user.*). With this pull request tmpfs will be able to support a limited number of user xattrs."

I am waiting to see which is the meaning of "a limited number", but based on the history of tmpfs, I suspect that this does not mean that the number of extended file attributes is limited to some value, but it means that there is a small list of names of user xattrs that will preserved (which are those used by some application that the tmpfs devs love, while they are hating the other applications), while the other xattrs will be lost, like now.

If "a limited number" is really a number, that would be much better, but this might still mean that the number is excessively low, e.g. ten attributes or less, instead of being high enough to not be a serious limitation, e.g. one hundred or one thousand.

> It is still just a oob store, you have to traverse the directory hierarchy old-style; the old BeOS did (and Haiku today does) something more: it indexed the selected xattrs and allowed live queries against those indices.

You are right and it would be nice if the file system itself would allow faster queries by xattrs than what can be achieved by testing the attributes of files selected by their names.

However, it is not difficult to create a database file indexing the files by attributes, but it may need frequent updates, to remain in sync with the indexed file system. I use such databases, but mainly for file system parts where I keep files for long-term storage, like books, research papers, handbooks, product documentation or older software projects.

Looking over the landed patch (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...), I cannot find a hardcoded list of user.* xattrs that will be accepted.

It looks like it will store as many xattrs as there is space for, based on their contents.

That would be nice, but then the "a limited number" expression does not make much sense.

All the file systems that support extended file attributes have some limit for the size of the attributes attached to a file, for instance of 64 kB.

After searching through the patch, it seems that the maximum number of xattrs is 128. There is also a size limit of 128 kB, which might be the limit for the sum of the sizes of all attributes.

If that is the only reason for saying "a limited number", then this is very good, the limits are high enough to not cause any problems in most applications, so tmpfs will finally no longer strip the attributes of the files copied through it.

Reply on news.ycombinator.com