A transcoding FUSE filesystem for music
khenriks.github.io
khenriks.github.io
On a related note, another FUSE filesystem I have used is ciopfs [1] which makes a case insensitive filesystem from a given folder. Very useful for running windows programs that are incidentally platform independent (as in written in a language like Java) and break on case sensitive filesystems.
For example: building a comic book library? Where do I store the publisher, title and issue metadata? Why not just infer it from the organization of the file system?
D.C.Comics/Batman/The Batman #123.cbz Marvel/Avengers/Avengers - Ultimate #12.cbz
I once built a text processing and conversion system that used a different scheme.
You mount or copy your text files into a folder "source" and it picks up whatever's in there processes it and dumps the converted result out to "sink".
Later I set it up to have some filters on the data. So "sink" turned into a hierarchy of different pre-filtered results.
sink/Car_Makers sink/countries/France sink/countries/England sink/Truck_Makers
And in the end, only documents that fell into one of those classes appeared in the relevant sink folder. At a glance you'd know if there was something relevant to that class of documents or not just by running "ls" or "dir" on it.
It cut hundreds of hours off of analyzing the results since you'd know a head of time if it was likely you'd find something productive related to one of the output classes.
This work with fuse is really exciting.
Imagine dumping in your text files wherever, fuse mounting it and then just exploring the results in all kinds of useful ways. Maybe you write an "entity extraction" filesystem.
So "Mobydick.txt"
Gets turned into
/mnt/mobydick/characters/ishmael/chapters/1 /mnt/mobydick/characters/ishmael/chapters/4 /mnt/mobydick/characters/ahab/chapters/4 /mnt/mobydick/characters/queequeg/chapters/3 /mnt/mobydick/characters/queequeg/chapters/4
/mnt/mobydick/chapters/4/characters/ishmael /mnt/mobydick/chapters/4/characters/ahab /mnt/mobydick/chapters/4/characters/queequeg
/mnt/mobydick/locations/Nantucket/chapters/1 /mnt/mobydick/locations/Nantucket/chapters/4 /mnt/mobydick/locations/Pequod/chapters/4
/mnt/mobydick/chapters/4/locations/Nantucket /mnt/mobydick/chapters/4/locations/Pequod
/mnt/mobydick/chapters/1/text.txt /mnt/mobydick/chapters/2/text.txt /mnt/mobydick/chapters/3/text.txt /mnt/mobydick/chapters/4/text.txt
And now you can use the fuse filesystem as an analytic tool
http://www.tagsistant.net/ http://web.mit.edu/6.033/2011/wwwdocs/writing-samples/sbezek...
Trying to shoehorn it all into the filesystem as directory names seems... destined for disappointment.
I've also figured out how to do audio streams (radio stations).
My understanding of Semantic file systems is that the links between files approximate directory traversal but the files or more or less that normal files you'd have on your file system.
The real direction to look is for systems where your tools are composable but also which have novel means of performing whole-pipeline optimization--like stream fusion in haskell, or a JIT'd Lisp machine. Of course, those examples are far from perfect, but they show some attributes of what partial solutions to the age-old problem might look like. They do have their own problems as well, like ad-hoc data processing in haskell being more thought-intensive (though, you know, correct in processing edge cases) compared to composed string filters in a shell.
You could take a job composition system like unix pipes, and add a buffer discipline that allows for zero-copy, or provide a way to declare buffering behavior, but at that point you're essentially building a typesystem on top of some real-world semantics, and it's a better idea to start from what you want to achieve in the first place and then design upward from there. If you start from the bottom up, you can also merge the operations performed inside each (memoizing pure work or fusing stream operations) or describe what kind of data you accept and produce, both of which can help avoid redundant computation. You'd probably end up with something with affine or linear types, a module system a la ML, and some novel combination of other handy features for ergonomics (which is largely what has kept unix pipes relevant to this day).
In concept, it should be relatively simple to recognize where a FUSE filesystem is mounting another FUSE filesystem and bypass the kernel to let the two processes talk directly to each other. You still have the problem that any operation would need to travel through N processes. You might be able to improve on this with a mechanism for a filesystem to say "I have to effect on operation X" and have it be bypassed when X occurs.
Like the scheme handlers in Polymorphic Identifiers: http://objective.st/URIs/
As a final step, you can export such a composition via FUSE, but that's just for legacy support.
I'd really recommend to anyone who wants to understand more about filesystems to write your own FUSE filesystem. It isn't difficult (assuming you have a basic understanding of C), they run in userspace so they can't mess things up too badly, and it is really gratifying to drag/drop files into a folder and have magic happen.
My car supports only MP3s, and only up to ID3v1 tags (will not play files with ID3v2 tags). That's the reason, why I convert everything into MP3. I'm not going to change my car or it's audio system for few more years.
When I get some music in lossless format like FLAC I encode it in Opus for regular usage.
I wonder what people here like outdated patented codecs. It's like a lose lose in all aspects.
Vorbis is OK, but Opus allows using lower bitrates for comparable quality, so it can save space and also should be more optimal for CPU utilization.
An interesting test: http://listening-test.coresv.net/results.htm
I keep FLAC for originals too, but encode in Opus for regular usage. FLAC allows reencoding in the future if anything improves, new codecs come out and so on.
I know opusenc uses VBR, but as far as I can tell there is no V0 equivalent that is accepted for high quality lame (MP3) encoding.