Hacker News Reader

Comment by selfhoster11 | Hacker News Reader

By analogy: if you were to store all your paper copies of bank statements, bills, mortgage papers, etc... Would you just dump them in one big pile in the middle of your living room, or would you sort them into vaguely themed folders to impose some organisation? Hierarchical filesystems are valuable to those that want to organise data in ways that tags can only emulate.

View on HN

Can I shout "Plumber invoice 2021" at that pile and the right document will come flying out? Or just "invoices 2021"? A pile that could do this would probably be fine for me.

As long as you know exactly what's in the pile I guess? A good folder/directory structure tells you pretty quickly what you've got available and makes it easy to browse and explore even if you've got no idea what you might find going in.

A directory structure : Invoices > 2021 > Plumber is easy to browse and makes it easy for a machine to retrieve it as well.

Please can I have all the Plumber correspondence from Joe the Plumber, regardless of year or document type?

Modern shells have double-star globs for traversing arbitrary numbers of layers, so I would do something like this with:

  ls **/*Plumber*

Thanks, you just got me all the docs from Samantha the Plumber and Amit the Plumber too!

And tags don't help with that, when you just tagged everything with "Plumber".

They could help if the tags were 'Occupation = Plumber' and 'Name = Joe'. Now a search for all files where both tags are present will get you Joe the Plumber's invoices. If you want everything from any plumber or from any person named Joe, then just leave off one of the tags from your search. It is very much like when querying rows in a relational database, just adjust your WHERE clause.

You just summarised the whole argument : you can go very far with a well structured relational database.

I agree. Unfortunately, right now the 'well structured relational database' is completely separate from the file system. Didgets was designed to combine the two into a single coherent system so that you can't update one without the other. By 'combining' doesn't mean I did what WinFS tried to do and just take a filesystem and a database and stick them together somehow. I built a completely new system from the ground up that incorporates traditional filesystem features (block allocation, stream management, metadata control, folder hierarchies, etc.) with solid relational database features (schema, tags).

Do you mean Joe the Plumber from London, UK or Joe the Plumber from London, Ohio ?

We all know how this ends up. It ends up being like Google where the search engine uses word embeddings and the like and removes word from your search queries or replaces November by December because they are both months so you can substitute one for the other right ?

Don't care, just give me both. Since I've never been to London Ohio I don't think it will be a problem.

It gets annoying though when you have just single invoices or documents that don't quite fit in your structure. I eventually started throwing all received documents in one folder and used the name as tags basically. E.g. 20210214_plumber_invoice.pdf or 20181204_someshop_invoice_playstation.pdf... (And yes I use the filename for tagging because I don't trust e.g. Windows tagging system to be there forever or be copied properly onto other operating systems.)

brute force grep -R for the win.

The difference is that if a file isn't sorted in a file system, it just sits on your Desktop or Download folder and is still easily visible. If you use tags and your file doesn't get tagged properly it just is lost in the pile and impossible to retrieve.

I find tags work very well for discovering other peoples content on the web, but really don't help much with organizing your own data.

One operation that seems especially problematic with tags is copying. If I want to modify something on a file system and keep a backup, I just copy the directory tree before I do my modifications. If I copy something with tags, I end up with two things with the same tags, which is not very useful for keeping them separate.

Also how do you deal with removable media (USB, DVD) in a purely tagged system? What if the tags on the media conflict with your own tags? Once you allow filtering by device, you are just reinventing the file system again.

> "The difference is that if a file isn't sorted in a file system, it just sits on your Desktop or Download folder and is still easily visible. If you use tags and your file doesn't get tagged properly it just is lost in the pile and impossible to retrieve."

Wouldn't a shortcut to a view of un-tagged files sorted by recent basically serve the same role as an unsorted Downloads/whatever folder?

The lost file might not be recent and it might not even be untagged, it might just be tagged incorrectly (e.g. `vacation-2019` vs `vaccination2019`).

The nice thing with a hierarchical directory structure is that every file has a place, even if a file is misplaced or misnamed, there is a good chance it will be near where it needs to be.

With tagging you don't really have that, it's just a pile and you have to hope that you can remember a query to make the file show up again.

The biggest problem however is that I don't see how you can actually work within a tagged system. How do you extract a `.zip`? How do you copy a file? How do you deal with removable media (DVD, USB)? Finding a file and handing it over to an app is not the only way we deal with files.

Your local file system is a work environment, where you are the one creating and modifying files. Tagging seems to works best when it comes to exploring around other peoples content, but that kind of exploration is not something I do on my local machine with my own files, since I already know where I put them.

Anecdotally, my short answer to this is 'no'. At least a Downloads folder has a loosely defined purpose. I have often gone back to try and find something I downloaded that's no longer available (or too big to download again), and it's tricky at best to find something already.

Now, imagine how many files are being shifted around on a regular basis. Temp files, cached downloads, automatic installs of software updates, all sorts of crap your IT department may remotely put on your work laptop, etc. Sorting by date isn't all that useful. At least with folders, we have an 'enforced' blast radius if random junk shows up; my temp files stay in my temp folders, install files stay where they should, system files stay with my OS, etc.

And don't get me started on how things can end up in odd places on mobile devices OSs.

The solution should be a combination of folders and tags. E.g. imagine a folder that just contains all your photos without any substructures. It would be easy to just select e.g. all "2018 photos", "birthday photos" or "photos of your parents" in there without needing specific subfolders for those things (especially since those subfolders would conflict with eachother).

But I agree that I wouldn't want those pictures to be mixed with e.g. other random screenshots or drawings that I made, even if they could be separated by tags somehow. So folders as a hard separation would still make sense.

If your memory is anything like mine, chances are you momentarily don't remember the word "plumber".

Hierarchical structures, while inflexible and sometimes prone to mis-categorization, provide navigational cues that tags don't provide. It's almost like with GUIs vs CLI - if you know what's already possible and want to express yourself precisely, you want a CLI (tags with lots of Boolean operators to precisely include/exclude). And conversely, if you don't know what's already possible, but could figure it out if you have the options laid out in front of you, then GUI (a hierarchy with all choices already laid out) will be more relevant.

I keep passports, degree certificates, deeds, health insurance docs.. the things I would grab if I was running out the door, in a single box file.

Everything else is basically unsorted, maybe vaguely sorted by date of putting on top of the pile, by placing things together after searching for them once, or by 'I think I know where I saw it'.

I have tried putting everything in themed folders, it's a waste of time. The time spent searching for something is much less than the time spent organizing everything in advance. The modal piece of paper will be thrown away after a few years without ever having been needed.

Is that not effectively a first-level hierarchy with no further subdivisions? The "important stuff" category and "everything else" category are already a useful taxonomy, even if very minimalist.

One of the biggest problems with folder hierarchies is that files can often be classified in several different ways. To take your paper statements analogy, do you organize by year, by institution, or by category? What if you have a 2002 bank statement? Do you put it in the '2002' pile or the 'bank statements' pile? Using existing file hierarchies allow you to store the digital document in the '2002' folder and then create a hard or soft link in the 'bank statements' folder, but that can be a hassle. Tags allow you to attach them to documents, photos, videos, etc. without worrying about how you might organize them. Luckily, Didgets lets you organize your file using either a hierarchical folder structure or just by using tags. It is your choice.

To me, it seems best to exclude as many possibilities as I can during each step of the (naturally recursive) search. Filtering by "is bank account statement" excludes a lot more files than "was incorporated into my files in 2002", since most people only have a few bank accounts but a lot of photos, videos and other things that they create or download in a given year.

I think the best system is actually a mix of hierarchy and tags. Top-level, very broad "semantic zones" (aka is this .PDF a bank statement, a cake recipe, a textbook, or some temporary file from the browser cache) would lend themselves to being represented as a shallow hierarchy, and items within a specific semantic zone could be then freely tagged or further subdivided into a hierarchy, whichever approach makes sense for that particular semantic zone.

You assume that there are fewer bank statements than 2002 files in your argument. What if you loaded in a million bank statements in 2005 but only created a few thousand files in 2002? With Didgets, I can tell how many objects have each tag attached so I order the search to eliminate based on how likely the set I am searching for has each tag.

How many bank accounts does an average person have? Even for the most extreme cases, we're looking at the low hundreds of statements annually, at the maximum. If you're not an average person but instead a business or an archivist, then you need a custom system anyway.

I'm really not trying to criticise or diminish the value of your system. All I'm saying is that even without an additional tag (or hybrid tag+hierarchy) overlay, a hierarchical system can be quite useful as long it's well thought-out by the user.

Reply on news.ycombinator.com