Computer Files Are Going Extinct
onezero.medium.com
onezero.medium.com
First-year CS students (middle school or high school) with experience using Apple products don't have a concept of a filesystem. The shift to auto-saving documents also corrodes the intuition that files get stored on disk in some non-magical way. In the same way that it's now easier to teach networking and graph theory because youth experience identity and relationships this way, it's now a lot harder to teach lower-level abstractions. In my experience, this has changed over the last five years.
She has her students go into the field and collect data in the form of photos, spreadsheets, and typed reports. She wants students to bundle these files in zip files, and email them to her.
Students have no idea how to do this. Some of them struggle with emails. She has had to make her tutorials on this longer and longer. And she has also been forced to start accepting assignments as a pile of attachments, because so few students understand zip files.
When I was in high school the vast majority of my peers understood how to bundle up a bunch of photos, documents, etc. into a zip file and e-mail them. This is because my peers were nerds, and we were excited about this new computer technology and eager to learn everything we could about it. There were no computer classes in high school. When we had high school science class, we were expected to draw pictures of our experiments (y'know, with a pencil, on paper) and fill out the lab report longhand. There was a large segment of the student body who had no idea what a file was, or how to turn a picture into one, let alone how to bundle it into a zip file.
It was a bit different in college, where we were expected to understand Excel and how to run a regression or graph some data. But then, that drew from a very different population than high school.
It's sorta like how SATs are dropping not because kids are getting stupider, but because a larger proportion of people who would never have been considered college-bound a generation ago are now expected to take the test.
I see it as progress - which destroys meanings as much as liberating new meanings. A word becomes a poetic palimpsest of needlessly complex layers.
I am an engineer-type, I like concrete words with focused meanings, I failed English at school while excelling at the hard sciences. The loss of meaning disturbs me, but I value the gains, and I accept that new language requires freedom from old dictators.
I don't think that's the reason. I did study with non-nerds and they had to deal with computers because they wanted music, or to read an article in the internet or print something with Word/Windows and install the printer drivers.
What changed now is that things are more "streamlined" and all the complexity is abstracted. Software companies have figured out that people struggled with files, so they removed them. This can make the average person less computer-literate since he doesn't have to learn much to be able to handle the computer.
Because of that, I prefer email. I'm more than happy to pickup new technology and use it but I have no desire to do so if it's going to take more time to do something I already have little desire to do.
My current location literally has 8 communication mediums and its miserable to deal with. Then I send emails and people complain... it's like people enjoy wasting their time.
I think 1 striking manifestation of the loss of the concept of files is that if someone wants to share a Note from their iOS device, the most common way for them to do it is for them to likely take a screenshot of it and share that image.
Since conceptually a note is not a thing of its own, but rather a part of the Notes app, people rightly believe you cannot share it on its own, and so need to go with the all catching screenshot fallback.
Perhaps I am just in that brief period of time where more people than just the nerds knew the basics of the filesystem.
This is in the same population of students. I noticed basically that IO tasks were eating up an increasingly big fraction of effort and time early on in courses, reflecting problems that didn't exist previously.
This seems to be for the reasons stated, that file IO stuff is being abstracted away. What used to be typical necessary computer fluency is now "low level" for many.
There it is.
It isn't just education, obviously. All software has to be subjugated to an ease-of-use that buries details.
I very rarely send emails at all outside of work. Maybe one or two support emails. I also fairly rarely create zip files either. Of course, I know how to do these things, but I don't really do it anymore. I haven't needed to, for the same reason that I don't own a printer.
I haven't needed to "compress" anything since Google Drive and Dropbox meant that all I needed to do to get someone a file is type their email into a box.
Perhaps instead of email, you teacher friend could find out how her students send things to each other and meet them half way? Make up a google drive or dropbox or whatever microsoft or apple are doing instead of insisting on whatever's most comfortable for you.
In short, I don't think this is an example of students falling behind, but teachers getting left behind.
It's a little rudde to say it that way, but I really don't know how else to say it.
I suppose, in the same way that people using crayons are moving away from the printed word.
> Perhaps instead of email, you teacher friend could find out how her students send things to each other and meet them half way? Make up a google drive or dropbox or whatever microsoft or apple are doing instead of insisting on whatever's most comfortable for you.
Isn't the whole point of school to learn new skills? If students have illegible handwriting and poor grammar, should the teacher really be meeting the students half way?
I actually told her that she should continue doing what she was doing, because it was a real boon to her students. They might not ever learn this stuff.
Her students objectively less able to share information with each other. I have certainly seen kids who have to go as far as taking screenshots of what an app contains and send a screenshot to someone, than actually be able to export text.
I could understand if what the students were doing was in some way comparable in flexibility, but it isn't. They basically only know how to click on the share button in an app, and text links for someone else to install the app. Or send them screenshots of them using an app scrolled to the relevant info.
But the biggest benefit is that having assignments in discrete organized bundles makes the most sense for everyone.
But the new way is exemplified by iOS. You buy pre-packaged "solutions" that keep your data...somewhere...safe, presumably, and away from you so you don't screw it up. If it's something you should be doing, there will be "an app for that" that you can buy, so don't worry your pretty head. If we gave you "tools", you'd just hurt yourself. Why not stop worrying and buy something nice. We have songs, we have shows...you can send a funny animoji to all of your friends. Good times. You look awesome. You're welcome.
And for market success, they seem to be right, although they're creating a situation where, if you don't have to learn to use a toolbox of specialized tools in customized combinations, you'll never learn how and really WILL need to rely entirely on pre-built "complete solutions".
True story: while performing support for development tools, I asked a developer for a source file. They opened it in their IDE, took a screenshot, pasted the screenshot into a word document, hit page-down in the IDE, took a screenshot...
We got a several dozen page word document filled with screenshots of the text file.
> and meet them half way?
In other words, OP is not suggesting that you force students to use another method, but work with them to understand what makes sense to them. In your specific anecdote, if the teacher showed the students how to share hyperlinks, or how to get text dumps and share them instead of screenshots, maybe they would respect her for that.
I used to export the text of my travel itineraries, and go through several steps to get it on my phone. Now I just take a photo of the text on my screen. It's just one step :-)
They have custom HTTP JSON RPC protocol, it uses POST and GET verbs. Their web site calls that one. They also have more performant TCP binary endpoints for use by native clients, with the same payload packed into custom binary format.
For users, it's a web app with download and upload links.
The "zip up your files and email me" is only hard when the concepts of files and email are alien, because everything lives in its own silos. Which is the point.
1. The privacy policy of Google & Dropbox is a catastrophe.
2. They can ban you at any time for all sorts of silly reasons and then you can't even turn your homework in any more.
I don't know if it's a coincidence or a department policy, but Google gets them early and if you think that's bad, teachers aren't to blame - maybe schools are or possibly the education dept.
Honestly I was a little surprised that there was no discussion about it. I talk to my kids about the way the web and privacy is going and I don't expect schools to do everything but they should be a little more aware of the issues imo.
2. The professor tells you to create a new account and submit it via that. Sure it probably violates the ToS but nobody cares.
They probably just don't. People are sharing images, text, and links. Mostly on non-accessible, ephemeral, and unreliable platforms. They are not used to share generic data and not used to reliable platforms at all (as even email is unreliable nowadays)... except at work, and that's where people use email.
The problem is all these services try to control as much as they can. So interoperability becomes whatever contracts on the use of APIs they can sign with each other. You as a user have nothing to say.
"Please turn in your assignments via Snapchat by 7AM tomorrow"
Another contract made a Facebook messenger group and shared all versions of files by pasting them in to the messenger chat.
Wasn't particular happy about those but fortunately both were fairly short term things and to be honest I couldn't really think of anything better that I believe I could convince them to do. The Line accountant was definitely not going to use anything other than Line.
The FB Messenger file sharing, well, what. Email? Dropbox cost $$$ per user. Github requires more training. I suppose maybe Slack would have been better than FB messenger but still requires getting the other party to install apps and get used to a different workflow, So yea, I just put up their chosen method of communication for the duration.
And despite this vast monopolistic ecosystem DaumKakao put ads in the app this year. Greedy.
Erm, you need to have an email to register for any kind of service out there, so "know how to use email" is pretty much a requirement even for everyone. And this is not going away, unless you want a future where everyone has to be reached thru uncompatible walled gardens.
I suspect it is the same progression as any other new technology that undergoes mainstreaming. Take automobiles for example. In the early days if you owned a car you either made yourself something of an expert (and if you were an early buyer you were probably kind of an enthusiast already) or you hired one. Today outside of enthusiast circles they are just an appliance: you get in, turn it on, and go do whatever it is you need to do.
But people still understand that they need to put gas in their car to make their car go, and if they run out of gas, their car can not go. We don't have to know what gas is, but it is still a quantity of something.
But our concept of files and data is becoming so abstracted that we do not even know where our data is anymore, how much we need, or how long it takes to transfer it. But until we have much more bandwidth and storage, there will be problems.
Imagine a world where we tried optimizing our cars interfaces so much that we removed gas gauges, and just had reminders that told people that they had to go to a gas station. And as a result people would frequently run out of gas, or be shocked by how much gas costs. But gas gauges were considered a power-user feature, and everyone was convinced that users would just never be able to understand them.
Nowadays, carburetors are gone, replaced by fuel injection, which controls the fuel mix so tightly that sparkplugs rarely get dirty. And the whole assemblage is so tightly tuned that if you need service, you take it to a dealer and let a professional deal with it.
The world you're asking us to imagine already exists, you've just forgotten enough of the old world that you only remember the gas gauges.
Or, simpler, wireless powering in most parking spaces so for most people never even having to think about power
Or, robot chargers. Put a QR code on your plug, maybe in infrared so it's invisible, and some simple device and find and start charging your car anytime you park it.
Anyway, as an old computer literate person the idea of not understanding files scares me but if people are getting by without them then it's just probably me being out of date.
Oddly enough the iPhone doesn't do this, it will happily tell you how many GB you have left when what the user really wants to know is "How many more songs or photos can I store?".
I'd argue the problems will only start when there's enough bandwidth. That's when companies will be able to fully transparently store everything remotely and that's when people will really get screwed. Because if Google locks down your access for TOS "violation" then even having a needy friend will not bring back fully remote data.
Of course the idiot light has achieved an apotheosis in the era of computerized cars, where the car's onboard computer signals a problem and illuminates the "CHECK ENGINE" light. Of course, what that means is that you need to take the car in to the garage and have a Certified, Licensed Professional Mechanic attach a doohickey to the engine computer to tell him what the problem is, so he can tell you.
My father is an old-time car geek. This shit drives him up the wall.
>> But our concept of files and data is becoming so abstracted that we do not even know where our data is anymore, how much we need, or how long it takes to transfer it. But until we have much more bandwidth and storage, there will be problems.
I think you're mixing up analogies at different levels of abstraction... yes people know they need to fuel their vehicles, and they know they need to plug their computers into power or charge the battery, and connect to the Internet. People probably don't know much about fuel injection, and plenty of them probably don't know what a spark plug is. It's possible that they also don't really need to know what the underlying storage abstraction for data is.
As others said, that’s because they don’t need to do that.
In my experience, they will know how to share their Google Docs (which is where their spreadsheets and typed reports in all likelihood get written) with you.
They probably also will understand that editing said documents after informing you of their existence may not mean you’ll see said changes.
From what she has told me, and from what I have seen, some of them don't even get that far. Their idea of sharing information is to send screenshots in MMS messages or over snapchat.
And, I think that if you graduate high school, you should have these basic skills. Just in the last year, I have emailed zip files to several different people, like my attorney, or when I applied with the local town clerk for a parking permit. There is never going to be an app for everything.
And call me old fashioned, but I think a high school graduate should be able to write a check, put it in a physical envelope with a stamp, and mail it. And if someone lets you expedite something by emailing forms to them directly, you should be able to do that too. God forbid a medical office asks you to fax them something. (Which I still have to do several times a year)
When I was in high school, everyone knew how to use AOL to send word documents and pictures and even mp3s to each other via email. Everyone. These are skills that can be taught.
Doing that was much easier in the days when computers came with fax/modems and most places had landlines. Now you have to search for some online service that will inject adverts in your fax or go to a copy place (if you can find one) and pay lots of money to send a fax. I'd almost rather mail a certified letter with return receipt instead :)
Office supply stores will often send faxes.
I'd say we're seeing that filesystems weren't a great user experience, at least the way they've been implemented in the past, and I'd place the blame on the other side. The design of computer systems today isn't the only possible way to design them. They exist to serve users. If our old systems aren't well-liked by users, and we spend all our efforts hiding their inherent design so people can actually use them, maybe our foundation is wrong.
Even the most technical people in the world have been trying to replace our filesystems with object databases for a few decades now, and plenty of people use only "services" (database in the cloud) or "apps" (database on phone). Isn't it possible filesystems have outlived their usefulness? For a lot of people, they really only exist to boot the OS.
Computer science students need to learn about filesystems and C and Unix, sure, like cooks and carpenters need to learn US customary units.
The comments about file systems are interesting, but I doubt that programming is going to get much easier just because you store your source code in database tables instead of files.
I first thought he was asking if there was some kind of Recycle Bin on Android, but then he said: “when we die, our body is burier or burned, that is it becomes bones or ashes, but what happens with the picture on the mobile phone, what does it become?”
And they remain in the same place until the file space they occupy is reclaimed by the OS and rewritten with new content.
Long before I studied computer science concepts, I was enamored with computers. I was a power user then. When I was in elementary school I couldn't have Internet access any time I wanted (it was literally a call on the family telephone) so I spent lots of time playing around with the operating system and the installed applications. Many nights were spent exploring the nooks and crannies of Windows and Office. I learned about the cmd.exe and wrote batch scripts before I had any idea what computer science was about.
Later when I moved from Windows to Mac in the early 00's, I did the same. The same kind of curiosity led me to naturally explore the various system utilities, from Mac-specific like hdiutil or diskutil to general Unix-y.
I don't know whether this is fighting a losing battle, but I doubt anyone who's not curious enough to learn about the innards of their computers can really become successful CS majors and hackers.
It's sad because I feel like I and a handful of other people in the program were the only ones actually enjoying it. It probably also has a lot to do with older people telling so many kids as they were growing up that they're "so good with computers" because they showed them how to setup their email accounts and it gave them a false sense of skill that made them think they should do it for a living.
I got the same "I'm not good with computers" response.
Modern web browsers are the cause of this they don't distinguish between an address and a search query -frustrating.
Yet I could not fix my car, my microwave or my refrigerator if my life depended on it.
Does it make me less? Is it wrong that I use my car 2 hours a day and the only thing I know is to press a bunch of buttons?
Not for me. I enjoy CompSci technology. I wouldn't like to spend my time fixing my car.
Google is in the process of making it worse by hiding URLs in the search results. (It's also a security risk. Google's site-name detection often makes terrible mistakes and points users to malware.)
I now have to teach teenagers the concept of "computer".
The idea that an app is a "recipe", a set of instructions, and something needs to execute that recipe, a "computer"--aka your phone, is now a foreign concept.
https://en.wikipedia.org/wiki/Constructionism_(learning_theo...
?
A lot of current approaches prioritize scalability over everything else, leading to canned and sandboxed curricula.
I'm curious, though, whether this is in tension with your understanding of constructionism.
I showed my son at a young age terminal commands to browse the filesystem. It really seems to be helping him now that he coding.
Can’t happen soon enough imho
It's like cable TV, originally one paid a premium to avoid advertisements. Today it's just the norm, and you get the advertisements anyway.
My project Aquameta is one such attempt, trying to rethink the programming stack, and by extension all user data, as relational data instead of files, with the filesystem still there under the hood, but basically as a bootstrapping mechanism to get the database running.
Here's an essay about the reasoning and approach:
http://blog.aquameta.com/intro-chpater2-filesystem/
The TLDR is that the filesystem lacks a sophisticated information model, and doesn't provide mechanisms for defining structures, and we really need to embrace this basic requirement at a systemic level.
Boot to PostgreSQL! :)
Can you ELI5 what this means? Very curious to understand ideas on alternatives to the traditional file-system abstraction.
I do a lot of self-study, and I use Calibre (ebook library software) to organise my reading materials. I used it because my Linux file browser has problems displaying covers for EPUB files. (And a couple of other) I like that my reading material has lots of ways for me to organise it without creating duplicates (tags mainly) search and browse my books. Then one evening I found myself wishing I could have all my other relevant 'files' in the same system: audio, video, source code, repos, blogs, podcasts, notes etc... Basically a metadata rich database - except not really a database in my case as I need to pull in 'stuff' from all over the place, and not only on one device.
Files are nice, and I hate anything that takes away my ability to organise stuff on a physical level (which, truth be told Calibre stubbornly refuses to let you set up your folder system, so I cheat by using tags) but at the same time, the problem with the skeuomorphic paradigm of files and folder is that there are so many more options open in the digital world than in the real world when dealing with items.
Imagine being able to sit at your office desk and open the drawer to pull out a piece of paper. When you look at the paper it shows you some text. Rotate the paper to the left and the text disappears and a video starts playing, turn it to the right and you hear a podcast, place it on your desk and 4 more pieces of paper appear next to it - or it calls someone relevant to the info.
All possible with digital info, but not with a real piece of paper - yet.
That is the only thing I see as a real negative about files and folders - it may be limiting what we can really do with our digital world because we're stuck in a file-system metaphor created hundreds, if not thousands, of years ago.
So rather than going to /home/kaslai/images/ to (hopefully) see all my images, I'd instead query my filesystem, perhaps like /owner:kaslai/type:image.
The specific syntax would need some serious thought, especially if we want to make it fairly ergonomic to slot in to existing systems.
There are a large number of approaches that one could take. I'm personally a fan of a hybrid, where you could navigate to a location and then make queries on anything within that location.
IMO one of the coolest features that such a filesystem could have is dealing with music. There's no perfect way to map CD rips to a hierarchical filesystem, but that wouldn't be a problem if I could just open a "directory" like /home/kaslai/music/?artist/?album/ which would automatically make virtual directories based on the artist and album, or I could just use /home/kaslai/music?artist=Smash\ Mouth to get a list of all 182 copies of All Star that I have in my library.
Basically the idea is that the OS identifies and indexes all data on disk providing primitives to access content by query rather than only by a hierarchical path.
This applies to user content but also object code, libraries (providing versioning, compatibility, integrity, whatever criteria.)
"You're so lucky you grew up using this technology, so you're used to it whereas I have to struggle - yes, do ignore that I had more than your whole lifespan to get up to speed on this ubiquitous reality"
but I encounter people under 24 that can navigate a touch screen pretty well but lack any fundamental knowledge, let alone typing on a keyboard in a proficient way.
fortunately most activities no longer require this kind of thing. auto-saving, and better UI solves alot.
I can see how wanting to teach these things will have challenges.
I don’t know. On the other hand, most older people like myself have had to deal with data loss from failed, obsolete or lost storage devices. There’s still no “one size fits all” very-long-term solutions for consumers.
If I try to view this from a non-technical persons viewpoint, this shift has been transformative, really. Most folks don't care if their data has been transcoded a million times as long as its still useful and easily accessible. This has protected a ton of businesses and users from being at the mercy of their IT team.
Every service is "extremely high reliability" until it isn't. Sometimes, the causes of failure are even beyond technical ones. Look at the situation with Adobe Creative Cloud users in Venezuela-- the discs and wires are still fine, but customers are losing real value because of legal mandates. Your high availability platform can withstand a network cable cut, but can it withstand a court order?
If you're still thinking local first, it encourages awareness of multiple baskets. iCloud is down? You can still pull the photos off your camera's flash card, or your workstation's SSD, or the external spinning-rust hard disc you used for cold backups, or the third-party dedicated backups service you use...
Until they try to print their photos for the family album and discover that pictures taken by their high-end camera or an expensive iPhone were degraded by successive transcodings to the point they look like garbage in print, and at that point there's no way to fix it.
> This has protected a ton of businesses and users from being at the mercy of their IT team.
And put them at the mercy of third-party vendors. It's a trade-off, but having a local IT team has its benefits.
Consider: unikernels (like, say, any old cartridge game ROM) don't have any need for files. They have precisely three abstractions they deal with data in terms of:
• a .data section in ROM (maybe needing bank-switching to get in place);
• some kind of byte-addressable NVRAM (like battery-backed "save RAM", or CMOS memory) either bus-mapped, or through MMIO.
• tape or (floppy) disk, sometimes at an extremely low level (send commands to the drive motor, write guard nybbles, etc.), sometimes through a DOS where you can just request to seek to a given track then read or write a given sector on that track. Either way, more like a block device than a filesystem.
---
For today's kids, I'd suggest: don't start with files. Teach key-value storage first. Interact with a library like LMDB without explaining where it's putting the data. They'll understand this just fine.
Then, teach object storage in terms of key-value storage. Object storage—especially once you add object versioning—is much closer to the modern metaphor that user-facing apps expose. You compose a complete new version of an object in an in-memory scratch buffer, and then it atomically replaces the previous object. You can't corrupt an object by half-saving it. Etc. Again, don't bother explaining how this works yet; just give them a scripting runtime hooked up to a Minio instance.
After they get that, you can ask them what they'd do if they needed to create an object that wouldn't fit in memory. Then you can explain block devices, as a "place where large scratch buffers can live"—but don't force them to figure out how to allocate those buffers from the block device! That's gonna pull in a whole bunch of prerequisite teaching. Instead, pull out another API: Linux's LVM. Logical volume management takes block devices in, and spits block devices out. The logical volumes are the scratch buffers. Explain mmap(2), and how these buffers end up a lot slower that memory buffers. Explain how these buffers survive a disk crash.
After you get to that point, then you can explain that all the other ways computers durably store data are built on top of these logical-volume durable scratch buffers. You can explain how LMDB works in terms of disk pages; and then you can explain how a content-addressable storage works in terms of combining an LMDB-like KV store with hashing and splitting.
And, after that—if you like—you can explain that sometimes, when we need something that's like object storage but where everything in the storage bucket is actually a tiny scratch buffer, we use filesystems. You can explain what an "extent" is by talking about how something like LMDB, that has a "freelist" of pages from its underlying logical-volume, can reserve a contiguous set of those pages and then let something else access them. Then you can explain a filesystem as a key-value store that has buckets called "inodes" with page-range keys, and extent-address values. (And an associated versioned object store of directory-objects, where each object is a serialized list of records (dirents), each containing a reference to an inode and giving it a name and other stuff.)
Filesystems are hard!
Huh, no. Just show how the FAT file system works. It's still used on USB drives.
Now that you have a simple model for the low level, you can gradually add complexity such as standard file operations, caching, wear-leveling, etc.
You say Minio instance, I say hex editor.
But the point isn't to teach students about a filesystem; the point is to teach students what files are when they don't understand why you'd even want the abstraction that "files" represent; when "files" aren't an abstraction they're familiar with compared to other abstractions. You have to justify "files" in the context of things they do know—to explain what purpose files would serve if they were invented today; what reason you'd introduce a filesystem into a data architecture that had all these other things (KV storage, object storage, logical volumes) but not files.
Students would ask: Why are files streams? Why are the streams arbitrarily writable, when this means files can become corrupted from a partial write? Why are files extendable, but only at the end, not in the middle or the beginning? Why do files have names? (And, once you explain inodes+dirents, they'll ask why there's a directory tree if hard links exist; and why it's the dirents that have the names, but the files themselves that have the other metadata—xattrs; alternate data streams, etc.) What is the difference between a file stored in a directory, a file stored in a filesystem stored in a mounted loopback image stored in another file, and a file stored in a subvolume or filesystem container? Why do directories need special system calls to read and write them? Why is a virtual filesystem used in many OSes to represent kernel objects like named pipes and device nodes? Etc.
(This isn't even a hypothetical; engineers at AWS and GCP probably had to answer this very question when asked to justify building EFS and Cloud Filestore. Why do we need a filesystem as a service, when we already have these other services that provide these other abstractions?)
This is not altogether unlike teaching a student what a programming language is and why you'd even want one of those, when they're immersed in an environment where software is created without them. Would you just sit down and show the kids C, because it's small and easy-ish to understand?
A filesystem is a data-storage abstraction, like C is a machine abstraction. But neither are primitive abstractions. They build on numerous other, theoretically purer abstractions. It's much easier to explain the motivation for the creation of a language like C, if you already understand the motivation of the creation of a simpler language that rests upon fewer other abstractions—like ASM, or like Scheme. Likewise, it's much easier to understand the motivation behind the creation of the abstraction that is filesystems, if you already understand the motivation behind the creation of simpler data-storage abstractions, like logical volumes or memory-page-backed search trees.
"Files" are a very specific abstraction; thinking they're the only way to transfer chunks of data around is a symptom of lack of imagination. There are very similar abstractions to files, such as object-store objects. The only practical difference between files and objects is that updates to objects are transactional+atomic, such that nobody ever sees an "in progress" object. But an object store (backed by a block device) is a simpler system than a filesystem (backed by a block device.)
You can control the objects that represent your data. You could also control, via an RBAC-like system, the K-V or tuple-store records that represent your data. Or the Merkle-tree commits. Or the blockchain transactions. Or the graph edges. Or the live actor processes holding in-memory state. You can transfer all of these things around between computers, both by replication (copying) and by migration (moving.) What do files have to do with any of this? An accident of history is what.
I'd also want to teach beginners in a way that lets them keep building and tinkering outside the environment I provide for them. This is where I'd love to see schools providing and encouraging the use of hosting space and resources like Minio, so that students can just count on them being available.
At the same time, I can see a great equalizing force in all this. Computers have reduced the technical acumen demanded from their users to the point that owning a computer and using a computer and even being a "computer enthusiast" doesn't put you very much ahead of anybody else on-average when it comes to starting out towards becoming a computer scientist. I think this, in combination with that everybody wants to be a software engineer these days, will eventually put us somewhere around the 1970's in relative terms with regards to technical literacy in the cohort of incoming computer science students. This might sound nightmarish to the current establishment, but it has at least a positive side-effect: computer science is getting more accessible because teachers can no longer assume that pupils come from a background of "quasi-technical" computer literacy (again, this is because conventional computer literacy has become decreasingly technical in nature).
I've heard one of the general causes behind the lack of diversity in gender and economic-background in tech workers is, at some point around the mid 1980's, CS instructors started asking of their students tasks like: "Open your editor and type..." and someone in the classroom would raise their hand and ask "Ah, um... what's an `editor'? And by the way, I don't own a computer either" and the instructor's reaction would be to privately advise that student to seek a different major, at best, or open derision at worst.
So I think we're getting away from that, which is at least a way to look at the bright side. It does make the teaching job a bit more challenging.
On a more personal gripe, a minor irritation of late is the number of students who want to do their OS homework in the Windows Subsystem for Linux, instead of even setting up a basic VM.
That's easy to solve, and can give the students a history lesson.
Give them some tasks that work with ~/con :)
Or you can have them work with files all named the same but differ with case. Windows based machines get "confused".
The iPad is strongly tied to the workflow that Apple suggests, and that's a workflow that removes files as a concept, abstracting them away. It's quite possibly a genius approach in terms of how accessible it is to non-tech people, even though it doesn't sit right with me.
And while iPod could be connected as a drive via firewire, that was never the way to add/remove music there, since (for better or worse) it was never a dumb "play this folder" type player, but maintained a associated library format with metadata etc.
That latter choice also put a human in the loop, since you were probably interactively importing media (whereas media-indexing usually happens in the background.) This meant the human could be relied upon to notice and fix bad ID3 tags, which was a big problem for most audio files of the era and made many MP3 players a useless "unlabelled warehouse" when you had a lot of files on them.
Great memories
You could never drag music to the iPod and have it recognized by the music player.
You could always drag random files to it and have it work as a hard drive. On Windows it was NTFS (Fat32?) on Macs it was HFS+
And even with music it was easy to drag music files from an iPod; they were merely in a hidden folder. I don't remember trying to drag music to an iPod with the Finder because it used to be so easy to do that with iTunes.
That's not a tool you can use to create things with, unless it's music, movies, photos or videos. It lacked the freedom to permit innovation.
(I have never acquired an iPad beyond the first one I got, which was a gift.)
My parents went from "scared of computers" to "ipad" pros, because it's really hard to break or get hacked, it doesn't get viruses they know how to force quit an app.
I'm scared of iPad getting files now, because it's going to be a mess for me.
I'd want to use command line tools - or UI's that are just as composable - to manipulate my files.
The iPad perspective just seems completely crippled from the POV. Like a toy.
Just to take an example - look at the Moon article also on the front page right now: https://medium.com/@sulej.robert/the-moon-made-twice-at-home...
The obvious buildup is to when the iPad transitions to a notebook computer.
You could create many of humanity’s greatest works on it! What are you expecting more?
On your iPad, point to where your latest video file is. Show me. Where is it? "On The Cloud"? On the iPad itself? Three clicks deep in some random purpose-specific app separate from the soundtrack file you want to integrate into the video? What about five years from now, can you still access that video however and whenever and wherever you please?
Why do you care where the 'file' is? That's an implementation detail. I can copy the video to somewhere else any time I want. I do this all the time with photos and videos from my iPad. It's not the issue you're saying it is.
I know people who are not able to properly manage their own files [no quotation marks needed]. They have copies of files here, there, and everywhere else and when asked which copy of the photograph they edited to their perfection they cannot tell you if it was the local copy, the cloud copy, the copy on the shared folder, the copy through that special app wherever that is... nevermind asking them about the concept of having backups.
Simple point-and-show-me information saves you an organizational migraine.
Starting with iOS 8 - 5 years ago, Apple has had extensibility points where third parties like Dropbox and OneDrive could be used to store documents and used in the common places where you would want to save and upload files across apps.
- of course the Files app was introduced two years ago.
- there have been third party document providers that exposed an ftp server on the device that you could send files from.
- apps like VLC expose a web server from your device you can use to upload files.
- with iOS 13 you can connect a standard USB device or network file share and it appears within the Files app and any app that supports saving/loading files via the standard APIs
- it’s still a sticking point that there is no easy way to add music to your library without going through the iTunes app.
I have a borrowed iPhone X and just a few days ago I wanted to download videos I took with it. I connected it to my PC and I don't see it as a drive. I had to look up the Internet to learn how to transfer my own files. The solution was to switch Windows to the horrible tile thingy UI, get to the "Photos" application, and from there, there is a button to import the files. I kinda felt treated as an infant.
Not accusing Apple here, there are countless other offenders. There just isn't a convincing argument to restrict the access through conventional channels in my opinion.
https://medium.com/codezillas/how-to-add-a-local-folder-to-y...
Think about it: You can't get files on your device in a useable format anymore. This means piracy becomes exponentially harder to consume (i.e. you can't use your iPad [easily] as a reader for your pirated ebooks). Quite honestly, that's a stroke of genius and likely a secondary motivation for this workflow.
I do this all the time. For me, it's convenient to use Dropbox as the intermediary, but there's plenty of other ways too.
Whether pdf or epub, and whichever device or type of device you use to add the file, once you do it will automatically be stored in your icloud and synced across devices. Same as any purchase.
I can later open up the iOS file manager and find the mp3 for my new favourite song in my music player's folder, click Share and send the file to my friend on Telegram or whatever.
iPhones are pretty reasonable piracy tools now.
Come on that’s not a sensible thing to expect from a non-specialist device, is it? I’m not sure this is a reasonable criticism.
I have no experience with OS X so don't know if the same applies there.
These days I use SSH a ton on my iOS devices through various apps and for many purposes (direct shell access, firing off scripts, port forwarding...) but that’s all logging into machines.
Maybe not reasonable to expect it built in, I shouldn't have the jailbreak my own device though.
Now then, don’t you feel so much better? /s
Mind, the filesystem access is badly crippled, but it does exist.
(And no, I'm not a fan of Android.)
Other approaches like using iTunes to transfer files (again, to some specific app), while clunkier and less intuitive than mass storage, still worked.
Apps like VLC could spin up a temp web server for file upload, and again if using device to pull from something instead of pushing from eg desktop to it, a lot of scripting and automation has been possible, essentially piping data and files between apps.
So at its core I think it’s more about having a very app-centric approach, including leaving it up to apps whether (and how) to work at file level. Not exactly helping but hardly preventing either.
Of course being accessible, nudging people towards sourcing their media through Apple etc are factors here, but there’s more to it than just limitations, and it hasn’t really been as heavy-handed or locked-down as many seem to think.
Edit: whoops, started writing this before the other posts mentioning the same things, then returned and finished without refreshing. Oh well.
I ran into the same thing when I acquired an iphone. After moving all of my music files to the iphone (with itunes, etc.) - look at this picture:
https://www.ilounge.com/assets/images/articles_tagging/2011/...
SO many choices ... and what is missing ? filename
This comes into play when you have uncompressed, raw rips (WAV/PCM) of CDs which have no metadata - there is nothing but the filename.
This is the most frightening thing for me... by far: the concept that I no longer have control of the destiny of my own shit. Mine. Stuff I created.
I understand the concept of how cloud storage works: "hey, we'll look after it for you, back it up and stuff, and all you need to do it pay us a small monthly fee".
Makes sense.
But add to that fact that if you stop paying they'll keep you from your stuff. Also if they don't like you for practically any reason they'll keep you from your stuff. Or if they suffer a huge data loss they'll claim that the T's and C's protect them and they'll still keep you from your stuff.
What the hell are we sleep-walking into here?
Caveat: I'm a tech. I have a local copy, a local backup (that I rotate on an ad-hoc basis with the other HD at my mum and dads house) and an offsite cloud-backup (using Sync) so I'm covered but this stuff isn't aimed at me, it's aimed at my mum and my non-tech friends who just think "hey, it's <mega-corp>. They've got thousands of people and loads of equipment and they know what they're doing!"
They sure do!
But if you just consume it... well, that's good too. Just keep in mind not all do, and curating a media collection is gardening to some.
If only there was a cloud storage provider that gave you an empty UNIX filesystem to do whatever you like with.
Maybe a proper .ssh directory to populate with your own keys and authorized_keys directives ?
Bonus points if it's a zfs filesystem with a .zfs/snapshot directory in the root.
If only ...
Most non-technical people I know cannot manage files even after decades of computing experience. I understand why many services try to abstract or simplify the concept.
For example, Google or Apple is processing the crap out of your email attachment image because they know you don’t understand the idea that trying to send five 10MB photos likely will not fit in an email. All you want to do is send your photos to Grandma. So instead your iDevice handles it for you and either compresses them or throws them in a temporary cloud storage location with a link. Grandma gets her photos - for most people the alternative is a confusing mail delivery bounce followed by frustration.
Apple added Stacks to the desktop because everyone’s tired of seeing other people’s computers with a mess of icons smashed together on the desktop, because that’s the only place files could be conceivably stored in the mind of a non-technical user. The whole idea of a file hierarchy seems either too confusing or too labor intensive to that type of user.
Nerds like us always have and had options. We are aware and capable of coming up with a more friendly solution to us, like a home built NAS or Synology box; ask a non-technical user what those are and be met with a blank stare.
For most people, they really should be paying someone else to maintain their data, iCloud, Google or whatever. Nobody’s home back up scheme can match what these companies are doing in their data centers. And no, these services don’t just delete your data immediately after your monthly payment method bounces - you are given time and nags to download your data or renew, often your data just becomes download-only.
Are we annoyed that most options are not for us, or have to be in a form of a workaround to a technology stack that’s marketed to non-technical people? Maybe. It would be wonderful if there was a device as nice as an iPad that also gave us full access like a Linux PC.
But also, nobody’s forcing us in 2019 to stop using our files as we did in 1999. We didn’t have all these services back then. File management was the option. It is still an option.
And in 2019, Apple finally adds mass storage support to the iPad. Sometimes things come around.
Instead, we've been struggling (and often failing) to dumb computers down. The fact that people still struggle with the same tasks is a testament to that.
I have lost track of the number of people who have lost precious pictures or documents because they don't understand what device they are stored on, and threw away their old iPad. By focusing on the most common case "emailing photos to grandma" we end up introducing all sorts of new problems.
Have we really improved things? I find these new file abstractions we have created to be much more confusing.
Old people understand the concept of physical media just fine. They know what a record or a roll of film is. And because of this phobia about educating users, we hide so much that they don't even know if their data has been synced, or if it is private.
We haven't, and here's why: a file system was one abstraction. A simple one, that made sense, and wasn't very far detached from actual implementation details. Which meant that weird black magic almost never leaked from underneath.
What we have now, with the cloud and dumbed down software, is many mutually inconsistent and sometimes incompatible abstractions, all completely detached from the reality behind them, which means they have inconsistent corner cases that make no sense unless you know enough about software to be able to imagine what those abstractions are backed by.
In the best tradition of commerce, we've helped people avoid having to learn a simple concept by introducing them to lots of "simplifying" lies.
You have met my mother I see. She still uses her multi-function printer like a copier because she's never quite grasped the idea that she can store her document scans on her computer.
What amuses me about all of this is that it seems to just further cement software developers as magicians. Back in the day it was the Windows control panel.
Nowadays? I have direct experience with tons of people navigating cloud service settings or whatever else via endless GUIs. Just give me the filesystem. It's all a bunch of JSON configs or whatever anyway. find -name blah -execdir sed -i blah. Bosh.
Well, I'll never struggle for work at least. Perhaps this is our 'guild'...?
I actually end up helping tons of random people in my company use conda, because they can't get the concept of virtual envs. They don't understand that the libraries are actually files on the disk, and your virtual environment is just a directory and some variables.
That's not to say all developers are like this, but I think the numbers that think like this are growing. Ultimately, I'm like you - they can pry the files from my cold dead hands, if I didn't know what/where/how files were used, I wouldn't fundamentally understand how a lot of different technologies even work.
Potentially because that means in 5-20 years they'll have a much bigger pool of labor available? They've already done illegal market-manipulating collusion which resulted in billions of dollars of salaries not being paid: https://news.ycombinator.com/item?id=19666545 ; in the end they got hit with fines for a few million bucks and some developers got a paycheck for a few thousand bucks (two orders of magnitude less than the wages stolen from them).
I'm just a college student, but I find it hard to not think that when I see the big companies spending tons of money to push people to learn to code.
The Steve Jobs market manipulation was def a dick move, but.. well, its Steve Jobs. There will be other such efforts too, but that alone doesn't indicate the kind of concerted market manipulation that you think exists.
The emphasis on coding is similar to the emphasis on schooling and mathematics at the dawn of the industrial revolution: societies realized then that basic mathematical literacy can be immensely beneficial to people. And we understand today that basic programming skills can have a similar transformative effect.
Jobs in tech are increasing at a tremendous rate. As we shift to more automation, the demand for computer literacy will only increase.
It wasn't "Steve Jobs", it was most/all of the tech giants colluding.
[0] https://docs.microsoft.com/en-us/windows/deployment/update/w...
most iPhone consumers I know of are constantly complaining about their phone running out of memory from photos and whatnot, and they don't understand how to transfer between devices.
If only all of these OS's just had one single, shared, reliable filesystem they could all mount and/or mirror, without .. oh, I get it .. I just install another 3rd-party app ..
C'mon, OS vendors. Pop the cloud already.
--
Sorry, you can't view or download this file at this time.
Too many users have viewed or downloaded this file recently. Please try accessing the file again later. If the file you are trying to access is particularly large or is shared with many people, it may take up to 24 hours to be able to view or download the file. If you still can't access a file after 24 hours, contact your domain administrator.
--
Note that this was my own file. I was the creator and I was the only downloader.
So, I guess I really didn't own it after all. That's the position that I'm going to have to recommend to anyone using any "cloud" service. If it's in the cloud, you don't own it. If you really care about it, back it up, and test your restores. Your cloud storage can disappear on a whim.
FTP provides exactly this experience. Every platform can interface with it and it can be made network-transparent.
A local file system? UDF does the job.
Some people run the opposite: they instruct apps and the app had better use the data given in the format given or the user moves to another app (or writes their own).
Someday I hope the second demographic's lifestyle is taught in highschools, because it is socially healthier.
That's a bold statement, can you elaborate?
People want to go for walks, meet up with friends, share memories, they don't care how it's done. Having the tools get out of the way and just do those things is a net benefit to society.
I'm strongly in the second group. I don't object to an app using its specific file format if it's really needed. But I want to have the data in files, which I can independently move around without invoking the app, and possibly view and alter in software other than said app.
We’re not given much time to consider anything but what’s good for the economy.
Pragmatic acquiescence trickles down.
A few days ago I opened a Jupyter notebook I had authored in 2015 - and it failed to reproduce my results. I relied on pandas, and pandas has gone through API changes. Fortunately, it wasn't hard for me to search around for the newer versions of those APIs and fix things - but this is only 4 years after I authored it. Can I have any confidence that 20 years from now I could reproduce the results - or even find documentation for the APIs I used to understand exactly what I was doing?
I mean sure - I could put the OS and all the libraries needed to reproduce the work in a container and be guaranteed[1] it'll run later, but the only people who will do that are those who need to for regulatory compliance[2].
[1] Not really. The software to run the container may no longer be supported, and the people with the knowledge may charge more than most can afford to get it to work.
[2] Something we're dealing with at work these days. Writing stuff for the automotive industry and they expect that any software + tools should work 15-20 years from now. Same level of expectation for availability as any other car part. Now that there is so much SW in modern cars, this is a good opportunity for being a consultant 15-20 years from now!
So no more books in your library, just access to books through a device under the control of some large - usually American - entity. No more music collections, just access to someone else's music collection. Ditto for movies, you don't have your own copies, there is just access on a pay-per-view basis.
And of course only the 'proper' artists, movie production houses and writers will be available.
I refuse to play this game. I have my own collection of music, books and movies. Yes, it takes up space. But that's fine with me, at least no billionaire will be able to dictate what I can read, view or listen to.
Extending this to your personal files was an obvious move.
This is exactly the story I went through a few years back as well. I had an iPhone 4? 9-ish years ago when I was in a music-kleptomania frenzy (back when what.cd was around too ;) ), and I remember just how frustrating it was dragging and dropping shit through iTunes all the time. Same for movies, photos, and documents! I couldn't see the appeal of using the iPhone at all - why would somebody use this when they could just plug an Android device into their PC and use their file explorer? (The update to iOS7 was the nail in the coffin - I found the neon design just repulsive.)
I guess my question is - has the Apple "ecosystem" gotten better for this? Can I actually edit metadata for .mp3s and .mp4s and just drop them into an iPhone or iPad now? And copy them out as well or download them offline, bit for bit? If so, are dongles still the norm?
*An aside - I've definitely eased up on the file-ownership thing now. I just let YouTube Premium handle everything. It was definitely a waste of time retagging thousands of music files out of some strange ownership OCD. I see it as a net positive overall - now I just find it amazing that someone even uploaded some white-label vinyl-only run-of-100 Foul Play UK garage record (and similar records going for >$100 on discogs - maybe even ownership has its limits?) at 128kbps at all, even if it doesn't stay there forever :)
Full disclosure: I used to work for Spotify. When people talk about services like Spotify and Apple Music, they usually miss how much value is added in the cataloging and search they do. These are a massive pain to do yourself, it doesn't scale past ~10,000 tracks, they've put more thought into it, and are better at handling the song/track/recording ambiguity in search results.
Spotify also says supporting more than 10000 tracks in "My Library" isn't necessary, so the people who hit that limit aren't helped by Spotify either
And it's not been for lack of trying.
While I share many of the author's misgivings, files are actually making a bit of a comeback. After many years of trying to void the very idea of a file, Apple was essentially forced to introduce the Files app for iOS and introduce mass-storage support for (some) iPads. This is a good sign.
The anti-file movement has also been very strong and vocal in the programming world, where many claim that many if not all of our programming problems would go away if only we put our code in databases, or (Smalltalk) images instead. Yet the files, they persist.
That doesn't mean that files, or hierarchical filesystems, are perfect. However, whatever the problems ("Unititled 54.txt", anyone?), the proposed cures for what ails files invariably appear to be worse than the symptoms. And almost inevitably, we have to bring back or reinvent the very same concepts that we gleefully got rid of. That's a resilient concept! One that is apparently useful and pertinent at a somewhat deeper level than we fully understand.
So how about we stop trying to "get rid of files", but instead see if we cannot improve the concept and the mechanism(s) to fix the problems.
I am a tech person. I do organise my files in folders. And I don't want any other organizing principle for my documents. But I realize that this concept does not work for most other people.
https://en.wikipedia.org/wiki/WinFS
The idea was that the OS used a relational DB-like file system to store things and recognise them for what they are and deal with them differently instead of everything being streams of bytes, e.g. pictures aren't the same as Word documents.
Didn't materialise but I remember at the time being dead excited by the concept.
Files get lost in the clutter, get overwritten or corrupted, don't get saved when you crash, are on this computer and not that one, don't make it across when you transition to a newer box, have to be mailed around or put on (gasp) physical media and passed around like a crude lump of plastic. And then your media gets a virus, oops.
It was an improvement to autosave. It was an improvement to version. It was an improvement to distribute across redundant backups. It was an improvement, to be able to get at it from anywhere.
There's UI issues in the new way of doing things, but that seems likely to shake out with experience. Files were often clunky at first too.
Files rot, unless they are intentionally low rot by design. Most weren't.
E.g. the insides of git are much better (although the abstractions layered on top are fucking gruesome.)
Something like IPFS is an obvious way forward. https://ipfs.io/
I'd love it if the WWW was e.g. Xanadu+Prolog.
People completely ignore the fact you would have no access to this data if you have no online connection (or access to the account). It's not that files are going away, it's that they're being increasingly managed by tech companies, and normal people don't know any better, as evident by the article's title... it shows the author doesn't what files are for and that they are invisibly managed by a third party.
"It's better to acquire a new customer at any cost, versus retaining an existing one."
There are plenty of verticals where if you develop a piece of software that's blunt, direct, and full of power-user features, it won't take off because it gets panned as "too hard to use."
Tricks like abstracting where and how the data is stored tend to be appealing to non-skilled users. They see "look how easy, I don't have to figure out which files are in the cloud and which are on disc, and can't accidentally try to open a spreadsheet in the photo editor." New users buy into it, the developers get their hockey-stick growth chart and everyone declares victory. After a few months, a small percentage of users end up dropping the product with a migraine because they can't break out of the jails and abstractions that are clearly standing in the way of improved productivity.
No real point in satisfying those users anyway-- they've already paid, so there's no further value they can provide to your ecosystem.
Aren't all things in computers eventually a file (descriptor)?
And with the rate of data volume increasing, things going the opposite of the headline?
And with the cloud, they are being replicated multiple times there and locally?
I wrote a piece offering a counterpoint to this view five years ago (in response to a similar article by Fred Wilson) and I think most of the points still hold true today:
https://blog.zamzar.com/2015/01/02/why-fred-wilson-is-wrong-...
I save everything to OneDrive anyway, but I'm a not-so-old fogey that is more comfortable using the filesystem myself, rather than crummy layers stuck on top of it.
So, the digital equivalent of https://en.m.wikipedia.org/wiki/Onomatopoeia
Somewhat related, I have made a deal with the devil and use Comcast and, until yesterday, I was using their combo modem/router. Comcast doesn’t let you change the router’s DNS settings (which are, of course, set to Comcast servers) but I was under the impression that the DNS settings closest to the client always prevailed. But fiddling around with WireShark yesterday I could clearly see my DNS requests going to 75.75.75.75 (Comcast) even though my laptop was set to use CloudFlare. Is that even possible? Could the router have been set by Comcast to somehow override my settings or to intercept & reroute DNS requests?
This is what happens when Capitalism meets Software. When it comes to winning markets, shipping simple stuff is always better than shipping a slightly more complex (in terms of code as well as UX) but correct stuff. That is why most companies prioritize feature development over bugfixes. That is why selecting multiple items from a list is still inconsistent across the web. That is why XMPP lost and Whatsapp and Messenger are winning.
There are efforts like Solid (https://solid.inrupt.com) by Tim Berners-Lee himself that's trying to bring back the concept of files and data-ownership to end-users, to enforce correctness and consistency. But I'm still skeptical if the efforts will ever win the market.
https://old.reddit.com/r/dredmorbius/search?q=information+th...
Same with the 'cloud'. What's the difference between running your own FTP server and a cloud server? Technically not that much IMAO, only the latter means the data is controlled by some corporation running that server. And why is there hardly any progress in the area of building a private turn-key 'cloud' server so you don't need gmail, dropbox, github, etc..? I'm sure it can be made, easy to setup, robust, with backup, you name it, just like a Mac pc that's made for non technical people. We know how to build it, but we don't do it, we build for corporations instead.
The further we go down this road the less control we will have, and the ramifications of that don't look pretty to me to say the least.
The war on general computing implies disempowering users, rather than empowering them. But that it's "not surprising" doesn't mean it's good, or that it can't be reversed.
I wonder what associations for craftsmanship in software, respect for users, and similar things there are? If there was a web ring about that, I would browse it.
Apple music is available in another 59 countries that Spotify is not.