An open source Linux client for Google Drive | Hacker News Reader

An open source Linux client for Google Drive

github.com

110 pointsg3orge49 comments

Open articleView on HN

A quick search for "google drive fuse" reveals a better approach to this...

https://github.com/astrada/google-drive-ocamlfuse/

https://github.com/jcline/fuse-google-drive

Didn't get much attention when it was posted here 267 days ago:

http://news.ycombinator.com/item?id=3887308

What makes you think the FUSE approach is better?

FUSE is the way to make a source look like a file system. It's the right architecture for this.

"Umm.
'userspace filesystem'?

The problem is right there. Always has been. People who think that userspace filesystems are realistic for anything but toys are just misguided." --LT[1]

[1] http://www.spinics.net/lists/linux-fsdevel/msg46078.html

Nice appeal to authority you've got there. I'd hate to see anything happen to it . . . like realizing that when it comes to storage Linus is no expert. He hasn't worked on non-trivial storage systems, and some of the world's largest systems make good use of user-space filesystems like PVFS. Is Intrepid a toy? Nope. He's simply wrong, as he usually is about storage. On to the facts.

The fact is that without being in the I/O path you can't detect and propagate updates as they happen - even with inotify, which doesn't scale well to very large filesystems anyway. There will always be some lag, and some reordering. Whether those matter to you depends a lot on your specific needs, but I for one don't like to rely on storage software that don't even try to guarantee correct behavior and might fail me at the most inopportune moment.

FUSE is the way to get in the I/O path without having to become a kernel developer (which I am BTW, since before Linus, since you happen to like appeals to authority). A FUSE-based result will give better functional guarantees and better performance than a hacked rsync. Whether it might have some disadvantages compared to kernel code isn't even the right question. Compared to what this thread is about, FUSE actually is a better answer.

I thought that the canonical way to use pvfs was with the kernel module. Isn't the pvfs2fuse interface an afterthought?

The kernel module is essentially an interceptor, much like FUSE; the serious logic is still all in user space. When I talked to the developers about four years ago, they had considered using FUSE on several occasions but it had never performed as well - for them - as what they had already developed themselves.

When Linus says that user-space filesystems are just toys, he's simply wrong. If he were to say the same thing about FUSE in particular he'd be closer to mark, but a large part of the reason FUSE has lagged is Linus. You don't get credit for predicting an outcome you control, in my book. It's worse than it needs to be because Al Viro and others actively resist any changes that would make it better. Linus, who delegated this part of the kernel to others nearly twenty years ago, backs them up. Making the VFS layer look more exactly like what XFS or even NFS needs is par for the course. Making the smallest concession to what FUSE needs is verboten. I guess some folks are afraid of losing their niche if those artificial obstacles were removed so that more people could build filesystems without begging for keys from the gatekeepers.

Ah, git is not a non-trivial storage system?

Git is not a storage system at all. I was talking about actual physical storage systems that drive requirements for storage software, and there don't seem to be many petabyte-level git repositories. Git doesn't even qualify as storage software, as it delegates all of the real storage tasks to filesystems and such. It's an application that uses storage, much like every other application from emacs to Excel.

Also, have you noticed that git is in user space? Must be a toy, then. Just ask Linus.

Wow -- did you read even a single paragraph beyond that line? Immediately after that he goes on to say:

> fuse works fine if the thing being exported is some random low-use interface to a fundamentally slow device. But for something like your root filesystem? Nope. Not going to happen.

"Random low-use interface to a fundamentally slow device" (like say a remote server in some faraway "cloud") sounds exactly like the situation at hand. Compared to a "real" filesystem (like something you'd mount on /), GDrive is very much a toy, and seems to me like an ideal candidate for a FUSE filesystem.

Is Dropbox a toy too? Or is gdrive just a toy because you have decided that it will be a low use storage device?

> Is Dropbox a toy too?

In this context, yes, for exactly the same reasons as GDrive. No one with any sense is going to be running a relational database or MapReduce out of either one.

Note that when I say "toy" in this context it's not really anything pejorative -- I use such "toy" filesystems all the time (sshfs most often, but also encfs and s3ql), and they're awesome and highly useful. But they're the kind of thing where, say, lock contention (or hell, even being single-threaded) isn't much of a big deal because they're not performance-critical.

My first thought on seeing Grive was "Oh, it's not FUSE"

FUSE is a natural choice if you want fully integrated and transparent access to so-called "cloud storage" services. Performance overhead isn't so much of an issue, you'll probably manage to max out most home broadband services even with the several layers of syscalls.

You'll have less to remember, throw a line in fstab (or mount it in your shell's rc) and it's all just there, no sync commands to remember, just grab your favourite application and start hammering away at your files.

Canonical's first cloud filesystem for Ubuntu One used Fuse, then it was later rewritten away from it, and became much more reliable.

Correlation, or causation? It's complicated.

But part of the attraction for Drive, Dropbox, and Ubuntu One is that the data is all on your local disk, so should still be accessible even if you're offline or there is a problem with the service.

Making access to the local copy of the files depend on the client program being up is inherently more fragile.

Now if you want to _not_ have them local, but rather fetch them lazily on demand, Fuse would be a good choice. It does open up a fair bit more complexity to do with scheduling and abandoning requests when programs assume the filesystem is as fast and reliable as a local one, similar to NFS.

I think a read-through client could be good for v2.0, but getting local sync to a real filesystem working well would be better for 1.0.

Good point; I'm compelled. Dropbox style local syncers vs remote file systems. I see.

I've had a Ubuntu laptop in development with root fs top of fuse (ntfs-3g) for 2+ years without a hitch. It's easily fast enough too.

This. Oh god no. No. Don't put your root fs on fuse. Or put ntfs on root. Please, for your own good?

Or am I being trolled? I'm probably being trolled. I'm being trolled, right guys? Right?

That's what I thought when I discovered how it worked. The "WUBI" install type is (was?) a supported install type in Ubuntu. But it's been working fine and I can't think any fundamental reason why it would be unsound.

I should say there have been non-fuse related hitches with the integration with Ubuntu (grub problems).

Fuse backed by a local filesystem is a different proposition to Fuse backed by a network service. There's nothing about running filesystem code through Fuse that makes it inherently more risky. However, accessing a filesystem over the network does make it unavoidably more likely it will be slow or unreachable, and not just in straightforward ways.

I've actually been using Insynq (https://www.insynchq.com/) with some pretty good success. I especially love how it handles having multiple drive accounts; the different Apps accounts can get kinda hairy.

I've been using grive for a few months on a few different Linux boxes, including some running on ARM/Linux.

It works really well in my experience. Not quite as slick as the install-and-forget-it nature of Google Drive for Win/Mac, or Dropbox for Win/Mac/Linux, but it does what it says on the tin and I like that because it is fully FOSS I can easily take it with me on ARM/Linux.

If you want slick, install and forget in an open source package try out git-annex in assistant mode.

http://git-annex.branchable.com/

I tried. I really, really tried, but I could not get it to install on Arch, with either AUR or cabal. Dependencies were broken, wouldn't even compile.

The only thing better than joeyh's software (git-annex, moreutils, etckeeper, etc) is the way he handles bugreports. Post a description of your troubles to the branchable wiki.

I'm guessing that you only use it for occasional syncing then? Or do you use something like incron to look for changes then run grive, giving it more of an automatic sync?

I haven't bothered to set up a cron job, partly because I tend to reimage the ARM devices I use grive on quite often, but yeah there's nothing I know of stopping someone from setting grive up to run automatically.

Still not quite as nice as the other clients which subscribe to file change notifications, but more seamless than running it manually for sure.

Ah ok. I looked at it and noticed it didn't sync when changes were done. Incron should work in theory, but I haven't tried it. The lack of a headless gdrive client for linux & the fact that I still use windows on a few systems is why I still use dropbox. Was just curious how well it worked in practice

Why hasn't Google made a client for Linux yet? It's been almost a year.

This is indeed quite weak of them. Perhaps their developers don't really believe in the product?

> Perhaps their developers don't really believe in the product?

Or that the biz guys don't think that Linux leads to monetization of the product?

If the developers on the team were passionate enough and the corporate culture was as good as I hope it is inside Google; there would be a Linux client by now regardless of its monetization prospects.

They use Linux in Google... I don't know how they can't think it's viable.

I bet there was a internal screw up, they said they were working on it pretty early. Maybe there was something with the team. Any insiders here?

Doesn't ChromeOS have integration with Drive? If so, the bulk of the code for a Linux client should already have been written. I imagine this is more of a product management issue, rather than one of engineering.

I guess that's just the Google Drive web interface - ChomeOS is designed to use webapps exclusively. Linux users can also access Google Drive on the web.

Probably because they don't care about Drive. It's basically a refreshed web interface with a glued on API that they use to promote the "store your stuff in google docs" feature.

Surprising indeed since Google's computers run on a flavored Ubuntu distribution. It probably means they are not even using it internally :)

I can't really see people using Drive on their work desktops. As for laptops, it seems most people at Google choose mac.

Why is it that you do not see people using Drive on desktops? It can be useful in many ways.

I guess I see my work desktop as exclusively for work-related activities (plus Facebook and HN). Whereas a work-issued laptop could be a mix of both. I noticed a similar trend in those around me. Don't really see people organizing their photo library on their workstations...

But Google Drive can be used to do real work by sharing work documents across many email accounts, etc... It's a good collaboration tool for certain things.

Have you ever seen a googler work? They do nearly everything on their webapps, and the programmers live in source control.

There was an article from a while back as to why Google wasn't working on gdrive faster and one of their engineering managers was quoted as saying something to the effect of "why are we syncing files? Who needs files, we have everything on the web". But unfortunately I can't find that reference right now.

But it doesn't seem too far a stretch to think that some web-heavy companies prefer web solutions with as "stateless" clients as possible.

I hoped this was using the new API (https://developers.google.com/drive/) and that it can be forked to Duplicity in favor of the existing google docs api, however Gdrive is using google docs as well: https://developers.google.com/google-apps/documents-list/

Still, an interesting implementation and an alternative for python-gdata.

We have created a project to access GDrive, Dropbox, SkyDrive and Box from web browser. We have some plans to open up the API in the future. If you guys think that's useful.

https://www.addfs.com/

If you are looking for a quick way to push files to your drive, I made this: https://github.com/pricees/google-drive-companion

This is really cool. I started writing something similar last year to get to know Python and when Google released their official client I stopped.

Will this work on OS X? The mac client is horrible. My internet connection is rendered useless while files are uploading.

Thanks for posting. This kind of stuff is why I read hacker news!

Any advantages of this over Insync?

Reply on news.ycombinator.com