Building Your Own Dropbox Equivalent | Hacker News Reader

Building Your Own Dropbox Equivalent

cloudfs.org

99 pointsjnoller23 comments

These things are a worthy effort (there have been several such posts recently) but there is no way it's equivalent to Dropbox. Dropbox is cross-platform and has web-based file access and many other benefits. Is Dropbox-like file storage destined to be one of those things (like anti-virus, for example) where there are some fairly good open-source implementations which are never quite as good as the commercial product(s)?

Calm down. Sheesh. He's only giving a tutorial on how to develop a very basic Dropbox. I thought any developer would know that.

A more useful article would be a how to on setting up secure FTP.

Edit: Looks like I was wrong.

No, the gp has a good point. These articles seem to come up every week on HN. The point is that it's super easy to write something that syncs a folder. You can start with rsync and a cronjob, and just go up from there. Dropbox is fantastic because they haven't solved the easy problem (sync a folder on a single platform when nothing goes wrong), they've solved a very different hard problem -- sync a folder across multiple platforms, deal with corner cases, handle errors gracefully, and wrap the whole thing up in a very easy-to-use GUI.

These things aren't dropbox replacements, they are reminders that dropbox is a unique product.

This is the difference between equality and equivalence. It's not cross-platform to the extent that Dropbox is, though the fundamentals are portable to any platform that supports FUSE and thus GlusterFS. There's also no web UI, though there are already plenty of web UIs that can be slapped on top of any POSIX directory on a remote server. If you're the sort of Dropbox user who primarily stores and retrieves individual files through the web UI, this is not equivalent. For what I believe to be the more common use case of using Dropbox as a mountable location to/from which to sync files, though, it's quite equivalent. The fact that it's open source, deployable under the user's own control, and actually secure even makes it a better fit for many current Dropbox users' needs.

As for "never quite as good" that's just anti-open-source crap. It only works if you define a category by reference to a specific commercial alternative, and use that product's feature set as your checklist. That's no more valid than starting with the open-source alternative's feature set and using that checklist to find the commercial product inferior. It's marketing, not real life. If open source did exactly copy commercial products, then (a) technology wouldn't advance as much, (b) people would bitch about the lack of originality, and (c) the people who make the commercial products would sue. That's a totally unreasonable expectation.

"Dropbox as a mountable location to/from which to sync files"

Dropbox isn't "mountable". The files are stored locally and synced. This is MUCH different as you can work offline. Additionally, you get the performance characteristics of local access.

So this is equivalent in the same way say a flash drive, or Filezilla would be equivalent. It doesn't miss edge features of Dropbox, it misses most core good features.

Being non-mountable isn't a core good feature. It's a serious deficiency. If you want to store locally and sync so you can work offline, this is only one of about a hundred solutions. It just happens to be one that also allows you to use the file directly, that keeps your data secure and private (unlike Dropbox), etc.

Unfortunately, an offline syncable folder is the core feature of Dropbox, and anything that doesn't have it isn't, by itself, a Dropbox equivalent.

Of the hundred solutions you mentioned, could you recommend a few that will automatically sync files in the background as they are changed with versioning/conflict resolution that don't choke on large files? I haven't found any which perform the task satisfactorily.

Edit: Tone is not intended to be antagonistic; I'd genuinely love to know what you use for this. :-)

It sounds like you've had some bad experiences and I don't know which projects address those specific issues, but personally I've had good luck with Unison. Somebody else pointed me toward lsyncd which seems to take an interesting (inotify-based) approach which might meet your "in the background" requirement better, but I don't have any experience with it. I know there are lots of others, with varying levels of maturity and "intelligence" regarding things like conflicts and moves/renames, but it's such a crowded field that I don't claim to be an expert.

My attitude is generally is that if you start with something that can be used live then it's trivial to use it for syncing instead, whereas something that's originally designed for syncing might never go the other way. Right now I just rsync to the remote directory, then from it, whenever I feel like. I could automate the process, or replace rsync with something more sophisticated, but my focus is mostly on the remote-infrastructure side because that's where I see a dearth of suitable alternatives.

I'm inclined to agree. Dropbox has made some hard decisions to be so simple and easy to use, decisions probably made after a lot of usability testing. (Smaller) open source projects usually don't have this luxury.

Agreed. It's like saying all of DropBox's competitors/progenitors were "like" DropBox. If they were truly like DropBox, they would have been DropBox. "If you were the creator of Facebook you would have created Facebook." Even just an inch distance short can be as bad as a mile, in the consumer space.

This doesn't seem to be anything like dropbox, but more an NFS replacement.

NFS doesn't mirror the remote data to the local disk, whereas glusterfs allows exactly that (plus some more stuff). http://en.wikipedia.org/wiki/GlusterFS

So, is it usable even when offline? How does it handles conflicts once it comes back online? The only mention I seem to find about mirroring is mirroring across multiple servers, but not client<->server (unless you run a server locally on the client too).

NFS and Git are exactly why i don't need dropbox. agglomerating large chunks of free space on all devices into universally-accessible paths without wasting any bytes on duplication or synch daemons, and a proper VCS for stuff that i want surgical branching/merging/history and redundancy

One thing I like about products that work... They work, and I don't have to. Which is why I like dropbox.

Does anyone know of a similar setup that allows client-side encryption as well, with some nodes (those in the cloud) acting only as dumb storage that never sees the encryption keys?

The only thing I'm aware of is drbd, which acts at the block level, so you can stack dm-crypt and any file system (including OCFS2) on top. As far as I know, drbd only supports 2 nodes, though, which kind of limits the appeal.

(I'm specifically not talking about tunneling a normal cluster file system via SSL, which only provides end-to-end encryption)

Try Wuala, all data is fully encrypted and acts as "dumb storage"

I already use that for some stuff, it gets rather expensive for vast amounts of data, and I've already got spare HDD space on servers.

These articles that describe how to "build your own dropbox" all ignore the client. For me, the beauty of dropbox is the simple, straightforward client. Backing up on my own hardware/cloud might be interesting, but not if I have to monkey around with shell scripts in a CLI.

When someone reverse-engineers the dropbox protocol so I can use the official dropbox client on my own infrastructure THAT will be news.

As I've understood it, it's already (partially) reverse-engineered by dropship. https://github.com/driverdan/dropship/blob/master/dropbox/me...

Still, I see no reason to use Droxbox servers when you could setup your own.

Thanks a lot. It's nice to see there are options, although they require a bit extra work, they're worth if you consider your privacy valuable.

For those interested in building cloud apps we at SpiderOak offer a 'Do it yourself' Storage API that is designed specifically towards backup, sync etc.

You can find more info @ https://spideroak.com/blog/20100928101634-spideroak-diy-a-sp...

Reply on news.ycombinator.com