Hacker News Reader

Comment by kentf | Hacker News Reader

interesting perspective. I agree and disagree. I think Google could over-engineer it, but UI aside, there needs to be some "intense" engineering to scale this reliably across the globe. I don't doubt that Dropbox could do it, but I feel Google is poised for something like this. To their credit, they have gotten a few things right ;)

View on HN

Isn't Dropbox built on top of Amazon datacenters? If that can't scale, I don't know what can.

Yes - you could regard Dropbox as simply a different pricing model for S3

Amazon, you pay for what you use, Dropbox you pay (more than S3) for 50Gb, you use less than that Dropbox make money from you. They also aggressively hash data chucks to avoid storing the same data from different people twice - while charging both of you.

None of this is bad - they have a great product - but it's how they do it.

Dropbox is so much more than a different pricing model though. i just think of the hundreds of insanely non-technical people i know who say to me "wow, dropbox changed my life". how many non-technical people say that about s3?

Uhm. S3 enabled Dropbox. By definition, it changed the world more than Dropbox.

I don't think so. Bear with me as the analogy is not all the way there, but small nuclear 'pebble' reactors already exist. Not that much people care... BUT the company that makes them available to the masses cheap and safely will change the world.

This is trivially wrong unless somehow you think Dropbox is simply a thin UI layer over S3. Which, if you really do think that, is the type of dismissive attitude that armchair Web 2.0 pundits like to spout off without any clue what goes into building a good user experience.

That makes sense. We should rewrite the history books to reframe everything as the achievements of carbon.

> you could regard Dropbox as simply a different pricing model for S3

I think a better analogy is that you pay more for a simpler and friendlier interface to S3...

Dropbox is one of many "applications built on top of S3/EC2". That's not an analogy, because you really don't need one. S3 is for developers, not users. Users don't care where Dropbox stores its files.

Amen

I wasn't aware of the agressive hashing - interesting. I wonder if they're actually operating at a scale where there'd be a real risk of a hash collision causing data loss for some very confused poor sod.

The Venti archival storage system paper might be an interesting read for you[1]. The researchers deemed Sha1 (160-bit) acceptable to hash 8Kb data blocks for an exabyte of data (10^18 bytes). The risk of collision at that level is less than 10^-20. There are now 512-bit sha hashes, so that takes it to another level.

So, I think Dropbox is safe from hash collision risks at this point.

[1] http://doc.cat-v.org/plan_9/4th_edition/papers/venti/

I'm pretty sure they added some secondary identity checking mechanism. After hash match, you can compare the first x kbytes of the documents for example.

>They also aggressively hash data chucks to avoid storing the same data from different people twice

If one user uploads a large movie and another goes to upload the same large movie, is the upload nearly instant then as a result? Do they sha1 the file, decide it is the same, and then just make an association that this file is now used by two users?

Yes - at least when I upload identical large data chunks on different machines the second one finishes instantly. It also means moves and renames of files in the dropbox folder are instant

Reply on news.ycombinator.com