Hacker News Reader

Comment by ericfrederich | Hacker News Reader

You need a problem to have a solution to it. What do you consider to be the problem here?

This is essentially something that can be expressed in relatively few bytes that expands to something much larger.

Imagine I had a compressed file format for blank files "0x00" the whole way. It is implemented by writing in ascii the size of the uncompressed file.

So the contents of a file called terrabyte.blank is just ascii "1000000000000" ... or the contents of a file called petabyte.blank is "10000000000000"

I cannot decompress these files... what is the solution?

View on HN

>You need a problem to have a solution to it. What do you consider to be the problem here? > >This is essentially something that can be expressed in relatively few bytes that expands to something much larger.

That seems to be the problem. I mean, if an object expands to something much larger to the point that it crashes services just by the sheer volume of the resources it takes... That is pretty much the definition of an attack vector of a denial-of-service attack.

There is a problem here, but it's not with data. It's with the service.

Being able to express trees efficiently in a data format is an useful feature, but it requires the code processing it not to be lazy and assume people will never create pathological tree structures.

I'm not following; why can't you decompress it? Of course you cant decompress it into memory, but if it's trying to that then there's a problem in the code (problem identified).

Naive solution, just write to the end of the file and make sure you have enough disk. More sophisticated solution, shard the file across multiple disks.

That's not a solution, that's sweeping the problem under the rug: "just have the OS provide storage, therefore it's not my problem any more, solved. (Never mind that with a few more layers, the tree would decompress into a structure larger than all the storage ever available to mankind)"

Reply on news.ycombinator.com