Hacker News Reader

Comment by jimmybot | Hacker News Reader

From the referenced Google Code Blog http://googlecode.blogspot.com/2009/11/use-compression-to-ma...:

Anti-virus software may try to minimize CPU operations by intercepting and altering requests so that web servers send back uncompressed content. But if the CPU is not the bottleneck, the software is not doing users any favors. Some popular antivirus programs interfere with compression. Users can check if their anti-virus software is interfering with compression by visiting the browser compression test page at Browserscope.org.

Serious? Anti-virus software that is doing that is acting almost like malware doing what the user didn't ask it to do. Does compression have any even minor security implications that would legitimize this? And anyone know which anti-virus does this?

View on HN

Does compression have any even minor security implications that would legitimize this?

The quote you gave said that anti-virus apps might do it to "minimize(sic) CPU operations" rather than for security reasons.

(sic)? Not where the post was written it's not :P

But it is not written thus where I wrote my comment, ergo I raise you a smiley

;0P>

Well anti virus software usually checks for signatures of known malware in the incoming data. I imagine compression would slow this down. They would have to uncompress the data and then check for signatures.

(You can say that they can just compress the signatures, but that probably wont work for the more complex compression schemes, because they are not that predictable. Or maybe it can work, but it was considered too hard to code by the AV people.)

Compression schemes have to be predictable when run on the same data. Otherwise decompression wouldn't work. If signature detection actually is more difficult on compressed data, it's not because the compression is unpredictable.

Compression schemes have to decode predictably, but they don't necessarily generate the same compressed data. gzip -9 will usually use less space than gzip -1 and decompress to the same data.

What's probably happening is virus scanners are taking the easy way out rather than implementing decompression algorithms in their scanning engines (it probably looks better on traditional benchmarks too)

Even if different data are being generated, the compression is still predictable. There must be something in the gzip header indicating the level of compression (to ensure that the correct decompression algorithm is used). Point being, there's nothing stopping the AV program from figuring out which compressed signature is the correct one. It can even compress the raw signature on the fly.

On your second point I agree. I'll refrain from ranting about AV programs here, but suffice to say there hasn't been enough innovation in the field because AV companies are able to sell substandard products and still make good money.

To be more specific, compression schemes decode predictably, but any given portion of the compression stream need not be independently predictable. Gzip (and DEFLATE in general) use adaptive compression approaches - the data determines the encoding. If you know your content of interest is in the beginning of the stream, pre-computing different compressions should work out. Otherwise...

You've got it backwards: it is the decompression that is predictable; compressors are free to transform the input in any way such that a compliant decompressor will extract the original file.

The gzip format documentation is available here btw: http://www.gzip.org/zlib/rfc-gzip.html

Yes they are ultimately predictable, but they are not easily predictable. That is one usually has to go through all the steps of decompression to predict them there are no shortcuts.

Imagine a data stream X that is compressed into another data stream Y. Imagine that a small portion of X is data stream x1 which is the portion of data used for a signature. That will get compressed into y1. Now lets define x2 as all the data in X that is not x1. Now if you are always guaranteed that the same x1 would get compressed into the same y1, then things would be easily predictable and you can just compare compressed signatures. But this is not the case. If x2 is different, then the same x1 can be compressed into a different string.

Reply on news.ycombinator.com