28 karma · joined December 4, 2017
We think our tool will provide features that can improve AV systems to be better and more accurate, yes (see Figure 7)! We tested that in the last section of our paperLots of future work in that direction is planned.
Most of the work we actually don't use any fancy sketching. Just a GIANT table where we count hashes, and ignore collisions, making it super fast to count. Then we just use Quick-Select to find the top-k counts out of 2^31 buckets.
Second phase we technically use the Space-Saving algorithm, but its generally not needed - we could just use a hash-map and naively count (which we know is actually safe is the data is truly power-law distributed). The Space-Saving algo gives us some greater peace-of-mind though since real data doesn't have to obey our theories :)
Features are just bag-of-words style!
The primary point of our paper is that now we can circle back and really test all these hypothesis about larger n-grams for malware! And even if you only want smaller 6-grams, we get a nice big speedup too :)
>So if I understand correctly... the application here is to take a large number of programs known to be infected with the same malware, and then run this to find the large chunks in common that will be more reliable as a malware signature in the future.
You basically got it! We also find large chunks in benign files too. While just _one_ chunk is pretty indicative, performance is much better by using multiple. Thats where Logistic Regression (+ Lasso) comes in to tell us how important each chunk is to making a decision.
Near term I think there are some NLP applications for this (though not out to 1024 grams!), and I'm hopeful extensions to this work will be useful to bioinformatics problems.
I'm also interested in seeing what craziness people come up with now that large n-grams are an option! Everything I had done/learned before put n>6 as an immediate "why even think about it, you can't compute them" bucket.
There may also be some applications for this in NLP. I'm chatting with a professor who thought this could help speed up some old algorithms that weren't scalable before.
This paper looked specifically at PE files because it's the hardest case of any of the file types (in our opinion & experience), and it's the one we have the most data for. We've built models for many other file types with success using much less data (though we are always looking for more).
At a more technical level, the approach we take in this paper (and most of my research) is fairly orthogonal to what most AV vendors are doing. Even compared to the AI based solutions.
The idea here was to throw away everything we know about the file being a valid Windows PE binary, and try and let the network learn what it needs on its own. Its making the problem harder, but allows us to re-purpose the same code for PDFs, Word Docs, RTF - basically any file format we can get data for. This gives us a lot of potential flexibility that others don't have.
For the Neural Network type stuff we used in this work, I would recommend Michael Nielsen's awesome website as a starting point http://neuralnetworksanddeeplearning.com/ and keras https://keras.io/ is the easiest NN library to pick up and get something going with. Andrew Ng's mooc https://www.coursera.org/learn/machine-learning is also a good starting point for some slightly more general machine learning background.
If you want to avoid the math (not my personal recommendation), I would start there and just mess around. Build small things and start reading more as you get comfortable. Its definitely an area I would encourage learning in an iterative way: try to learn a small amount, try and apply it, repeat.
The reality is there is a big gray area between the two classes. Some cases are really hard to determine, and would be something that would lead to errors in production. Some examples:
What if it is of malicious intent, but the author messed it up and so it doesn't do anything. Is it still malware?
What if it's a program used for encryption for security, but used by malware to create ransomware? Is it malicious now?
What if its a benign program, but a bug causes it to destroy files. Is it malicious?
Some programs are maybe not malicious, but just annoying (like browser toolbar installers). What do we call it? Some systems have a "Potentially Unwanted Software" category for these guys.
Ultimately, it's not easy. Thankfully most binaries are fairly cut-and-dry in terms of which side of the fence do they belong. The hope is that with enough labeled data, we can do a good job for the majority of cases. We don't expect it to ever be perfect. Hitting the hard to distinguish samples is definitely something dig into in the future.
Our work so far has found that data quality used in training is the biggest factor in the performance you should expect. Which isn't surprising, but it seems to be a bigger problem in this space. Some of our first work was dealing with that issue and showing how critical it can be http://www.readcube.com/articles/10.1007/s11416-016-0283-1?a...
Its also a space where data collection and ground truth are expensive, so people try to boot strap solutions in odd ways. Its tough.
As the paper author, I'm not going to name any companies. Especially since one was sharing data with us for this work, and I don't want to inject any of my biases into a particular name. But we've done some fairly extensive testing of the models we've built. Both on data they gave us and data we got ourselves, and our model don't just label everything as malware. It is not quite AV quality at the moment, but we also have limited training data and are really still exploring the different ways one can tackle this problem with AI.
Jon did a great job in this blog post, but if you want more details on why there is so much work to still be done, I'd recommend reading our paper! We tried to make the intro section accessible to people with no AV/malware background.
The dataset is small by AV standards, but we aren't an AV company. We can only use as much as real AV companies are willing to share with us. If you'd like to share more, we would be happy to take it :)
The model is fairly robust to new data, and we tested it with malware from a completely separate source than our training data - so there shouldn't be any share items like icons between the training set and the 2nd testing set. However, we aren't arguing that is of an AV quality today. The main purpose of this research was to get a neural network to train on this kind of data at all, as it is non trivial and common tools (like batch-norm) didn't translate to this problem space.
We are looking at the modification issue! I can't share any results yet since we have to go through pre-publication review, but the issue isn't unknown to us!
Endgame has a great paper on this problem, and showed how they can defeat regular AVs with some fairly simple modifications that don't impact the malware's execution. https://www.blackhat.com/docs/us-17/thursday/us-17-Anderson-...
A lot of that issue comes from people using bad datasets. One of our first papers was about that ( http://www.readcube.com/articles/10.1007/s11416-016-0283-1?a... ), and showed that using the data most people use in their research, benign data collected from clean Microsoft installs, is not sufficient. The model will literally learn to look for the string "Copyright Microsoft Corporation" to decide if something is benign. Everything else ends up getting marked as malicious.
We are using better data in this work, and it does not suffer from this problem. It is not ready to be a real production AV, but it does a fairly good job at separating out benign vs malicious files and dealing with non-trivial examples of both.