RAM Disks and Saving Your SSD from AFL Fuzzing | Hacker News Reader

RAM Disks and Saving Your SSD from AFL Fuzzing

cipherdyne.org

25 pointsmichaelrash12 comments

I'm not sure how one would get statistics on this, but I can't help but wonder how much of those ~2,000 writes/sec are actually getting written to disk.

Basically, the same property that gives AFL fuzzing such high write amplification (it's repeatedly changing only a few bytes within a single sector of a file) should also mean that the data in question is mostly staying in cache-- the OS is going to flush that cache out to disk periodically, but it's probably not going to be flushing that cache after every write; that would be pretty slow and silly.

My 840 EVO tracks its total number of LBAs written, and makes it available as a SMART attribute. Assuming your drive does the same, it might be interesting to see how quickly that number goes up while fuzzing versus how quickly it goes up with a more "normal" workload.

I can't help but wonder how much of those ~2,000 writes/sec are actually getting written to disk

Approximately zero would be my fairly confident guess, although I'm more familiar with the Linux page cache than OSX. For repeated writes to the same small file without an explicit sync, I don't think there should be any difference between a RAM disk and a regular drive. One way to test would be to measure the number of writes/sec to the the RAM disk, the SSD, and a slow spinning drive --- my theory says they should be very close to the same. But I'd love to learn by being shown otherwise.

Remember from the article that AFL is being run from a VM. So, my guess is that OS X (host) doesn't know anything about whether what is being written on the VM is a small file or not. There are two possibilities: 1) either the OS X activity monitor is actually measuring disk+cache when it says "disk writes" or 2) disk writes really are happening as fast as shown in the graphs. My bet is the later. What saves things here is the usage of the RAM disk on the guest, since at least that is a clean mechanism for really only using RAM instead of disk - and that carries through from the guest to the host.

You make good points, and I hadn't considered how the VM would affect things. I'm still doubtful that frequent rewrites of the same file would actually hit the flash of the SSD, but with much less certainty. If it is writing this frequently, this seems like a place for an easy optimization of the VM. Or maybe I'm misunderstanding, and it's writing a series of small files rather than rewriting the same one?

On the VM, AFL is rewriting the same small file millions of times, and the fuzzed program is reading this file from disk (as opposed to stdin). I think the main question is why does the OS X activity monitor show a massive spike in disk writes when AFL is started and a corresponding decrease when it is stopped? This is 100% reproducible, and the magnitude of the change is unmistakable. It is absolutely clear that the spike in disk writes is due to AFL activity. Further, this same spike does not happen when running under a RAM disk on the VM. I'm confident you would see the same result. So, either the OS X activity monitor has some special way to measure caching in the VM and that is what it is showing (surely this is unlikely), or the writes really are hitting the SSD when not running under the RAM disk. If not the later, what other (realistic) possibility is there?

How about this interpretation (which is only a guess, although I think it's likely):

Activity Monitor does not distinguish between cache hits and misses, and counts the number of reads and writes to the host filesystem rather than summing the requests to the physical devices. Parallels presents the guest with a virtual filesystem, and the device driver for this filesystem does no caching of its own, as this would waste memory due to double caching. Instead, it merely passes requests through to the real filesystem on the host. These requests (read or write) produce system calls, and this is what is being reported in Activity Monitor regardless of whether they use the page cache or physical SSD.

One way to test this would be to use DTrace, which can distinguish between requests to the filesystem and activity that reaches the device. The scripts "hfsslower.d" and "iosnoop" are here: http://dtrace.org/blogs/brendan/2011/10/10/top-10-dtrace-scr...

Ok, that sounds like a reasonable alternative, and Brendan Greeg is certainly a powerful force in this area. I'll run dtrace and see what happens, but I'm going to let a current long-running fuzzing run finish first.

Aside: I was looking for an SSD recently and was considering the 840 EVO. If you haven't heard, this drive has a write performance issue which has been corrected in a recent firmware update.

http://www.anandtech.com/show/8617/samsung-releases-firmware...

I'm not sure it's really that big of a problem. If you look at http://techreport.com/review/27436/the-ssd-endurance-experim... you can see that even consumer drives vastly outperform the given rates.

They are not regular consumer ssd drives, they are the most expensive ones (among the consumer ssd drives)

The difference in price is not that great

They are regular consumer drives. They are even markedet and priced as that.

They don't even have super capacitors, which more and more consumer drives are coming equipped with standard these days.

Reply on news.ycombinator.com