Hacker News Reader

Comment by maxxxxx | Hacker News Reader

A long time ago I found some memory problems in a C++ codebase we had inherited and that crashes from time. In Windows you can protect memory blocks with a function call so I overloaded new and malloc and added some protected blocks before and after the actual memory. Then we ran the code for a lot and it would fault right at the point of the memory overwrite instead of some place later. It made the code very slow but we found problems that were incredibly hard to find in any other way.

View on HN

On illumos you can use libumem to replace malloc() via LD_PRELOAD, and then you can instruct it to do the same thing via the UMEM_DEBUG environment variable. It's a fantastic technique for catching these issues without completely destroying application performance.

These days the gcc and clang memory/address sanitizers can do this quite nicely, and they tend to have low overhead as well (in my limited experience so far).

ASan has lower overhead than Valgrind, and is great when you cross-compile something to an embedded/constrained system. I have however observed noticeable overhead on code doing many small allocations.

Which was nice to be able to observe, because the code didn't need to do many small allocations.

When running a test suite at work with ASan + UBsan turned on, the test goes from taking ~30 seconds to ~20 minutes. So there's certainly some pathological cases.

Try it with just ASan. Some of the UBsan checks can be quite expensive.

Also, check how much free ram your computer has when running with ASan. It may be running out of RAM, and e.g. spending a lot of time looking for other memory to free, or even swapping.

The Windows system heaps include a similar feature called PageHeap. You can enabled it for any process using Microsoft's pageheap.exe utility. It uses a lot of extra (virtual) memory and doesn't work well for applications that have their own memory sub-allocator like jemalloc or dlmalloc.

https://docs.microsoft.com/en-us/windows-hardware/drivers/de...

This approach was implemented in Bruce Perens' "Electric Fence" library [1987].

https://en.wikipedia.org/wiki/Electric_Fence

I think he might have developed that while at Pixar.

I had similar memory issues once and was able to trace it using valgrind, which afaik did something similar under the hood.

We did the same thing for some device driver code and also use a tool called "Boundschecker", back in the 90s. I just checked and it still exists!

We also had BoundsChecker but it was my little library that found most bugs eventually :)

Reply on news.ycombinator.com