Comment by PythonicAlpha | Hacker News Reader

Comment by PythonicAlpha | Hacker News Reader

Parent Full thread

More discussion would really help here.

I think, in some cases, spaced allocation will come (nearly) for free, since with 64bit addressing, there is enough space reserve (most processors can only use a fraction of this space as physical RAM). The other point is, that with the use of MMUs and a proper spacing (regarding the MMUs page size), the gaps come nearly "for free".

But I think, that needs very careful thought, to make it right and not unnecessarily introduce new tradeoffs, as you very talking about.

What you should not do, is use such spacing for small allocations. An other point would be, to inform the allocator, that it is likely/unlikely that a reallocation will be done ... in most cases, such information is available.

An other point would be, to inform the allocator, that it is likely/unlikely that a reallocation will be done ... in most cases, such information is available.

I agree. A lot of blocks which get allocated are never resized (e.g. objects), while other blocks may be subjected to frequent resizing - expanding buffers and strings being the most common example. malloc()/realloc() are very simple APIs, and work for the general purpose of "I want to allocate some memory/resize it", but this simplicity also means that they can't take advantage of specific usage patterns very well.

Working with data of indeterminate size is always going to involve more complexity, and while it may look like a solved problem (just resize when there is not enough space left) as evidenced by the proliferation of string implementations in various languages, the efficiency aspects are more subtle and still well within the area of "programmer must know what he/she is doing". Designing systems and protocols that don't need resizing of allocations, for example (how long something is, is easily calculated and known in advance.)

The "one size fits it all" notion of malloc/free/realloc is just wrong. When you want optimal allocation performance, you can't get around specialized allocators of various sorts for various needs.

Reply on news.ycombinator.com