Ask HN: Does any language compiler/runtime implement somthing like this?
Together with the "mov edi,edi" style hot-patch track described by Raymond Chen, this could make stack allocation, protection and randomization runtime-selectable at essentially zero cost.
Expanding on myself from http://news.ycombinator.com/item?id=3030183 :
If you're allocating the stack frame on the heap, you can just replace the prologue (e.g. x86 intel syntax, cdecl calling convention)
sub esp, LOCAL_SPACE_NEEDED
with push LOCAL_SPACE_NEEDED
push ARG_SIZE_TO_COPY
call allocate_frame
(where allocate_frame allocates enough space for storing link to previous stack frame, all args and all locals; then, it copies all arguments, return address and stack pointer, switches stacks to the new stacks and returns from the call)You can modify the return to call "deallocate_frame" (which would switch stacks back before returning), or you can just make sure the return address on the new stack points to "deallocate_frame_and_return", and use a regular return in place.
The call to "allocate_frame" could be configured at runtime to patch its caller to just do "sub esp, LOCAL_SPACE_NEEDED" for maximum speed. Or it could just check stack overflow; or it could allocate frames on heap for better memory use; and it could mprotect/virtualprotect the edges of this frame for protection.
The cost could be essentially zero if this is not used, if the function prologue is replaced from:
entrypoint:
sub esp, LOCAL_SPACE_NEEDED
to pre_entry:
push LOCAL_SPACE_NEEDED
push ARG_SIZE_TO_COPY
call allocate_frame
jmp stack_set
entrypoint:
mov edi, edi ; or jmp pre_entry depending on default
sub esp, LOCAL_SPACE_NEEDED
stack_set:
Where, at runtime, "mov edi, edi" and "jmp pre_entry" can be patched into the entry point to diable / enable stack modification.(Starting with the jmp pre_entry in place would let the "allocate_frame" patch each call first time through, thus making it possible to the symbol table. However, with W^X, this might slow down first call to a routine significantly)
Of course, there are other details to work out here, like alloca() support, frame unwinds and debugger interaction, but they are all easy to do.
And then there's the issue of calling a function not supporting this convention. That would require allocating a frame big enough for that function to work (which might be a large frame for recursive functions, but it would be discarded when they return).
Your thoughts?
edit:minor formatting and type fixes.