It's a native code execution exploit.
He exploits the parser to corrupt the stack frame, and executes raw x86 instructions embedded in that file.
How exactly it converts that pile of gibberish into a valid x86 program is beyond even my comprehension.
Alcaro's always been a wizard at stuff like this. Here's an example of an SNES ROM image he crafted to execute native x86 code from inside ZSNES: https://www.youtube.com/watch?v=Q3SOYneC7mU (Yes, you can potentially get a real PC virus from loading up an SNES ROM in ZSNES. Or a specially crafted source code file in xkas v06.)