I struggle to understand how GPT-3 executes code. Is it simply running a python (or any other language) interpreter? Or is GPT-3 itself interpreting and executing python code? If the latter question is true that would be amazing.
It does not execute code, it "guesses" what the output of the code should be, given all the data it has seen during training - and, surprisingly, for many types of problems these guesses are accurate or close to that.