Super Mario 64 is a 3D game. The emulator can render the polygons at any resolution. The 2D textures should be replaced or carefully scaled, though.