It sounds like you have one of the last of the early 64-bit Macs that had a 32-bit EFI. They were difficult beasts to get working with anything other than officially supported macOS versions. You can follow the guide linked below but basically you need to create a 32/64 hybrid EFI boot image. There were some Windows machines (mainly BayTrail and similar vintage tablets) that also had this issue and the solution is very similar.