It's not that simple. If you have a model that actually does something useful (e.g. not just doing matmul & conv2d) your model will fail to run on ANE and, instead, the device will move it over to CPU/GPU and turn your iPhone into a heater. I literally had to continually wipe down my iPhone with a wet towel to keep it from overheating so I could build, ct.convert, run, and debug a model I was working on. Apple doesn't document how to keep operations on ANE. A model created by coremltools may run on either CPU, GPU, ANE, but you don't get to choose. And, if you don't know what you're doing and naively build a model, you will likely run on CPU/GPU only. If your batch is too large, too small, if you need to transpose tensors, if you need to expand mismatched tensors to matmul them together, if your model has an IF branch, or a loop, if you breathe the wrong way, your model silently falls off ANE. But you don't know what caused it. You have to open Netron and guess. Also, it may run on ANE on one device, but not another. There's no documentation from Apple about any of this. So, no, you do not simply "use CoreML to program it".

It's a damn shame, too. If we got the APFS documentation we were promised I feel like we'd be one step closer to world peace.