I recently found an article by Apple explaining how they optimized a transformer to work on Apple Neural Engine [1]. This was before their recent hardware refresh with 192 GB unified memory upgrade.
Apple could have a real chance to push their boxes as an alternative to dedicated GPUs.