I think it's also about prefetching amp pages from Google search. You can't do that without the cdn.
Sure google could modify chrome to allow this and not block it. but you'd be asking for even more technical blockers. And you'd have something that only works for a small portion of the web.