Yeah if http://www.notebookcheck.net/Mobile-Processors-Benchmarklist... is to be trusted, it's indeed quite a bit slower.
What a bummer, I was seriously hoping to replace my 2012 MBA, but now... apparently the Core M processor will unfortunately be a good deal slower, than the one used 3 years ago. Which means I'll probably have to go with a Windows based laptop or wait for the MBP update.