That's the big blocker - but, without wishing to be too mean to NXP (sorry Joppe!), the A700x MX51 microcontroller used in the YubiKey Neo is pretty old. Really, most crypto chips are at their core: they almost always just make them out of old ones with new bits glued on to (reasonably) try to reduce (high) NIST/etc certification costs. (Seriously, it's still carting a hardware DES implementation around.)

I believe it has a 2048-bit multiplier with a particular (patented?) form of blinding to protect against side-channel attacks. That's used for RSA; it can't do 3072/4096 bit RSA keys without tricks as it stands unfortunately.

The existing NXP ECC implementation is also not in JCOP's applets by default, I think? From what I saw, it (ab)uses the RSA multiplier. That can go up to 320 bit curves in theory without running out of scratch space, and is a generic implementation of prime fields over GF(p), so you'd think it'd be able to do some operations on Curve25519 fine in principle.

And it can, but in my testing it did what I thought it'd do, and leaked side-channels when used with the NIST curves and djb's curves, because the blinding on the RSA multiplier was simply not designed to cope with the regular structure of the special prime fields that most elliptic curves use. And the performance of random prime fields (like Brainpool) in software is awful, so no-one else wants to use those.

You've also got the problem that the existing implementation there can only do ECDSA (not the EdDSA used by Ed25519) and doesn't have SHA-512 around, too.

A pure software implementation that avoids using the RSA multiplier would be much safer, and I think in principle would be able to do the other CFRG favourite, Ed448-Goldilocks, as well. That is what some HSM vendors are doing. It'd be a very tight fit on the NXP A700x, however. I'll leave that to NXP, if they want to.

I myself want an open crypto chip, not one that still needs blobs in 2015. And I've seen a few movements in that direction.