More importantly – will they provide zero-copy I/O like you can get with Intel network cards via their DPDK [1] or PF_RING/DNA [2]?
[1] http://dpdk.org/ [2] http://www.ntop.org/products/pf_ring/dna/
Obviously I have no idea on the network controller or its sdk/driver support.
WRT to the opteron A1100 yes, I could see your comment. Something like a box of A1100 blades plugging 802.3ap to a common backplane, a trident chip there, and then a bunch of (Q)SFP+ northbound. A couple hundred gbs for around 150 watts of networking.
When I see A1100 I think an IO node with 10s of SATA disks attached. In that case Im only getting 10 or 20 per rack. A backplane makes less sense to me, running two DAC phys per box to a TOR I could see.
It doesn't matter. Modern processors have the complete opposite problem. It isn't that transistors are expensive, it's that they're so cheap you end up with too many and they generate too much heat. If you can stick a block on there which 60% of your customers can use and the other 40% can shut off to leave more headroom for frequency scaling, it's a win.
Also, the number of gates you need for a network controller is small.
As an entry into the server market, this sounds awesome; for things like storage aggregation and vm migration, gigabit ethernet is becoming a real bottleneck for many applications as the core counts has gone up.