It’s possible that in the specific sense that NIST defined, KYBER-512 isn’t as strong as AES-128. However, that doesn’t mean that it’s less secure in general. E.g. DJB himself wrote a good article[1] on how even though 128-bit AES and 256-bit elliptic curve crypto are thought of as same “security level”, actually there are attacks against AES that just don’t apply to ECC when you consider multi-target security models (i.e., when you consider a population of users not just one). I wouldn’t be surprised if similar things applied to lattice-based crypto, but I don’t know enough about it. And even if we take the reduced security level given by DJB, it still seems big enough to be out of reach to any realistic attack.
But by all means feel free to go one bigger and pick KYBER-768, and I believe lots of people do recommend this. Obviously, there is a performance penalty (as there is when moving from AES 128 to 256), and for PQ schemes there is also more importantly also a big increase in the size of bytes on the wire when public keys have to be exchanged (e.g. in TLS) - in this case a jump from 800 bytes to 1,184 bytes (a 48% increase). (Compare this to ECC public keys which are typically around 32-65 bytes, depending on encoding).