When people refer to karma, they don't mean it literally, it's meant more in the sense of "reputation".
I've heard arguments such as "I know god exists because karma exists" which just seems like nonsense to me.
General use of karma is basically saying "what goes around, comes around". It's a derivative of the concept from dharmic religions.
Also, on the other side, your reputation of being an asshole can get around pretty quickly.