The concept is simple. The syntax can be terse, but it doesn't have to be. Modestly capable Excel users take advantage of something just like it.
The only time I've ever seen it become difficult is when either the conditional or result expressions get hairy or depend on undescriptive variable names. And when that's the case, well, you're going to have the same problem with the expressions in your if/else, albeit spread out over a few more lines by convention (which one could do with a ternary expression just as well).
I'm at the point where I suspect complaints about the ternary operator are a cargo cult repetition thing, and the developers who are scared of it are scared because they were either taught by someone else to be scared, or they're suffering from a confirmation bias problem where they notice confusing conditionals more when they're associated with the operator.