Totally agree. Even if you like your manager, there's still a huge risk of saying the wrong thing and being labeled a problem child.
I'm worried that 1-1s are becoming the next Agile - just another process used to increase control under the guise of empowerment