What I do is pretty similar to what I did before in my CTO roles, just a bit more strategy and less execution. Clients come to me with questions and problems and I help them solve them based on my experience. If you've got multiple C levels, middle management and a board, you're in a similarly consulting position also as a full time CTO from what I've seen.
I don't go anywhere _near_ LLMs for this, I'd find it somewhere between disrespectful and unethical. They're paying for _my_ expertise. I can imagine it could make a decent alternative to Google and Wikipedia for research purposes, but I'd have to double check it all anyway. I don't see how it'd make my work any easier.