If you’re not completely driven to write a book, and there isn’t a book inside you that you just have to get out, then do not bother writing a book. Agents and publishers have nothing to do with it - you don’t need to ask for permission to write your book, it’s a piece of art. Agents and publishing come afterwards - and nobody is going to be interested really anyway, at least in the early stages. So if that is enough to make you confused about writing your book or makes you not want to bother, then don’t write your book. If you still have to write a book, you’ll love doing it despite the difficulty and futility, because the person who enjoys it the most will be you anyway.