As far as I'm aware, you can certainly enter a library outside of your city of residence and read the books there. You just can't borrow them out, presumably because if you then stole said book, they wouldn't be able to fine you.
I've had many public library cards for libraries outside of my local city or county system; as long as you can document who you are and where you live so they can come after you at need, at least in CA as a CA resident, I've never has a problem.