You are correct, you have to make time for reading by reducing other entertainment time. I hit 100 books last year without too much trouble, but I've really struggled with my goal of 125 this year (123 now, and two days left).
2 books per week was an amount that makes me happy, so I kept up with it easily. I fell behind early this year because my wife and I wanted to catch up on a couple of TV shows, which meant I needed to average over 2.5 books per week. That was enough to kill part of the fun, which is why I've struggled.
A goal is great because it helps you get started on the next book when you might want to do something easier. But if it's too big it just makes reading a chore.