In my experience, the DB layer is always the hardest to scale. At a certain point, vertically scaling system resources starts producing ever diminishing returns. At that point, the natural progression is to move to a clustered setup with multiple DB nodes, etc, but that introduces a whole new universe of complexity and failure points.
My takeaway is that proper data modeling at the initial stages of designing a software architecture is absolutely essential to long-term scalability. You really need to consider the type of data you are storing, its shape, and the access patterns involved with reading/writing it. Of course, since it is basically impossible to get this correct the first time, it is even more important to have a good abstraction layer between your business logic and your database to make it possible to change your data store in the future.