The problem is that I can't think of any other design that a modern smartphone would have other than "piece of glass encased in plastic/metal, with maybe a button underneath". As far as I can see the screen is the phone, and given that constraint there isn't really anywhere else to go.
Don't forget that to make that form factor work, you need a whole software stack that is able to replace the physical buttons adequately. You need an on-screen keyboard capable of at least equalling a physical keyboard. You need to have virtual buttons that are easy to use. You're going to be doing a lot of scrolling on that tiny screen, so you're going to need some pretty good graphics routines. To make that scrolling anything other than an awful experience you're going to need low-latency right from the capacitive input right through to the content actually moving on the screen. Not easy, especially with the chip used in the original iPhone.
Think about Android. If Android had have been first to market, and they had actually undergone the post-iPhone transformation without the iPhone being present (because don't forget, it was originally going to be a phone with a physical keyboard), they would have released a touch-screen phone with Android 1.0 on it. Android 1.0 was awful. Battery life was rubbish, graphics performance was pathetic, even the keyboard was full of fail. If that had have been the defining capacitive screen smartphone, we may have decided that touchscreen phones were not the right direction.
It took Google, with a large chunk of the smartest engineers on the planet, about 4 years to ge to a point where Android was truly competitive with iOS. It took about 5 years for Microsoft to get Windows Phone to the same place.
Without the software engineering effort, the capacitive screen was not an obvious solution. The fact that Apple invested in that effort made the benefits obvious to everyone, but this was not the case pre-2007.
It's not that nobody was trying to figure out how to build a smartphone with a big touchscreen, just that they were approaching it from the enterprise space. Usability wasn't important because it wasn't an enterprise selling point. Pen input was important because signature input was key for enterprise. Apple ignored enterprise at first and designed for the consumer market, which allowed for blasphemies like throwing out pen support. The big change was making smartphones a consumer product instead of an enterprise product.
Google didn't take people off of other projects, and turn every engineer that they had at the problem of "making Android competitive," so this statement is a bit of an exaggeration.