Prius had 0.91 kWh battery and the EV1 had 26.4 kWh with NiMH. The EV1 was expensive, $80k to produce in 1996 money. A large part of that had to be the battery.
It's a pretty decent approximation to say that the battery is the only problem for EVs. Everything else is either standard car stuff, or small/cheap/simple enough not to be a major problem. Nearly all of the progress in EVs that we've seen since the EV1 has been down to battery improvements. Take the EV1 and give it a modern battery and it'll be pretty decent. Take any modern EV and give it a 1999 battery and it'll be absolutely awful.
The problem I see is that many people are skeptical of EVs, most people don’t have experience with EVs or any existing EV infrastructure (a charger at home), and having an extra car kind of sucks (parking, registration, insurance, etc).