Right now a lot of people who buy electric cars are wealthier people who also own other cars. This is not inherently bad and, for example, if a household that is going to own multiple cars either way chooses to make one of those electric it may be a win for the environment, but if people are buying extra cars because they are excited about electric cars but also need a conventional car when they go longer distances, that may not actually be that great.
Until there are sufficient charger networks everywhere, it might actually be better to incentives phevs to build up the infrastructure and ensure that people who can only afford to own one car but need to travel longer distances can still use electricity for shorter trips in the form of phevs rather than continuing to use conventional cars. It's not completely clear if EV adoption will continue to accelerate or if it's going to hit limits when we run out of people who can easily switch. Toyota may have a point in that the current timetable for full adoption of evs may be unrealistic, and if it is, it might be better to compromise slightly more if it makes it easier to build up the infrastructure on a more realistic timetable rather than risk hitting a wall
If we set the goal as 100% of vehicles being EVs in a relatively short amount of time, we're going to run into problems who are currently unable to use EVs because of the lack of a fast charger network. (In reality, battery swapping would probably make sense for longer trips once everyone is using evs, but there is zero infrastructure for that right now.)
That said, another issue with electric cars in general is that for the type of short trip where they make the most sense, it might arguably be better for the environment to focus on electric public transportation when possible anyway (especially non-battery-powered street cars and light rail which aren't affected by supply of some of the elements required for batteries).
Also, if we can't hit the targets for EV adoption we may have difficulty hitting environmental goals in general.