Yes, very much so. Have you seen the drivers' ratings? They usually hover around 4.8-5.0. If you're rated below 4.7 for both Lyft and Uber, I believe you're either on 'probation' or removed from their service.
You accidentally rating someone 3 for an average service can actually cost them their job. I'm hoping their algorithm uses some sort of a weighted average to offset the ratings of individuals such as yourself who consistently rate very low.