Their database prices are actually similar to AWS RDS Postgres now, and weren't that much higher before, especially with the nice features & interface you get in return.
Their dynos are insanely expensive though. A dedicated instance (VPS) with 2.5GB of RAM will set you back $250/month, while a similar EC2 instance will cost <$75/month.
It's hard to justify that the Heroku platform makes up for that $175/month/server difference compared to what AWS & co have nowadays.