Heroku's value-add is that you don't need a system administrator: if they support the services your app needs, then they automate system administration for you.

Digital Ocean will sell you a VPS (essentially a machine) and grant you root-level access to it. You can run whatever you want, and deploy and manage it however you want.

Heroku will sell you a dyno (essentially a worker process) and grant you access to a set of deployment and management tools. You push your application code up to Heroku, and it automatically sets up the front-facing web server, the database, the cache service, etc., according to the add-ons you've selected. If you need to scale up, just purchase more dynos.

For complex apps, Heroku might not meet your needs and you might want some VPSs instead. In terms of price, though, you'd need to include system administration man-hours when comparing.