Easy way to prevent CSRF

What is the easiest and most secure way to prevent CSRF?

I was thinking about this for a long time. Every request, for instance images from a site send a HTTP Header called referer. HTTP referer is set in the browser based on what url the request comes from. Since Cross-site resource forgery, means that you are doing resource forgery on a cross site. That means every CSRF attack must send a HTTP referer. So if you are building a website, let say domain.com. Than you can easily check the HTTP referer that it has domain.com and not any other domain to prevent CSRF attack. And I think all web browser sends HTTP referer in every request.

I can not see any holes on this solution can you?