There's a need for a website like this - meetup is the closest thing I've found, otherwise you just kind of have to be in the know. Find one person who's well known in the circle you're trying to find conferences for, then email them or google them to see other places they've spoken. All you really need is that one insider to find out everything that's worth going to.

But outside of that advice, someone needs to build this!