A good example of this is the SQLite documentation page "Why Is SQLite Coded In C". Among other things, it describes those memory safety issues as "the easy problems" compared to "the rather more difficult problem of computing a correct answer to an SQL statement".
Not all of our programs are like SQLite, of course (and not all of us mere mortals are like its developers). But I would certainly say that just because you've eliminated memory safety bugs doesn't mean you've eliminated all bugs. Depending on the program, you might not even have eliminated most bugs.