I've had problems with performance over large data sets, maybe 1M records and a few < 300 char fields per record.

I've never implemented full text search using a dedicated database so maybe 1M is too much to ask for.