Do you have actual memory usage issues? Is a lack of memory causing problems for other applications?
I am asking because RAM is meant to be used, high allocations to Firefox on a (mostly) idle system isn't too interesting as a metric on its own and should automatically decrease if others need it.