When you joined a project is irrelevant. If you were hired two days ago and only found out about this now: tough luck, you have some work to do [0]. Even "stable", older platforms like Cobol have to deal with this (Micro Focus makes money off it with products like their Cobol set of tools (http://www.microfocus.com/mcro/cobol/index.aspx) [1]).

[0] Or maybe not. Your non-core [2] apps that run on 1.8.7 will still work tomorrow. Most of these applications are internal too so the risk is even less.

[1] There's almost always a 3rd party vendor who'll take advantage of a situations like this. They will make it your life easier and they'll charge you accordingly. See http://railslts.com/ for another example.

[2] If the applications that are putting the food on your table are running in an environment where no one thinks about this kind of stuff, then maybe it's time to take the wheel and start educating your team on why this is important.