They added it to avoid game console specific tariffs, so they must have run the numbers and realized paying those would cost less than subsidizing a bunch of clusters.
Sony only removed OtherOS on the Fat after George Hotz discovered and published an exploit that allowed users to break out of the OtherOS sandbox and gain full control over the PS3. This allowed running custom firmware and pirating games with a bit of additional work. Sony judged that they couldn't patch the exploit without disabling access to OtherOS entirely, so that's what they did. Hotz announced his exploit in January 2010 (https://web.archive.org/web/20100129034435/http://geohotps3....), the Yellow Dog Linux team posted a rumor that Sony was getting rid of OtherOS in February 2010 (https://www.xtremeps3.com/2010/02/20/rumor-alert-otheros-to-...), and Sony officially announced the removal of OtherOS due to security concerns in late March 2010 (https://blog.playstation.com/2010/03/28/ps3-firmware-v3-21-u...). As you say, this removal was later found to be illegal in court.