Note that the text of both editions (as well the Fate Toolkit and some other goodies) is dual-licensed under CC-BY or OGL: http://www.faterpg.com/licensing/licensing-fate-cc-by/
The Fate SRD site is a third-party creation that presents the texts in a more convenient/organized way: https://fate-srd.com/
There's really no need to buy any books, but they're very reasonably-priced; the Core and Toolkit books are also very well-constructed and can withstand a good deal of usage. Though you can technically play with standard d6's, I'd recommend picking up some Fate (or Fudge) dice; I got a couple sets on Amazon, each set having enough dice for 3 players.
For stories/settings, there are loads of freely-available "Worlds of Adventure" on DriveThruRPG: http://www.drivethrurpg.com/browse/pub/2152/Evil-Hat-Product...
--
You might also like to check out Dungeon World if you haven't already: http://www.dungeon-world.com/
It similarly is CC-BY and has an "SRD" site: http://book.dwgazetteer.com/
IMO it's considerably easier to grok than Fate, especially for people new to RPGs, but YMMV. It also has a pretty expansive section of "rules" for the GM, which leaves room for little doubt about what the GM should be doing (and is, in large part, applicable to other games as well). It also uses standard d6's, so there's no need to invest in special dice (though the Fate dice sets aren't terribly expensive anyway).
I'll also plug Open Legend (I'm unaffiliated other than being a KS backer): http://www.openlegendrpg.com/
--
No matter what you decide to play, I highly recommend spending some time watching some "actual play" videos beforehand. I've found the Rollplay R&D series enjoyable: https://youtu.be/ooa-apRt2wk