I was just about to say this. In the last few years the only people I've heard call it the Freedom Tower are people who are NOT from NYC.
I mostly call it The World Trade Center, but sometimes I use Freedom Tower in a sort of pejorative way.
I've been in NYC for about 10 years and I've lived in the financial district for about 7: I hear both pretty regularly.
I guess that works but the term seems overly broad – WTC 3-7 are also part of the "World Trade Center". So what do you call that building specifically? WTC 1, I guess?
Kinda like when people say they're going to "the city", they mean Manhattan. If you're going to Brooklyn, yes, that's part of New York City, but then you say you're going to Brooklyn, you don't say you're going to "the city".
Typically, I've heard "World Trade" when referring to 1 WTC, and "One World Trade" when needing to disambiguate.
I usually hear people here in NYC refer to it as "One World Trade Center" which at the same time covers its building number in the complex, and the fact that it is a single tower as opposed to two.