But people hike/climb to the top of 14ers (14k+ feet) here in Colorado all the time without additional gear/oxygen. 4000 meters is only 13k' and change.
Ears would pop going up or down if they didn't pressurize it, but it would be livable -- especially if you pumped in extra oxygen on the higher floors.
But if you're building it that big anyway, you could certainly pressurize it too, though that would make the HVAC tricky, though you'd probably need to have zones of no more than 100 floors that were sealed from each other, or the weight of the air on the higher floors would raise the pressure on the lower floors.
If you're spending a trillion dollars to build a building, might as well make it comfortable, right? :)
* El Alto (Bolivia, near La Paz) 4,150m (13,615ft) population of 974,754 in 2011.
* Potosí (Bolivia) 4,090m (13,420ft) population of 240,966 in 2012
I've never been so high. Once I went from my city Buenos Aires (25m, 82ft) to Abra Pampa (3507m, 11505ft). I had to be quiet (walk instead of running) and even talk slowly, but I was fine. (Some people get worse symptoms.) I stayed for a day, but after some weeks you get use to the height an can live quite normally.