This isn't something people can or should have to do on their own. For it to work, it needs to be a company wide expectation on a company wide schedule that immediate interruptions are off limits during certain times.
I also generally agree with "you don't need an app for that" for pretty much everything... except that this app would need to integrate and work around online calendars. And I want it to queue people so I can answer them in order. And I don't necessarily want to have a specific time for interruptions, I really want people to try to answer their own question for 5-10 minutes before interrupting me. I'm happy to answer questions, but I get a lot of questions that people can easily answer on their own with 30 seconds of effort.