It sounds like you want to hold office hours. You don't need an app for that, you just need to respond to requests by telling people when your office hours are.
This isn't something people can or should have to do on their own. For it to work, it needs to be a company wide expectation on a company wide schedule that immediate interruptions are off limits during certain times.
I also generally agree with "you don't need an app for that" for pretty much everything... except that this app would need to integrate and work around online calendars. And I want it to queue people so I can answer them in order. And I don't necessarily want to have a specific time for interruptions, I really want people to try to answer their own question for 5-10 minutes before interrupting me. I'm happy to answer questions, but I get a lot of questions that people can easily answer on their own with 30 seconds of effort.
The last open floor plan I worked in established that the meeting rooms were allowed to be used for engineer quiet time, and that someone in the meeting room was not to be interrupted. It was great for the first couple of months in the building, but eventually the meeting rooms were crowded with meetings and there just weren't enough for engineers to use for focus time very often. Headphones became the only means of staying sane.
When I tried office hours, I was actually in shared offices. 4-6 people to a room. It was a game company, and I was supporting artists. The artists were the worst about not respecting office hours, most of the programmers left me alone because they also wanted to be left alone to concentrate.
Ultimately I decided I was setting a bad example by trying to establish office hours for myself. The company wasn't that great at planning, and the interruptions I was getting were a symptom of not enough training and formal communication. What was needed was a little more communication, not less. (*But the right kind, of course.)