I know a woman that cannot stand it. Doing it in internal meetings will get you a beating. If you were to do it in a business meeting where she was the client, she would probably never speak to you again and take the business elsewhere.
If you do it once or twice then no big deal, but if you do it repeatedly, or a bunch of fingers one right after the other then it's rude.
Some people hate it, so you might want to be aware of that.
Some people find it distracting, like tapping a pen, so you might want to be aware of that too.
It's kind of rude when talking to someone or in a meeting, otherwise not a big deal.
"Rude" things are 100% superficial, pretty much by definition of rude. (As soon as something becomes an issue in terms of actual consequences, we stop using the word "rude" and starting saying other things: unacceptable, not right, fucked up)