Right. You shouldn't "just" skip those meetings. You should skip those meetings.
As in, you should communicate with the organizer and give your reason for the meeting not being a good use of your time -- or rather, not a good use of the company's money that is paying for your time to be in the meeting rather than doing something else.
If the meeting really is overly broad, then the organizer probably won't really care; they invited everyone who they thought might need to be involved, and from their standpoint it's better to overinvite than underinvite. If you point out that you're extra, then you're taking the potential work to figure that out off of their hands.
In the best case, they'll reconsider the invite list and/or scope of the meeting. In the worst case, they'll fire you. In which case there's a decent chance that it was actually the best case, but you couldn't recognize it at the time.
But I suspect that it's a lot more common for the outcome to be "uh, ok, thanks for letting me know."