Delivering software to your customer is an interaction. You need more of those, not less of those.
"Stories are "just right" when the whole team can finish four to ten per week, or about six on average." [0]
"For stories that are too big, work with your customers to split them into smaller stories." [0]
If you think it's just "Everything gets shipped every day or two and no other changes get made to how the team functions." then I have not been clear. It's a massive mindset shift, and it comes with a number of other important changes, all of which are best read from the experts themselves, rather than interpreted through me in an HN comment.
Suffice it to say, it's all been accounted for. Agile works.
[0] https://www.amazon.com/Art-Agile-Development-Pragmatic-Softw...