First, they get bored out of their mind. They're young, they want to grow, but they can't. While you're off to work, networking, learning, your spouse is at home, trying their best to chase away the feeling of stagnation.
It's OK for the first few months - they go for walks, cook, read, go to the gym - anything to keep busy, but unless there's a child that takes up their entire time, they just get bored, then they get sad, and then they get depressed.
It takes a hard toll on the relationship, which, ironically, is the primary reason why spouses agree to this in the first place. Most just want to live with their husbands or wives and have a normal life. Long distance isn't a realistic option when you're talking about a 6 year H1B visa.
Being unable to work makes the relationship one-sided, because one person goes to work, provides for the family, and has various experiences which slowly add up and allows them to grow personally and professionally. In addition, if you come from a society where gender roles are heavily promoted culturally, it is especially hard when the roles get reversed when the couple is in the US. Typically, men lose confidence in their ability to provide for the family and get very depressed. Women from cultures where there's an expectation that they'll be a homemaker, have much less stigma associated with not working, especially if there's a child to take care of.
They are also competing with other couples in various areas: social status, disposable income (that lets you go on vacations) or ability to buy a house, and they'll be losing ground simply because of the income level (less so for FAANG employees, but there are plenty of places hiring on an H1B visa that pay the minimum they can to meet the H1B criteria).
There are only 2 realistic options most spouses end up taking: the spouse works illegally (clearly a risky move) or the spouse changes their legal status: divorces you and marries a citizen (risky and further damages the quality of your relationship), tries to go to school on an F1 status (which now requires even more income and puts more pressure on the H1B holder), or the relationship deteriorates to the point where the non-working spouse goes back to their home country because they are unhappy here.
Finally, I will say that yes there are many real issues with the H1B visa, but disallowing the spouse to work is not a solution to any of them. It only manages to get these young couples behind in life.
I am fortunate enough to have come out of this ordeal fairly unscathed, but others I know have not been so lucky.