> I thought they just returned unsold boxes back to the vendor?

Even in the cases where the vendor/distributor allowed that (and discount bins were a thing because they often did not, at least at no restocking fee), the retailer still accepted the risk that the vendor or distributor would be defunct. If they had a consignment model where they only paid contingent on a retail sale, that was different, but (at least AFAIK) that was never the norm for boxed software.