While some important records are public, many are not. It isn't obvious that all debtors want the details of their debt made world-readable, for instance.
For stronger privacy we'd need ring signatures or zksnarks, so a person can't be identified by the flow of payments, but those technologies exist on various blockchains already.