The limits weren't always exactly 32 bits, but it was common to find that the usable address space supported by a motherboard or its chipset was at most one bit wider than necessary for fully populating its DRAM slots; ie. there
wasn't physical address space to spare for a significant fraction of it to be assigned to peripherals. That meant there was continued pressure for peripherals to map only increasingly small portions of their memory into the host address space, even though the CPUs themselves usually supported physical addressing that was comfortably spacious enough.
My first desktop with DDR3 had four DIMM slots supporting 8GB modules, but the total address space was limited to 32GB including all address space used by peripherals, and the peripherals were all mapped below 4GB. So it was effectively a 35-bit system, even though Intel never shipped an x86-64 CPU with less than 36-bit physical address space and AMD never shipped less than 40-bit. Significant parts of the PC ecosystem were being stingy with address lines.