The first transistor was about the size of a Raspberry Pi but I think your point still stands.
I should probably qualify my "first transistor" statement; the device I was referring to is the first prototype created at Bell Labs in 1947. The first transistor that someone could actually buy to build a circuit out of came out a year later and was only a little larger than today's through-hole mounted transistors. They did get used to build early minicomputers, before integrated circuits took over.
> Building something useful out of that would end up comically large.
The transistors are MicroSD-size, not RPi-size, but still, it has been done: