That's not true in my experience. Loading outbound cargo is way more complex, since the stowage plan of the ship dictates where each container goes. Theoretically a lot of containers can be swapped as long as weight is similar, the container type is identical, and the port of discharge is the same. In practice it's still incredibly complex compared to just unloading stuff. While you may need to do less 'digging' on shore, the nitty gritty of the actual operations are way more complex than throwing some boxes ashore.

Import cargo is annoying in that it is mostly random access on pickup. For pickup by train, barge, or feeder ship, a vast minority, you typically don't have cargo manifests until a day or two before pickup at best, so in practice this is also random access-ish. The customs processes are also trickier.

My experience is mostly in Rotterdam and Antwerp, and I'd say the problems in the US probably don't have to do much with automation. Rotterdam and Antwerp have very different automation levels at the biggest container terminals, yet productivity is quite similar.

There is lots of low hanging fruit in optimizing operations, like more collaborative stowage planning, simultaneous unloading and loading operations, and 'modal shifting' from road to rail and water combined with early preannouncement of manifests for trains and barges.

Disclaimer: I'm in the business of consulting and building software for container terminals, so I'll generally be biased towards those solutions.