It’s not simply diffusion complexity. A lot of cellular transport is dependent on directed transport via the cytoskeleton and myosin. For example a 1m long neutron would need years to move proteins from one end to the other if it just relied on diffusion.