All else being equal, could the efficiency of malloc_trim depend on heap fragmentation? Could the heap end up with "gaps" of free'd memory that couldn't be returned to the OS independently?
However: Most (all?) general memory allocators have small-allocation buckets for minimizing this effect, which basically groups allocations of specific sizes into different memory pages.
Despite that, virtual address space fragmentation in 32-bit applications is definitely a thing. Memory might still become fragmented over time so that "big" allocations (bigger than a memory page) are failing because there's no big-enough gap left in the processes virtual address space, despite having enough free physical memory available for mapping.
This usually isn't a problem in 64-bit processes, because there's always enough room at the "front" (new memory allocations would basically move like a tidal wave through the 64-bit address space, leaving a mess of fragmented address space behind).
The best way to prevent memory fragmentation of course is to minimize allocation frequency in long-running applications (ideally only allocate memory when the program starts up).