Haven't done much Perl hacking, good to know it's more-or-less pure reference counting.
Paging can occur as a consequence of copying a reference.
var p1 = *A // let p1 be a reference to A
// something in the interim
var p2 = p1 // create a new reference to A via p1
If A is no longer paged in incrementing its reference count (as a consequence of copying p1 into p2) can cause a page fault. There's are also gotchas with destroying references such as a collection having to effectively iterate over its contents to decrement each reference count, again possibly leading to additional paging.
This assumes an object carries its reference count with it, which every reference counting system I'm aware of does. I suppose you could technically keep reference counts in a common space, though growing that would be a bit rough.
A GC isn't a magic bullet or anything, but they don't need to cause any paging as a result of reference changes until a sweep; and even then, they only need to page in the live set which is more likely to be resident in memory.
I'm... skeptical Apple's introduction of ARC is really performance oriented. Seems more like they tried and failed to build a workable GC for Objective-C with Tiger. Haven't worked with much Objective-C since 10.4 though, so I'm not completely settled either way.