Even though I have two kinds of cons cells (regular and lazy) as well as the ability of objects to implement car and cdr and then work with those functions, I'm still fairly reluctant.

I wouldn't want source code to use objects, but real cons cells. Objects are heavier-weight. Each object is a cons-cell sized object, plus something in dynamic heap.

There are print-read consistency issues. Lazy conses and regular conses are indistinguishable in print. If you print some lazy conses, and read that back, you get regular conses. Of course, an infinite list made using lazy conses will not print in a readable way, so we can sweep that under the rug.

Objects implementing car and cdr have arbitrary print methods too. They won't print as lists. Those programmed to print as lists won't have print-read consistency.