you can, but they become a special second-class type of reference separate from ownership.
Note that relational databases don't have back pointers either. All references between tables are symbolic. Last century, people tried to make databases that were general pointer graphs (google "network database" or "object database") - broadly speaking, it was a disaster.