How is there a smaller set because of OOP? Just filter the functions by parameter type to restrict which function may access your pointer.
This lets you isolate where your contract/invariant violations are more easily. Once addressed in the member functions, either it fixes it in the callers, or their use is now invalid and becomes a compile or runtime error.