Umm, wouldn't this break if you moved a directory that is somewhere up the path? Seems like a security issue if you cache what policy applies but the policy could change by user action.
Additionally, we are thinking of evicting the inode associated with the directory from the cache if a directory is moved. Doing this would probably catch a ton of edge cases and make it simpler.