In general for machine learning, yes, but deep learning models with large numbers of parameters such as transformers and stable diffusion do actually memorise part of their training data verbatim. The problem is mainly duplicate data points. E.g.
https://arstechnica.com/information-technology/2023/02/resea...
https://arxiv.org/pdf/2202.07646.pdf Quantifying Memorization Across Neural Language Models