That's simple. If it is a model that was trained with only that person's data and is therefore connected to it, it is PII (personally identifiable data) and must be deleted.

If the model aggregates the data of multiple people and is disconnected from the individual it does not constitute PII and can be kept.

Generally there are rules about using only anonymized and aggregated data in ML systems inside corporations that are reviewed by lawyers.

Also if you are using the data to help the user, and it's explained to the user, it is legal. But I'm not a GDPR expert, others here on HN know much more about it.