I don’t think that question is very tough. It’s almost certainly copyright infringement. The method of copying doesn’t really matter.
In that case, the consequence would be: If you don’t validate that your AI model is producing something sufficiently different than all of the inputs, you’ve created a copyright infringement Russian roulette.