mmap() basically adds a new swap file that is only to be used for the contents of that specific file.

This means it does page eviction on memory pressure and also results in deduplication as mmap'd memory of the same file can be used by any process that has access to it.

If the data access is sequential in nature (or at least has some form of locality in function of time) it could end up using less memory indeed, even if the whole file ends up being read during the training process, because unused pages will be evicted by the OS to make room for needed ones.

What is interesting to me here is that it's such a marvel to these ML guys. It's a fundamental OS facility and looks like using mmap() instead of using read() on malloc()'d memory should have been the strategy here from the start.