In Python you can just loop on anything declaring the __iter__ method, and it will lazily yield results little by little.
IO related objects implement it plus an additional layer of interface so that you can do:
with open('file', [mode, encoding]) as f:
To get an auto closing file handle and then choose:- for `line in f` to lazy read it line by line. This calls __iter__.
- `f.read([byte_count])` to read it all or some bunch of bytes. `f.seek(index)` to move around, etc.
- `f.readlines()` to get a list of all lines in memory.
- `f.close()` if you wish to close the file manually instead of letting the `with` keyword doing it work you.
This interface works for most IO, including files, sockets, in memory buffers, etc.
So you can choose an automatic lazy loading, a manual loading, load everything in memory, etc. And still have a lot of control.
I think rust should get some traits to expose such a common high level interface on top of the current way it deals with files, to ease simple operations. Just make sure the documentation states what you can do to go lower level.
There is a similar trend with many basic topics trying to do that as 3rd party libs too. click makes creating cmd UI very easy on top of argparse which is lower level. pendulum is higher level than datetime. requests higher than urllib. etc. Those all make doing those very common operation super easy.
Now rust is not meant to be Python/Ruby/whatever, being very low level, and checking safety at compile time implies very different requirements.
But those communities have some good concepts on API ergonomics and it would be a shame to not steal ideas from them.
The interface for reading is split into multiple parts. Read is the lower level interface for any stream of bytes, and is what lets you read a specific number of bytes into a preallocated buffer. Seek is implemented for files, but not all streams, because in most cases it makes no sense. There are also a number of file-specific methods directly attached to File for handling permissions and sync and such.
BufRead is the trait that lets you read all the lines, but it is explicitly not implemented for File, since using it for a File would be slow, and generally the wrong way to do things. It does have an implementation for Stdin, because that is already buffered, but if you want to associate a buffer with a file, you have to do it yourself by wrapping it in a ReadBuffer.
If I were to find a problem with the File documentation, it is that it does not mention BufReader.