Ask HN: What is the fastest way to load .csv file into pandas?

Loading the CSV through read_CSV then maybe querying it as json/dict.

So read_csv(file).to_dict(orient='records')

That changes the CSV data into a list of objects. I had this problem in a project I was working on and it speed up the processing.

You could then also revert the data back to CSV through DataFrame(list).to_csv(path to save the file)

Parsing ASCII simply takes time. What you describe sounds like a use case for SQLite. Parse once when building the database. When indexed properly, searching should be much faster.

Perhaps loading a file from disk into pandas every request is not the right strategy to begin with.