Hacker News Reader

Seems a bit overly simplistic and not realistic. Most real life data wouldn't be ASCII based. I deal with a lot of large CSVs and do a lot of processing (multi-terabyte CSVs). Null bytes, invalid unicode, all kinds of issues are very common in real world data. Not to mention improperly formatted CSVs (not closing quotes, etc).

View on HN

I don't know, huge amounts of real-world data are ASCII in my experience, but it is also easy to program and cheap at runtime to scan ahead in your input to check if the high bit is universally clear. Current computers can do this at absurd speeds (~64B/cycle or ~200GB/s).

Reply on news.ycombinator.com