Hacker News Reader

As someone who has also worked on various bits of HTTP and URI parsing I have also wished for such a test suite - or even that these RFCs and specs would come bundled with such a suite.

There are a lot of weird cases to consider. For example, how many tools correctly handle character encoding in HTTP headers?

https://tools.ietf.org/html/rfc8187#section-3.2.3

View on HN

I attacked it from both directions: an arguably supervised dataset of urls (hundreds of GB from various internet survey projects), and unit testing the insane markov gibberish the parser makes when you cram random data where it doesn't belong and run it in reverse. The dataset was way less helpful for a strictly conforming design, but I imagine it would be useful for a mostly correct parser.

Reply on news.ycombinator.com