Hacker News Reader

Comment by craigching | Hacker News Reader

No, "hits.total" is the total number of documents that matched, regardless of what you specified as the number of hits to return.

View on HN

Right, but if you set the size to 10, then you either return 10 or (hits.total - 10) documents. If hits.total is greater than 10, you know you returned 10 documents; otherwise, you returned hits.total documents.

Ok, yeah, I see what you're saying and that makes sense. But, even my response is premature because you generally have to parse the whole response to get "hits.total" anyway. So unless you have written a partial parser (does an open source one exist?) you are going to be parsing the whole document to get even "hits.total".

Based on the bug filed[1], "Return number of documents in the result counts as HTTP header or in JSON," the implication is that including the number of documents in the result in the JSON would be an acceptable solution. If that's true, then they're OK with parsing the JSON.

Although, to be fair, the related issue[2] specifically asks for an HTTP header.

JSON parsing is generally pretty fast, but I can understand it being unnecessary overhead (although in general, there are other more significant opportunities for optimization).

If you've ever implemented a JSON stream reader/writer, then you've done partial JSON parsing. Check out the tool `jq`[2], which will parse partial JSON surprisingly well and fast (and this is just a utility).

[1] https://github.com/elastic/elasticsearch/issues/18312

[2] https://github.com/elastic/elasticsearch/issues/16993

[3] https://stedolan.github.io/jq/

Reply on news.ycombinator.com