Hacker News Reader

Comment by eightnoteight | Hacker News Reader

exactly, a typical REST server spends a lot of compute time in json encoding and decoding. which could be sped up significantly if there is better memory bandwidth

View on HN

My first guess would've been that JSON encode/decode wouldn't be memory bandwith-bound because it involves so many operations on single bytes at a time, most of which would be in the cache after the page has been brought in initially. Of course maybe much of that is done with SIMD these days?

Anyway, I have near-zero experience in this area so I'm mostly just posting this hoping for someone to explain why I'm wrong.

See https://github.com/simdjson/simdjson

I don't see how the link clarifies things. simdjson is fast enough that JSON decoding can approach single-core bandwidth capacity, but it doesn't follow that a JSON-heavy load is bandwitdh constrained. In fact the existence of simdjson points to json decoding being compute bound.

It’s a bit more nuanced than that. Scalar code that does lots of branching and loads one value at a time (e.g. single character) often can’t hit memory at peak bandwidth rate. On the other hand well vectorized code that loads vectors of multiple values at a time can properly saturate the available hardware resources.

The earlier post was referring to "a typical REST server" in a context of the claim that memory bandwidth "is becoming the ultimate spec".

simdjson (impressive as it may be) existing as an interesting project seems likely to imply that most JSON parsing isn't done with such performant methods on "the typical REST server".

EDIT: Seeing that it's used in many projects including Node.js shifts me back to thinking more highly of the claim that memory bandwidth is becoming the ultimate spec!

Reply on news.ycombinator.com