The fastest Go JSON parser I know of is https://github.com/buger/jsonparser. I've used that one in production quite successfully.
I don't see that in the comparative benchmarks for Gojay.
I don't see that in the comparative benchmarks for Gojay.
Look again, JasonParser is included in some, but not all, of the benchmarks. JasonParser [apparently] allocates no memory, which, to me, seems pretty compelling.
The lack of allocs for JsonParser was definitely a big win for my project. When processing at high rps (40k msg/sec), GC is near the top of all my performance profiles. Driving allocs down with Buger's parser was very helpful for getting the throughput up.
Wonder what can be done to realistically generate something approximating a real world benchmark to cover the throughput scenario.
Even with the GC overhead, I will still able to hit my performance goal of 40k msg/sec per AWS c3.4xl.
In a vacuum I would have picked Rust over Go for this. But Go is widely used at my work, whereas I'm the only Rust guy.
The memory access patterns are straight-forward enough that the remaining GC could definitely go away. If I had to make a wild, from the hip, and unsubstantiated guess, maybe a 2x to 5x performance multiple. That would include a comparable amount of time spent optimizing and would depend greatly on the quality of the Kafka crates for Rust (which I've never used).
you had me actually looking for something called JasonParser for a few moments. :P
But often that struct isn't your end goal in life, such as when your data is being transformed, dispatched on, or being moved into another yet data structure. In that case unmarshalling to an intermediate structure, while convenient, is also extra computational work.
Buger's parser isn't the first thing I'd reach for, but if JSON parsing performance is an issue, ObjectEach() can be used in many places where you'd otherwise invoke json.Unmarshal.