Hacker News Reader

Comment by mdwrigh2 | Hacker News Reader

It's actually the JSON parsing. Python is using a highly optimized C parser, whereas Go's json package is in pure Go.

Ah thanks! So nothing to do with GC or poor IO but a poorly selected operation to do in a benchmark.

As it seems like is always the case in these fairly naive benchmarks. (Not specifically about Go here, it seems like people often cite python when they're really using some optimized C code).

>>poorly selected operation to do in a benchmark.

Considering the history of benchmarks, it is quite likely the opposite: The library was selected very carefully, to argue a "point". :-(

PyPy is using a pure-python JSON parser as well.

I am the author is there a better JSON parser that I should have used for Go?

I don't think so; I think Golang could use a more well-optimized parser implementation, even if it remains written in Go.

I appreciate why you use "golang" instead of the general "Go" or current release "Go1", but the former is the name of the open source organization, and the latter is the name of the language.

Fair. Then what's the name of the implementation? The same as the language? How do I refer to:

* Go, the specification

* gccgo

* 6g and friends: the compilers

Go, gccgo, and gc respectively.

You are mistaken. There is no organization that maintains the Go project. The project and language are both named Go.

"golang" is just the name of the domain and a more searchable term than "go." There is nothing more to it than that.

It certainly could be more optimized. For example, all numeric literals are converted to strings before being parsed. That's a mostly unnecessary copy/allocation per number. Similarly, it doesn't make use of the fact that a []T can only legally contain Ts. It's all very clean and generic, which is great, because when working on the compiler they can work on making good code faster without being mislead by code that tries to work around lack of optimizations. As an experiment, I replaced the []int in the Go code in the link with an IntList that implements json.Unmarshaler, and does a single-pass integer list parsing there (without overflow checking, but eh). With that quick 40 line change, the Go code was faster than the CPython, at least on my machine.

That and similar type-specific handling could be built into the json package fairly easily, and would make those cases quite a bit faster. Then again, I bet the Go team would have a better and more general idea.

I spun the JSON decoding off into a few goroutines and managed to get it down to ~1second (but it was out of order, so didn't quite solve the same problem - though this is fixable). This is was ~10 lines of Go (and only using a single core).

This was my bad I have updated the post. I should have been marsheling to a more comparable datatype (map[string]interface{}{}) and now the performance is more in line with what I would expect. I need to look into the Unmarshal code to see why this is so slow.

You could use one of the C json implementations such as json-c. However as has been pointed out you will still be just comparing libraries rather than the languages themselves.

Reply on news.ycombinator.com