Hacker News Reader

Comment by jlarocco | Hacker News Reader

This uses std::string all over the place, which allocates memory under the hood.

I'm not sure how any JSON parser could avoid memory allocation without a difficult to use interface. Numbers, arrays, strings, and objects are unbounded by the JSON spec, so a truly malloc free library would need to provide a kind of streaming interfaces where things are returned in fixed sized chunks.

JSON wouldn't be my first choice for data storage in situations where I needed to avoid dynamic memory.

View on HN

The parser does not extensively use std::string. to parse json keys, It uses an internal stringview class mirroring the json string (string_view is candidate to get into the next C++ standard). When I said malloc free, I meant that no intermediate memory is allocated: actually, If your json object only contains int or floats, the parser will not use malloc: it will just traverse the string and fill the scalars of your destination object (thanks to static introspection), a big difference with state of the art json parsers returning an intermediate dynamically allocated tree that you have to interpret after the parsing. Of course, if the destination object stores instances of std::string of std::vector, the iod library will allocated them and fill them with the json values.

It also uses vector, which is almost certainly using malloc and/or placement new even if the SIO somehow really does reserve space for a single object. Its also using the heap unless there is some hidden Allocator object in the code that explicitly is using stack allocated space.

I wrote a JSON-Parser in C with minimal validation in ~250 LOC that does not allocate memory, but leaves the type check and conversion to the user. https://github.com/vurtun/json

That's a very nice code.

This is absolutely beautiful. I didn't know you could get an address of a label in C or to do a goto to an address from variable instead of a literal label. Is this new in C, or was it always there?

It's a GCC extension. https://gcc.gnu.org/onlinedocs/gcc/Labels-as-Values.html

Is there an equivalent for Clang/LLVM?

Yes.

http://blog.llvm.org/2010/01/address-of-label-and-indirect-b...

A venerable interpreter implementation trick.

Unfortunately it won't port to IAR, which is a shame. I don't understand why pointers to labels never made it to the C spec; it's one of the (equivaling) features that actually makes assembler look elegant.

That test.c file looks familiar!

Yeah I was looking for unit test macros and found your project. I added your project as a reference in the README for clarification of the origin.

It would be nice if the README mentioned it was originally based on my https://github.com/quartzjer/js0n :)

Added your project as a reference. Sorry for the missing reference I originally never planned to make the project known publicly and therefore never added the references.

I wrote a JSON parser that avoids using dynamic memory but works a bit differently. It's a callback interface like SAX, but instead of pushing already "typed" data to the user, the user calls a function to interpret the data. In this fashion, if the user wants a float instead of an integer, they call the function for float. Likewise, if the user has some space allocated for string values, the library can convert the already inmemory JSON format string to UTF-8 in the user's buffer.

The C interface is purposely ugly but the C++ is much easier to work with.

https://codehero.github.io/benejson/

> JSON wouldn't be my first choice for data storage in situations where I needed to avoid dynamic memory.

I work a lot with very low-resource embedded systems and we like to use JSON host-side for structured data configuration, but convert to BSON (http://bsonspec.org) for target-side storage. This can be parsed and iterated with very low-overhead and no dynamic memory. Similar idea to protocol buffers but with JSON-like data.

Avoiding a build-time conversion would be nice, though, and this idea is interesting.

sajson allocates once: it turns out you can structure your parse tree such that each byte of input only requires one machine word of output. http://chadaustin.me/2013/01/sajson-why-the-parse-tree-is-bi...

https://github.com/chadaustin/sajson

That's correct in the worst case, but in the best case, std::string does not allocate memory anymore (short string optimisation), e.g. libc++: http://stackoverflow.com/a/21710033

So it might be "malloc-free" for configuration files where the possible values are limited, etc. :)

jsmn is a nice JSON tokenizer. You malloc a chunk of memory up front, but that's it. It gives you pointers to the tokens and you can do what you like with them. I am using it in an app that has a fixed-size data structure that is populated from a JSON file. So my code is malloc-free also.

Link for those interested: https://bitbucket.org/zserge/jsmn/wiki/Home

jsmn is pretty awesome, and contrary to this (std::string, std::vector) actually puts you in charge of allocations. It's also in C.

In a similar vein, and also possibly already mentioned:

- yayl (C): https://lloyd.github.io/yajl/

- microjson (C): http://esr.ibiblio.org/?p=6262

I've got a streaming JSON parser in GCC C around here somewhere that does basically that - numbers and keys are truncated, but everything else is chopped up into chunks and streamed. Someday I ought to clean it up, document it and release it. Runs quite nicely on an Arduino.

Reply on news.ycombinator.com