Hacker News Reader

Comment by seiferteric | Hacker News Reader

Currently I am facing an issue where I am unmarshalling json into a protobuf struct. Unfortunately the json tag names are a bit different than the protobuf struct field names. I would love to be able to use struct tags here, but to my knowledge protobuf definitions don't allow this, and I don't think I can put them in after the fact. So I am stuck loading the data into an interface{} and manually pulling out the fields.

View on HN

One technique I use is to create a struct that mirrors the layout of the target struct, then just cast between them. So if you have a

  type Person struct {
    FirstName string `json:"first_name"`
  }

And you get json like `{"name": "Alice"}`, you can decode it something like this:

  var doc struct {
    FirstName string `json:"name"`
  }
  json.Unmarshal(data, &doc)
  person := Person(doc)

I have not looked at the assembly so I can't say whether this incurs extra overhead or not.

I thought of this, but I would have to do this for every single protobuf message. Also, anytime the protobuf changes, you would have to update the code.

Your protobuf messages are changing that much? My condolences.

ETA: Also, you already are needing to add code to every encode/decode point for the `map[string]interface{}` handling. :P

is this for the protoc generated code? otherwise, could you not define different struct tags for the different serialization types?

  type Person {
      Name string `protobuf:"name",json:"given_name"`
  }

Yes it is for the protoc generated code. It would be nice if the protobuf spec allowed for something like tags as well. The main difference is the json coming in uses hyphens, whereas protobuf<->json only allows for underscore or camelCase field names.

> Yes it is for the protoc generated code. It would be nice if the protobuf spec allowed for something like tags as well.

Have you seen Protobuf options [1]? They allow for a tag-like-behaviour, but are strongly typed. It's notably used by gogoproto [2] to allow specifying custom field types or names [3] when generating Go stubs.

[1] - https://developers.google.com/protocol-buffers/docs/proto#cu...

[2] - https://github.com/gogo/protobuf

[3] - https://godoc.org/github.com/gogo/protobuf/gogoproto

Thanks, I have looked at options. I think it could work, but I think it would require writing a custom marshal/unmarshal function for each protobuf struct, even if they are essentially the same for each one. I will have to take a closer look at the links though.

gogoproto worked for me, using the moretags extension, thanks for the links!

Code-gen with help of the 'template' library might be a way out of the code repetition.

If you use gogoproto, it includes an extension to the protobuf spec, to override json tag for the emitted Go. https://github.com/gogo/protobuf/blob/master/extensions.md

Use it like:

    import "github.com/gogo/protobuf/gogoproto/gogo.proto";

    ...

        Type myField = 1 [(gogoproto.jsontag) = "my-field"];

Reply on news.ycombinator.com