Comment by jchw | Hacker News Reader

Comment by jchw | Hacker News Reader

Parent Full thread

Kaitai Struct is the better way to go for an ecosystem solution, but the tooling could certainly use improvements. In addition to 010 Editor, there's also KDE's Okteta. It does not have a lot of good OSDs and the OSD format/scripting for specifying formats is a little anemic (I'd like to help improve it if I can find time...) but it's very serviceable and a decent open source alternative to what 010 has. Shameless plug, I made a decent Windows EXE/PE OSD for Okteta. (It's even got a bit of support for NE16 executables, just for fun.)

This entire genre of tools has been a long time point of interest for me. In addition to making a couple OSDs and contributing some tiny improvements to Kaitai, I also have my own binary schema library for Go, restruct, which, biased or not, remains my favorite way to poke at arbitrary formats, since it's really easy to sketch stuff out and read and write to files quickly. It's basically Go's encoding/binary but with struct tags for more advanced things.

> This entire genre of tools has been a long time point of interest for me.

Same for me! It turned into a long journey and I am working on a solution that I am very happy about.

Somewhere mid-journey I learned about kaitai struct and lost a bit of steam seeing it was similar. But I think my offering is superior in a more simple template format with less programming required and a nice cli app.

I am yet to announce it publicly, but i been meaning to for so long already.

If you would like to check it out I would be happy!

You can use it to map / view content from a format there is a template for. Alot of common formats is already included and you can extend it using your own templates.

https://github.com/martinlindhe/feng

As someone who had a great time with Kaitai, may I suggest that you write an interface so that fq can be used with any format that Kaitai understands (and any that people add in future)

The functionality that I am personally interested in from a binary parsing framework like Kaitai is generating an encoder implementation in addition to a decoder one. In other words, given a description of a binary format, I would like to be able to construct an instance of a class whose memory layout matches the format. For instance, if the format has an int n, then an array `a` of size `n`, and then a double `d`, it would be awesome to be able to construct a corresponding object with fields `n`, `a` and `d` and when I change `n`, then the size of `a` changes accordingly. And then, if I pass a pointer to this object to the decoder, it would be able to parse it correctly, as if the memory representation of the object came from some external buffer.

Kaitai support for serialization has been a long time issue. It's obviously non-trivial given that it has at least one case that doesn't exist today (instantiating a structure without loading any existing data.)

That said, it does exist in some form.

https://doc.kaitai.io/serialization.html

Restruct supports serialization. I use it all the time to hack on proprietary formats.

Reply on news.ycombinator.com