Veles: Open-source tool for binary data analysis | Hacker News Reader

Veles: Open-source tool for binary data analysis

codisec.com

128 pointsLorenDB23 comments

The GitHub repo [1] was made a read-only archive on Sep 30, 2020. Are there forks picking up where the original author left off?

[1]: https://github.com/codilime/veles

See by yourself. There is only one fork wich fixed the msgpack encoding breakage: https://github.com/0xBattleSong/veles

Since this is an old tool and it fails to deal with large files, I made a completely new version from scratch a while ago, using a completely different rendering approach that's more like how you'd render a volumetric data set like an MRI scan. It loads the file and processes it into a 256x256x256 volumetric data set, which is then rendered using shaders. As a result, the file size doesn't matter for rendering, only the loading time depends on the file size. Unlike the original Veles, it also doesn't need any subsampling for huge files, but you need a powerful enough graphics card.

The source code is on github and unlike the original Veles, it doesn't have countless dependencies and build problems on modern systems: https://github.com/hackyourlife/veles

You should add some screenshots!

I added a few screenshots of different files right now.

Oh, I know something about this. Some people that made this are part of DragonSector [1] polish hacking group that broke DRMs in trains quite recently [2]. Sadly I joined company after there were long gone. Really surprised seeing this posted here as it was pointed out, the repo is pretty old and archived.

[1]: https://dragonsector.pl/

[2]: https://youtu.be/XrlrbfGZo2k

One person, IIRC, don't think anyone other than q3k was involved in CodiSec, and I think he left the company before proper development on Veles started (I think he was present when first talks about were happening and gave a lot of inspiration though, but isn't in credits for first open source version)

ImHex has some similar data pattern analysis tools. And as it is actively maintained, might be worthy to take a look at.

Not sure how new the tool is because it was new in 2017...https://news.ycombinator.com/item?id=15966021

There was a major push for development back in 2015-2016, back before it was open sourced, but AFAIK it went dead pretty soon as internal funding dried up.. Looking at github commit log, it seems it died by mid 2018.

First time I had to setup CI on a Mac, and what a horrible experience it was (Windows was comparatively easy, though finding out that starting with Windows Core wasn't best option took some time to learn)

Did you work for the parent company?

There was no "parent company" per se - codisec was a team inside CodiLime, starting out as an initiative to build more cybersecurity presence, including being the name we participated in various CTF contests. (This is all public info, you can find codisec at various CTF participation trackers too). And yes, I was there (worked at CodiLime late 2014 - early 2017), including in the early days of Veles (those who know my full name can easily find me in credits in the github repo, direct link with my account was lost when the repo was migrated into current one).

With regards to Veles, I mostly did infrastructure and tooling for CI/CD - so including the CMake stuff and making Jenkins setup for the builds. I think it was actually the first project on which I used Jenkinsfile, which was then very fresh thing. I was never much of a C++ fan so I didn't participate much in the editor code itself, especially with another project taking lion share of my attention later on.

> First time I had to setup CI on a Mac, and what a horrible experience it was

Could you share some details what was horrible?

I've created pipelines for Linux, macOS and Windows, and think that Windows is by far the worst of all three to setup anything related to CI.

One of the issues was making builds that worked on OS versions older than the build host, general annoyances of admining macOS, fragility of the environment (this was before image based upgrades)

The main page of codisec [0] lists that Veles has been retired. They since have a new project, CodiLime [1], which seems like is a general technology consulting/outsourcing company.

[0] https://codisec.com/

[1] https://codilime.com/

CodiLime was the actual company, CodiSec was "brand" for various security related activities, starting with CTF competitions where we used the name first, then it was used as brand for the team that worked on Veles (we continued to take part in CTFs)

None of the CodiSec members were among the management/founders/etc of CodiLime - we were AFAIK plain employees, though one of us was in fact mber of DragonSector which is partly how CodiSec started.

Neat, I was looking for such a tool a few days back and ended up not having the requisite google skills to find this (embarrassingly, it seems easy it find now)

I ended up writing my own script for visualizing my particular problem (trying to work out what's going on in the "mett" data frames in videos from my pixel phone, I was able to make some headway but I've gotten distracted by other projects, if anyone has info I'd appreciate it)

I was summoned? ... oh sorry!

I am honestly surprised this still shows up...

learnt so much CMakeFile pain...

Their linux build is in ubuntu 16.04 ...

Sorry, that was what was current LTS when I setup CI builds

When these tools come up, I like to post my little list of similar tools, for those interested. Apparently it's a hobby of mine:

* fq - like jq for binary data: https://github.com/wader/fq

* Kaitai Struct - https://kaitai.io/

* visualizer, for the above: https://github.com/kaitai-io/kaitai_struct_visualizer/

* HexFiend - a hex editor, but with "binary templates" feature : https://github.com/HexFiend/HexFiend

* binary templates, for the above: https://github.com/HexFiend/HexFiend/blob/master/templates/T...

* binspector - https://github.com/binspector/binspector