Hacker News Reader

Comment by mg | Hacker News Reader

Holy moly, my 3 year old post on the front page! The power of blogging. Cool.

Quick update: Yup, I am still doing this. I enhanced my tooling in VIM quite a bit, and logging is super fast now. Earlier today I entered log line number 30558.

I also wrote a bunch of software to visualize and analyze cause and effect in this type of log. Some of the many potential cause/effect relationships look promising.

Since the data in this type of free-form log does not fit into the typical statistical tests very well, I have no "significant" results yet. That's why I have not posted an update so far. I first want to improve my statistical abilities.

If anyone knows of established tools to analyze such a log, or has an interest in building new ones, get in touch!

View on HN

for unstructured message analysis -- it's not a silver bullet, but you may want to try BERT or another newish embedding tool

(you can do this on your laptop, you don't need to upload the data anywhere)

then use like sklearn OPTICS to cluster (clustering method that doesn't require you to know the number of clusters in advance)

If there are obvious clusters, you may be able to label a few useful categories. If you get lucky and most entries have a cluster, you can potentially manually label the outliers.

I think I saw an example where you said you use it to answer questions like 'How many curls did I do with these weights last time' [1]. That was from a couple of years ago, and I'm assuming you are the commenter there. If that's correct, do you have a way to make this system portable? I feel like the place I'm most likely to need to answer such questions would typically only have my phone available.

I greatly appreciate the data you're collecting and the overall method of doing so. However, I'm curious to learn about the portability of this information, and how you access it when you're away from a computer with vim.

[1]: https://lobste.rs/s/ri5utx/syntax_for_self_tracking

> If that's correct, do you have a way to make this system portable? I feel like the place I'm most likely to need to answer such questions would typically only have my phone available.

If your phone runs Android, you can just install Termux and then do everything exactly the same. Stick the files in a git repo, edit with vim. If the ergonomics bother you, I think it's possible to have termux open files in an android native text editor app, but I unironically read and edit my files in (neo?)vim and it works fine.

Reply on news.ycombinator.com