tstack

> why there are so many colors in the example?

Identifiers, like process-names/pids/IPs, are semantically highlighted by default to make it easier to visually match up values that are the same. And, it's this way because I like it this way, so that's the default.

> For example the http 403/404 is red but it does not stand out.

To me, the red stands out. But, I would also have a much wider window, so the red would stand out even more with the rest of the text.

> Also why a 404 would be red in the first place.

Because 4XX are error codes.

Of course, lnav is pretty customizable at this point, so you can adjust it to your liking. The theme can be changed to something less colorful and the log format definitions can be patched to change their behavior.

As mentioned in a sibling comment, https://goaccess.io is probably best for simple analytics on log files.

If you want to write more complicated queries, lnav exposes log data through SQLite vtables[1]. So, you can do a SQL query and get a simple bar chart visualization.

[1] - https://docs.lnav.org/en/latest/sqlext.html#sqlite-interface

I'm the author of lnav and had never heard of logviewplus before, thanks for mentioning it. It looks like we've independently arrived at a very similar feature set, that's pretty neat. I will now shamelessly steal some of your ideas :)

lnav does support JSON-lines and logfmt logs. For JSON-lines, it will pretty-print the log messages to make them human readable.

For logfmt, I seem to remember the spec not being very clear on quoting semantics (maybe I'm wrong). Anyhow, I would suggest using JSON since it has pretty broad support at this point.

The data consumed from stdin is stored in a file to support scrollback.

> something to probably add to my clean up script.

These capture files are deleted if they are older than a day the next time lnav is run.

(I realize they should've been stored in ~/.local/state instead of ~/.config ...)

> One very useful feature for this I didn't see listed for lnav was to support adjusting for timestamps in different timezones, like maybe you have a desktop's log in local time that you're trying to reconcile with servers. (A related but less-important bonus would be to be able to specify smaller arbitrary clock drift/offsets for each log.)

You can adjust the timestamps for a file using the ":adjust-log-time" command (https://docs.lnav.org/en/latest/commands.html#adjust-log-tim...). For example, if you wanted to shift the timestamps back an hour, you can do:

    :adjust-log-time -1h

The shift is applied to the messages in the file that the top line in the view belongs to.

If you want to do the shift programmatically, you can UPDATE the "time_offset" column of the "lnav_file" table directly (see https://docs.lnav.org/en/latest/sqltab.html#lnav-file)

Yes, my wife and daughter really loved The Mystic Maze puzzle and the "secret" was a real treat.

Sorry for the crashes :( I've been trying to improve it's internals more than adding features as of late. If you haven't already, please file bugs on github and/or submit crash logs to the mailing list. (I've taken a break from working on it lately. So, if you've done that and I haven't gotten back, I apologize.)

lnav has some support for tailing files on remote hosts -- https://lnav.org/2021/05/03/tailing-remote-files.html

It works by uploading a stub that is an "actually portable executable" to the remote over ssh. lnav then talks to the stub to sync files back to the local host. I don't know the scale of the hosts or files, but if they're not too big, it might work fine.

I don't think multitail really understands logs like lnav does, it's just following the last lines in the file. For example, if you try to follow multiple files in multitail like so:

    $ multitail /var/log/install.log -I /var/log/system.log

You get a view with the tail from one file followed by the tail from the other, they are not collated by timestamp. In contrast, if you do the same thing in lnav:

    $ lnav /var/log/install.log /var/log/system.log

You will get a single view with all of the log messages from both files and they're sorted by their timestamps. Here is all of what lnav is doing:

  * Monitoring files/directories for updates
  * Decompressed files/archives
  * Detected the log format for each file
  * Created SQLite vtables that provide access to log messages
  * Built an index of all the log messages in all the files so
    you can jump to a certain point in time or to the
    next/previous error message.
  * Display all log messages with syntax highlighting

Rendering markdown in the terminal[1] was also recently added to the Logfile Navigator (https://lnav.org). It was partially inspired by Glow. There are some extra features in lnav's implementation, like a breadcrumb bar so that you can jump to different sections of the document.

(You might question why a log file viewer needs markdown support... It's used mainly to make the builtin help text look nicer. But, it can also be used in other places, like adding comments to log messages and generating reports. It's even used to implement an interactive tutorial[2].)

[1] - https://lnav.org/2022/08/06/markdown-support.html [2] - https://lnav.org/2022/09/01/playground.html

> Of course in a perfect world you want to ingest these logs into some better searching tool that lets you run SQL or some other standard query language over the set to prune them by date and pull out specific fields but there are times when you'll be hanging on by your nails with nothing to help diagnose the issue except an SSH connection to the server and the log file open in vi.

That tool is the Logfile Navigator (https://lnav.org). It's a TUI that reads plaintext logs, JSON-lines logs, and others. It will collate messages from all the files into a single log view that you can filter and search. The logs are also plumbed through SQLite virtual tables, so you can do fancy queries.

When setting up ELK or paying for Splunk/Sumologic is too much, lnav is a pretty good alternative.

Oh, okay... https://lnav.org is a log file viewer for the terminal that integrates with SQLite so you can use SQL to query your log files.

No benchmarks, though.

Another option would be to create the virtual table used for the table-valued function using a schema to define the columns:

    CREATE VIRTUAL TABLE students_json_read USING lines_json_read(schema="students-schema.json);

Then, that one table could be used for multiple files:

    SELECT * FROM students_json_read('school1-students.json');
    SELECT * FROM students_json_read('school2-students.json');

I work on a TUI logfile viewer that uses SQLite as a backend for doing analysis on the log messages (https://lnav.org). However, writing SQL interactively is painful since you can't really provide good auto-complete or preview, which is something I try to provide for most other operations.

The PRQL pipeline syntax would make for a much better experience for lnav since you're able to progressively refine a query without having to jump around. (You've probably noticed that many log services, like Sumologic, already provide a pipeline-style syntax instead of something SQL-like.) The nice thing is that you can simply keep typing to get the results you want and get a preview at each stage. For example, entering "from" and then pressing <TAB> would make it clear to the program that table-names should be suggested. The program could then show the first few lines of the table. Typing "from syslog_log | filter " and then pressing <TAB> would make it clear that columns from the syslog_log table should be suggested (along with some other expression stuff). And, then, the preview of the filtered output could be shown.

In the current implementation, pressing <TAB> just suggests every possible thing in the universe, whether it's appropriate or not. This leaves the poor with not much help after they've typed "SELECT". I find myself having to lookup docs/source to figure out column names or whatever and I wrote the darn thing. Ultimately, I think the analysis functionality just doesn't get used because interactively writing SQL is so user-hostile. So, I'm looking forward to seeing this succeed so that I can integrate it and still be able to use SQLite in the backend.

I used to use jasmin to generate invalid code to test an old JVM. I assume this has the same relaxed attitude that will let you generate invalid code?

The Logfile Navigator is a TUI for viewing/analyzing log files and will pretty-print anything with a keypress, see https://lnav.org/features#pretty-print-view

It's not specific to JSON/XML either, it works on any text that follows the usual conventions for structuring data in a textual way.

You can filter by level as well, but, usually I just press e/E to jump to the next/previous error from the current location.

There are, of course, many other features as well.

The Logfile Navigator (https://lnav.org) is a TUI for viewing log files that can be configured to process structured JSON files. It has quite a few features, like filtering and color coding.

The documentation for writing a format file is here -- https://docs.lnav.org/en/latest/formats.html

The format file tells lnav what common fields are in the log messages and how to format them into plaintext for display. Here's an example configuration for JSON logs -- https://github.com/tstack/fullsail_lnav_config/blob/master/f...

Sorry for the crashes, have you sent the crash logs to support@lnav.org or opened an issue on https://github.com/tstack/lnav/issues ? I try to take a look at crash reports, but I’m not always able to figure out the issue from just the logs. If you have some time to spare in replicating the problem, I can take a deeper look.

The Logfile Navigator (https://lnav.org), a TUI for viewing log files, has support for opening logs on remote hosts via SSH in its latest version. See the following post for more information: https://lnav.org/2021/05/03/tailing-remote-files.html

As for integrating with a “genders” host DB, there’s no direct support for it. But, lnav is scriptable, so I’m pretty sure it’s possible to write a script that does what you want. I can help with that if post in the github discussion: https://github.com/tstack/lnav/discussions

I'm asking about end-user documentation, like a manual. Not, developer documentation. Sorry if that wasn't more clear.

Can you elaborate on why you don't think it's a tailing tool? lnav monitors the files it was pointed at and will load new data as it is appended or reload the file if it has been overwritten. Also, if you are scrolled all the way to the bottom of the view, it should automatically scroll the view to keep the latest messages in the view.

Yes, one of the advantage of language support is that "sOpen" will be consumed by the "close()" call and can't be reused. I don't think that's really possible in Java without an extra checker. But, it's usually not a problem in practice if the user sticks to using the API in a fluent manner.

Try out the Logfile Navigator (http://lnav.org) for working with logs instead of a text editor. It can provide a lot of benefits by default, like collating messages from multiple files together, syntax highlighting, decompressing files and so on.

(Sorry this is a little off topic, but there’s a lot of room for improvement for working with log files over a text editor without scaling up to a log service)

If you're open to new tools, the latest version of the Logfile Navigator (lnav) supports remote hosts over ssh:

https://lnav.org/2021/05/03/tailing-remote-files.html

The main advantages of lnav over multitail is that it collates log files by time and does syntax highlighting. There are many other features as well -- https://lnav.org/features

The Logfile Navigator (https://lnav.org) is a terminal-based tool for digging through logs. It's visualization capabilities are pretty limited at the moment (it can draw stacked bar charts for the results of SQLite queries). It would be interesting to know what other types of visualization you would be interested in.

Angle Grinder supports analyzing a stream of data and updating the results on-the-fly. That's not possible in lnav since it uses sqlite for analysis.

Ideally, it would be nice to integrate Angle Grinder into the lnav to support both modes. I also think the the angle-grinder/sumologic can be a bit friendlier than SQL in some cases.

The Logfile Navigator (https://lnav.org) supports glob patterns and does what you want by default:

https://docs.lnav.org/en/latest/usage.html#viewing-files

So, you would just run it like so:

  $ lnav 'mylog.*'

I'll add this timestamp to the set of defaults for the final v0.10.0 version later today.

The documentation for the format files is here:

https://docs.lnav.org/en/latest/formats.html

The "timestamp-format" property in the format definition is an array of the formats to be tried beyond the builtin set.

Here are some example formats written by a user:

https://github.com/aspiers/lnav-formats

I think some of them define their own timestamp format.

Timezones are not automatically handled by lnav at the moment, which isn't great. Timestamps are parsed and treated as UTC internally, which usually works out fine. You can manually adjust all of the timestamps for a file using the ":adjust-log-time" command:

https://docs.lnav.org/en/latest/commands.html#adjust-log-tim...

That can be useful for doing minor alignments if clocks are slightly out of sync or doing timezone adjustments.

You can also adjust the time programmatically by updating the "time_offset" column in the "lnav_file" table:

https://docs.lnav.org/en/latest/sqltab.html#lnav-file

Sorry I don't have any examples written up to illustrate this at the moment...