shepmaster

A solution is mentioned in the article, but perhaps obliquely:

> while I also wanted to measure hardware counters

As I understand it, hardware counters would remain consistent in the face of the normal noisy CI runner.

The article talks about using Cachegrind (via the iai crate) and Linux perf events.

I use iai in one of my projects to run performance diffs for each commit.

Leading a team is tiring work, so I hope Pietro has some time to recharge and then get back to the fun stuff.

Honestly, I’m not even sure why this has as many upvotes as it does: this is a pretty boring change, all things considered.

Not the person you are replying to, but the JSON performance of Emacs was a well-known thorn in the side for using LSP.

Further details...

... from the same source as the top post: https://www.masteringemacs.org/article/speed-up-emacs-libjan...

... from lsp-mode: https://emacs-lsp.github.io/lsp-mode/page/performance/#json-...

I've been seeing a lot of talk about htmx but I've been blissfully ignoring it as I've been reasonably successfully using Turbo and Hotwire. This move is a sign for me to take a deeper look into htmx as a potential replacement in the future.

Does anyone happen to have any good resources bookmarked for replacing Hotwire / Turbo with htmx? I'm also open to hearing about other similar tools in the area.

How would signing malware prevent that malware from being distributed? I feel like there’s something implicit you haven’t stated that I’m missing.

> and presumably there’s a backtrace option too

There is [0].

However, I'm not personally a fan of backtraces (or the lightweight location information, really). I'd much rather provide what I call a "semantic backtrace", which is where almost every Result-returning function is annotated by the programmer with a unique `.context(...)` call. This often results in a better error to the user ("couldn't connect to the server <because> couldn't get the URL <because> couldn't open the config file <because> couldn't open file ..."). The programmer can identify the location of the error due to the uniqueness of each context.

Backtraces are an easier way of getting similar information, but much less useful to the end user. I'd rather people prioritize the end user over the developer, so that's what I try to encourage in SNAFU.

That being said, plenty of people want location / backtrace information, so SNAFU tries to support those as well.

[0]: https://docs.rs/snafu/latest/snafu/struct.Backtrace.html

You will want to use `Location::caller` [0] instead of the `line` macro (and friends). You will need to annotate enough places with `#[track_caller]` so the location is correct.

[0]: https://doc.rust-lang.org/std/panic/struct.Location.html#met...

In SNAFU, you can add a Location [0] type to your error which will automatically capture that information when you add context to your error

[0] https://docs.rs/snafu/latest/snafu/struct.Location.html

I often want to sort within a line:

    [“zoo”, “cow”]

Would become

    [“cow”, “zoo”]

Any tips on how to do that? Ideally using just built in functions…

I usually insert newlines between the elements, use a line-based sort, then recombine the lines. That works but is more tedious than I’d like. I’ve never taken the time to wrap that logic it in a function because it’s infrequent and it seems like it’s always slightly different.

FWIW, I believe the person you are replying to means "paged" as in "an alert was sent to my pager at 3AM that the entire system is down and I need to wake up and fix it", not the paging in/out of memory.

> Which operates by immediately dumping the "errors" to terminal without any error types involved whatsoever

I haven't looked at your code, but if I take this sentence at face value, I'd be very hesitant to use your library in many contexts.

One example would be anything similar to a web server, such as an API that accepts JSON via a HTTP POST. It would be very strange to have my JSON parser print to the console where no one is reading!

A lot of your comment indicates similar focus on a human interacting with a terminal, which is a very valid usecase, but not the only usecase.

> instead of trying to structure data to leave the task of actually reporting the error to someone else, and kicking the can down the road.

I agree with this in broad strokes. I think that it's very important to use your error types to ensure that they provide value. For the JSON example, I think that I might have an error that indicates the specific error as well as a byte offset that the error occurred. This could be built on by higher level errors that convert to line/column numbers or attach an excerpt of the bad input, as appropriate. These types could have methods or implement traits that allow formatting for the console (considering optional coloring, etc.) or be formatted for a logging system.

> an unrecoverable bug should perhaps instead log and then terminate

I technically agree, but I have a very high bar for when a library I write is allowed to unilaterally terminate the process — it's simply too drastic of a decision to make in most cases.

I also hope that the act of logging is appropriately abstracted. Thankfully, Rust only has a few common logging interfaces so it's easy to fall into the pit of success there.

> Why expose it

I read "expose" in two possible ways: (1) as a public- / user-facing API (2) existing at all.

When an error type is a public API that is bound by semver, I do think you should be very careful about what you expose. I suggest starting with an opaque [1] error type and only exposing exactly what you are comfortable with supporting. That may boil down to basically only a string (a.k.a. the `Error` trait).

> Why expose it as a programmatically-accessible information at all?

A few things come to mind...

I'm a huge fan in testing my error cases as much as possible. To that end, a bunch of my tests are semantically `assert_matches!(Err(MyError::Case { .. }), function_result_value)`.

Not all errors are equivalent. For example, if my library fails to read a configuration file, perhaps the caller of the library can recover from that by downloading a file. However, this requires the caller to be able to tell what caused the error.

Interacting with an external API, such as HTTP status or command line exit codes. In this case, you can categorize your errors into domains like "server error" / "client error" / "authorization error".

These could all be done by string matching, but that tends to be comparatively brittle.

> the same information in a string

Aggregating strings like that requires dynamic allocation, which isn't universally available in Rust programs. For example, SNAFU works fine in a `no_std` environment and I know that it's been used in cases like embedded and Windows kernel drivers.

> if the only purpose is for a developer to read it

I don't think that's always true.

[1]: https://docs.rs/snafu/latest/snafu/guide/opaque/index.html

> as opposed to the article being discussed.

I'd actually say that the article and my suggestion are basically compatible. If you check out my gist [1], you'll see that I have basically the same number of user-facing types (or maybe a slightly smaller number as I merged a few structs and enums in some spots).

[1]: https://gist.github.com/shepmaster/fb7f4c9519a074ea7186ca7b7...

That's an interesting question that I find difficult to answer. To me, it's as if you are asking "why provide any detail about what happened?". The nonsense end result of that line of logic is that you return a single boolean that corresponds to failure/success, so I assume that's not what you mean.

Contrast the two error messages:

> one end of range is not a valid hexadecimal integer

> the beginning of the range ("0xZZ") is not a valid hexadecimal integer

The latter has more information and points the user to the general area of the problem, and probably allows them to fix the problem themselves. It could be even more improved by providing line/column information.

By splitting out more and more error types (and cases), and artificially limiting yourself to using each error once, you push yourself away from "range is bad" style errors and more specific (and ideally descriptive) errors.

I encourage people to check out my SNAFU crate [1]. I urge users to create many distinct error types (usually enums but also structs) and compose them — so much so that I advocate that you never create each specific error in more than one source location. That means that the collection of error types produces a unique trace through your program, uniquely identifying the source of the error with no runtime cost (compare this to a runtime-collected stacktrace / backtrace).

Here's how the final example in the blog post would look like with a quick transform to SNAFU: https://gist.github.com/shepmaster/fb7f4c9519a074ea7186ca7b7.... If I spent more time, there'd probably be a bit more changes, but hopefully that gets the idea across.

[1]: https://docs.rs/snafu/

Pedantically, `yield` is already a keyword [1]. As the project's README states, generators [2] are an unstable part of Rust, and the `yield` keyword goes with them.

[1]: https://doc.rust-lang.org/reference/keywords.html?highlight=...

[2]: https://doc.rust-lang.org/std/ops/trait.Generator.html

> So mostly people writing high performance code

I would have assumed those people would be deliberately using another allocator with known/documented/guaranteed qualities instead of whatever ambient allocator was in play at the time.

That or avoid allocation as much as possible so that the choice of allocator effectively matters very little.

When would using optimizing for specifically glibc’s allocator be maximally beneficial?

> Rust had a chance to fix this, but also dropped the ball.

By default, a Rust project will panic on integer overflow in debug builds and will overflow on release builds. Two key points to note, however:

1. You can change the setting so that your project panics in release or overflows in debug mode.

2. We reserved the right to change the default at some point in the future. This will probably be widely communicated before it ever happens, and last I heard we are still waiting for the cost of performing those checks to be "reasonable" before thinking about making such a change.

If you enjoy reading Rust code, you can see some examples I have...

1. This example [1] compares a SIMD-accelerated implementation of an algorithm vs the naive implementation. This is usually referred to as an "oracle".

2. This example [2] tests an XML document object model library. The tests construct a sequence of operations to apply to a document ("add an element", "delete an element", "move an element" etc) and then assert properties that you expect to be true for a DOM tree (a parent and child are always cross-linked, for example)

[1]: https://github.com/shepmaster/jetscii/blob/8d7e44ad7da990ef1...

[2]: https://github.com/shepmaster/sxd/pull/21/files#diff-fc21cbf...

Thanks! This page is familiar so I've definitely read it before, but didn't recall it when remembering examples. I like that it links to the site I provided; maybe I should try to search for other articles that also link there to find more inspiration.

Thanks; I've added that to my Christmas gift ideas!

Most posts about property-based testing focus on convincing the reader to do property-based tests and the mechanics of writing the tests.

Unfortunately, I'm already convinced and know how to write the tests — what I wish I could find is some resource that would hone my ability to identify worthwhile properties. The best one I know of and go back to time and time again is https://fsharpforfunandprofit.com/posts/property-based-testi..., but I really wish I could find more.

I like using https://altsysrq.github.io/proptest-book/intro.html for those very reasons. https://github.com/BurntSushi/quickcheck came first, but I prefer proptest’s implementation more than quickcheck’s type-based one

> public errors shouldn't have From's for concrete types

> enum errors are more generally too tied to implementation details to be used in libraries

I generally agree. SNAFU addresses these problems in two ways:

1. The `From` implementation is not created for the underlying error but for an intermediate type (by default). That type is private to the crate (by default) and cannot expose implementation details.

2. There's an opaque error facility to completely hide the enum details.

Put together, that looks something like...

    use snafu::prelude::*;
    use std::{
        fs,
        path::{Path, PathBuf},
    };
    
    #[derive(Debug, Snafu)]
    enum ErrorImpl {
        #[snafu(display("Could not read the config file {}", path.display()))]
        UnableToReadConfig {
            source: std::io::Error,
            path: PathBuf,
        },
    
        #[snafu(display("Could not write the config file {}", path.display()))]
        UnableToWriteConfig {
            source: std::io::Error,
            path: PathBuf,
        },
    }
    
    #[derive(Debug, Snafu)]
    pub struct Error(ErrorImpl);
    
    pub type Result<T, E = Error> = std::result::Result<T, E>;
    
    pub fn do_stuff_with_config(path: &Path) -> Result<()> {
        let config = fs::read_to_string(path).context(UnableToReadConfigSnafu { path })?;
        fs::write(path, config).context(UnableToWriteConfigSnafu { path })?;
        Ok(())
    }

Other things about SNAFU:

- It's very easy to add valuable context to the errors. See how the `&Path` context is transformed to a `PathBuf` with low ceremony in the example.

- You can create struct- or enum-based errors.

- You can use "stringly-typed" errors (akin to anyhow) but in combination with strongly-typed errors. This allows you to start out with a loose error handling regimen and make it stronger as you go along.

- There's support for capturing backtraces or lightweight file/line/column information.

- There's a pretty error reporter for usage with `main` functions or tests.

- There's support for the nightly-only Provider API.

> Something we are discussing is changing `defaut-features = false`

I’d be generally in favor of such a change as this is what I use for all of my projects when I start out. I find it much more comfortable to deliberately opt in to things that I need as I find that I need them.

The biggest pain point I’ve experienced is for features that are performance based when I am writing an application instead of a library. It’s hard to know that those exist, and that I probably do want them, I just don’t require them.

Thank you, but I don’t see any references to “rust” on that page, so I’m not clear on how it serves as a comparison specifically to Rust.

I am familiar with Rust; is there a good document that compares Val versus Rust?

Skimming through this article, I see a lot of similarities between the two but would be interested in knowing more.

Is likely based on what you compare it to. For example, this passes as well

    let array = core::array::from_fn(|i| i);
    assert_eq!(array, [0, 1, 2, 3, 4, 5]);

The magic of type inference

> This is not as good as having an official standard (say, in Cargo.toml)

It’s present now: https://doc.rust-lang.org/cargo/reference/manifest.html#the-...

Wouldn’t you want to use static instead of const, otherwise you might get a new unique mutex at each usage? I think there’s a Clippy lint for that.

The playground rebuilds overnight (roughly after the nightly is released); releases don’t have as fixed timing so I tend to trigger a rebuild when I notice. This HN post was delivered before the git tag was published even, so I’ve started it.

Playground is updated now.