dureuill

You could maybe use something equivalent to the "index hot swap"[1] feature we have at the Meilisearch level at startup, so that you make the reindexing in a another index at startup, and then atomatically swap this fresh index with the old one when it is ready? That way, you have fast startup at the cost of having possibly out-of-date information for a while after startup.

(you could even reindex from scratch completely in the background at startup, so no need to discover deleted files at all)

[1]: https://blog.meilisearch.com/zero-downtime-index-deployment/

Hello from a Meilisearch team member,

wow your project looks very interesting. How do you handle things like the filesystem changing while your indexer is offline? Do you reindex from scratch at startup?

Regarding typeahead, is this what we call "query suggestions"[1]? At the moment, we think that this is something that frontends and SDK can provide rather than the engine, so that means you wouldn't find it at the Milli level. We think you could maybe build an ancillary suggestion index and make two queries instead of one when typing, so as to get both results and suggestions at once.

Here's a chat link[2] to our latest discussions on the topic; feel free to come and weigh in if you're interested!

[1]: https://roadmap.meilisearch.com/c/31-query-suggestions

[2]: https://discord.com/channels/1006923006964154428/10685073658...

I guess a raw pointer is good enough an approximation in that case, it is just annoying that suddenly you can no longer use an optional because your type is a reference

Other languages allow to return `Option<&T>`. That's what I'm using these days.

I suppose a raw ptr is what I would use on C++ to model that case though, yes.

I had more bugs pop up from people calling front() on an empty vector than from direct out of range bounds. I guess people are more cautious when they are manipulating indices than when calling a method that takes 0 arguments?

Debug asserts are nice, but not sufficient when the code path causing the empty vec is only encountered by a production user from a different country at 2am. Enabling the asserts in release mode will at least prevent undefined behaviour and malevolent users from harming the system, but the best interface is the one that returns an optional, so as to make the "vector was empty" case explicit. That way it has a higher chance to be considered by the authors of the code or their reviewers.

i suspect most of that time is link time. Possibly the linker in use is not very parallel, and so linking one big executable with cargo build takes longer than many smaller test executable whose linking can actually be made parallel?

Yes, security researchers got quite a lot of work exploiting the antiquated tools of C coders

[1]: https://www.opencve.io/cve?cwe=CWE-134

> For many languages, print is special and the implementation is provided as part of the language implementation.

what's the point of not having print provided as part of the language implementation, though?

^ this. there's a lot i like from Austral, but I think I disagree on some fundamental level with the "simplicity" goal. I simply don't think it is necessarily conductive to less errors, and expressivity of a language matters. There is a balance to be found.

For example, Rust-style iterators are "complex", yet they cause less errors than explicit indexing. Their combinators, that afford a lot of their expressivity, would cause very difficult to write types without type inference.

There's a similar tale of expressivity vs simplicity in the current implementation of the borrow checker in rust that is not using lexical (tied to a syntactic scope) lifetimes. For having used rust before non lexical lifetimes were a thing I can guarantee that from a user's point of view it is worth all the complexity in the rules and the implementation. I also have a hard time to imagine a serialization framework like serde without the ability to derive traits with an annotation.

Separating interface and implementation opens up to (compile time, fortunately) errors when they don't match, and brings no gain: the public interface can be generated by tools like cargo doc.

Ada style syntax I find difficult to read (I prefer braces) but this is purely a product of familiarity I think

The document also says the language does not feature unwinding panics. What does it has then?

The capability system is interesting, but I am surprised in the hello world example that printing to the console doesn't seem to require a dedicated capability.

I think an extension to this capability system would be some sort of tie-in with the OS so that a given process can start with only a subset of the root capability

> Exceptions actually do that, except hidden and unsignaled.

Which IMO is good in exactly one situation: when raising the exception means that the program contains a bug.

Using panics (ah sorry... exceptions) in this case is justified as it should be really exceptional (if there is a bug anyway we have more pressing problems than performance) and in the absence of bug if we were to use a Result type it would mean we would have a "BugError" variant that is actually dead code everywhere where the program is not buggy.

So in my opinion a correct approach is to unwrap whenever you have an invariant that guarantees that there should be a value, with a panic handler set at the boundary of the logical task to fail the entire logical task in case there is a bug. A logical task can be an asynchronous light task, a thread, or the whole process depending on the situation.

I much prefer it not being the whole process when the process is e.g. a web server or a word processor (and the failure occurred somewhere in an ancillary function)

> You wouldn't say C "has exceptions" would you?

Maybe that's just me, but I wish more people would say that C "has exceptions", if only because longjmp/setjmp[0] interacts with exception systems of other languages, sometimes badly.

That being said, I think an important difference is that in C setjmp/longjmp do not "unwind" objects, due to the lack of destructors.

However panics at least in Rust are just exceptions by another name. The difference solely comes from culture. Even the fact that unwind can be turned to abort is not significant, considering that pretty much all production implementations of C++ have fno-exceptions

[0]: https://learn.microsoft.com/en-us/cpp/cpp/using-setjmp-longj...

C++ jobs are not going anywhere, quite the contrary. A lot of systems won't be rewritten. Besides, at least at the moment "C++ expert" is a perfect background for becoming a Rust expert should the want arise.

Rust *is* much more enjoyable to program in than C++, though

how do you observe data you don't own if you never use a raw pointer or reference?

if you use shared ptr for this:

1. shared ptrs aren't made for this use case, they're for shared ownership 2. why using C++ at all if it means reducing its performance to an (atomatocally) performance counted language? Rust allows for much more performance with its safe borrowed references.

That also not what the CPP core guidelines, that are supposed to define modern C++, prescribe: For general use, take T* or T& arguments rather than smart pointers[0]. This rule incurs the risk of use after free if the returned reference is kept for too long by the caller. The rules for reference validity are non trivial (eg you returned a reference to an item of a vector, if you push into your vector you might invalidate the reference), so this is a significant source of bugs even in modern C++.

if you don't use shared ptr or references/pointers for this i'm curious as to which mechanism you're using to observe non owning data

[0]: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines...

The reality is that leveraging closed networks to gain effective monopolies (with all their known harmful effects) is a winning strategy with little counter strategies.

Users cannot switch to another OS because if they do they won't be able to communicate with social circle. This is also completely artificial because the networks have otherwise pretty much the same feature sets and are only distinguished by their accreted userbase.

Concurrent cannot compete because they can't gain enough users to get a critical mass

I wager that no single entity should have so many captive users.

Regulation is clearly in order

This is the only correct answer.

Although give it 10 years and that issue should be resolved.

this really speaks to the urgent need of making interoperability legally mandatory for the bigger actors so one isn't socially forced to use one or two applications that are provided by a private actor

This simplicity translates to afternoons of work lost to CMake.

cargo just works, and isn't really complex in design. just less anticated.

in case you don't know and it would be useful to you, there's tooling out there to generate license files from the Cargo.toml.

cargo lichking is an example, and I think cargo deny has similar functionality?

This runs contrary to my experience.

1. Coming from C++, my productivity is x2-x3 in Rust, making Rust a middle point between C++ and Python (about x8 productivity). What's more, if we factor maintenance time in, the lower costs of maintenance of Rust code makes the multiplier tend to x10, which is equal or better than Python (whose maintenance costs are important).

2. I have a colleague coming from Python (so a very different background than my C++ background), and he doesn't "get lost in the complexity of Rust" but after some use of Rust makes pretty much the same conclusions as I do: initial coding slower than Python, but roughly equal when you factor in maintenance time. He now writes the quick tools that could be Python scripts in the past in Rust when we suspect that they won't be one-off scripts (which happens very often). We get ease of distribution (static binaries), portability (to Linux and Windows), and better performance out of it too.

Although this is a comparison with C++ and Python, not C, the reasons why are simple and apply equally so to C:

1. Easy access to a good ecosystem. Adding dependencies in C or C++ is a pain. Very easy to do in Rust, preventing the need of reinventing the wheel (squarely). C suffers even more from this, given its lack of standard library and data structures (everything is a linked list :-D)

2. Memory safety and lack of UB in safe Rust brings a definitive simplicity in coding, code review and debug.

3. Result-oriented APIs and generally expressive type system are what end-up bridging the gap with Python with time.

What Rust definitely has is a learning curve. It is not optimized for taking the language without deep diving into it, or learning it in a short time. IMO it is a reasonable trade-off, given that the experience past the learning curve is so good, and that many of the things that make the learning curve so steep are integral to that experience (exclusive borrows, trait system, ...).

> Things like "kernel error reporting" doesn't exist as discrete element.

I'm not familiar with kernel development in general or Linux in particular. I would have expected there to be an error reporting subsystem, so that if a given subsystem fails the failure is reported to the error reporting subsystem (which hopefully exposes a more modern interface than serial cable), but this might be naive on my part.

> For all kernel knows, the only way to get something to the outside world might be through USB Ethernet adapter and connection that is tunneled by userspace TUN device, at which point essentialy whole kernel must continue to run

Again I'm missing context on this discussion. For all I know this could be an error originating with a driver, since rust support for Linux is for driver development now. It would make sense to me that an error in the GPU driver doesn't prevent the ethernet driver to report the bug

Glad to be of service. Note that the idiomatic error handling in rust is still Result based rather than panic/catch_unwind based.

Nevertheless a long living application like, e.g., a webserver will catch panics coming from its subtasks (e.g., its request handlers) via catch_unwind

But panics in rust are pretty much exceptions though?

The differences are they are actually meant to be used for exceptional situations ("assert violated => there's a bug in this program" or "out of memory, catastrophic runtime situation") and they are not typed (rather, the panic holds a type erased payload).

Other than that, it performs unwinding without UB, and is catchable[0]. I'm not seeing the technical difference?

[0]: https://doc.rust-lang.org/std/panic/fn.catch_unwind.html

Hmmm, the linked email is not providing a lot of context, so surely I'm missing something, but there's something I definitely don't understand: is there not a third option between stopping the whole kernel on an error or allowing an incorrect result?

Maybe my misunderstanding comes from my ignorance of the kernel's architecture, but surely there's a way to segregate operations in logical fallible tasks, so that a failure inside of a task aborts the task but doesn't put down the entire thing, and in particular not a sensitive part like kernel error reporting? Or are we talking about panics in this sensitive part?

Bubbling up errors in fallible tasks can be implemented using panic by unwrapping up to the fallible task's boundary.

To my understanding this is exactly what any modern OS does with user space processes?

I always have the hardest of time in discussions with people advocating for or against that "you should stop computations on an incorrect result". Which computations should you stop? Surely, we're not advocating for bursting the entire computer into flames. There has to be a boundary. So, my take is to start defining the boundaries, and yes, to stop computations up to these boundaries.

I don't know about Java, but in Python, 99% of the modules I create respect the 1:1 mapping between modules and filesystem.

Same in Rust, the overwhelming majority of modules I create is in the standard filesystem <-> modules mapping. For generated code, I use the special syntax that allows not respecting this mapping, but that's once in a blue moon.

IMO, C++ should have taken the same steps: providing sane, correct and easy defaults, while allowing the flexibility to override them when necessary (with special annotations).

I'm disappointed that a modern C++ feature was designed in the long tradition of having bad defaults instead.

modules were my #1 wanted feature in C++, but they are very disappointing to me:

- they allow `.` in their names to signify hierarchy, but have no built-in notion of hierarchy or submodule. This means possible error messages will have to be suboptimal

- they are orthogonal to namespaces, meaning if you have a module `foo.bar` any identifier it will export will be in the global namespace unless you put it in the `foo::bar` namespace in the `foo.bar` module file. As a user this means that I can import a `foo.bar` module and expect symbols to be imported in any namespace.

- btw, what happens if different modules import the same name? I'll let you guess:-D. Modules were the chance to eliminate this kind of issues by eliminating the notion of shared namespaces, but oh well...

- modules have interface units, implementation units, partitions, interface partition and implementation partition. There must be exactly one non partition interface unit, called the primary module interface. Instead of, you know, just having modules and submodules files like all modern module systems in existence.

- Modules add new and fun ways to get ill-formed, no diagnostic required programs. Such as not re-exporting all interface partitions.

I really hoped that modules were the chance of introducing a simpler compilation model for C++. I'm not pleased with the result.

Some references:

[0]: https://vector-of-bool.github.io/2019/01/27/modules-doa.html

[1]: https://vector-of-bool.github.io/2019/03/10/modules-1.html

As expected modules are relegated to some later time ("will be its own experiment")

The number of excessive keystrokes is largely irrelevant (as long as it isn't an order of magnitude too much), because typing the code on keyboard is generally far from the limiting factor for the speed of writing code.

Most of the time you spend thinking about what you should be typing.

So optimizing for the number of keystrokes is optimizing for the wrong metrics

Also, if you look at the stats for saw-related accidents, 70% of them are impossible when the safety mode of the new gear is enabled (which is the default way of handling it). This is even in professional companies that keep using the old gear due to inertia and invested a lot in "safe handling practices" for it.

It is a very unergonomic choice, though. Having warnings you can allow during development and deny in CI is the best of both worlds.

Otherwise it leads to absurd situations where some code gets commented to avoid errors caused by unused things, which defeats the point of the lint.

When coding in Rust, I extensively use warnings as a way to know what remains to be done before my code is done.

Treating all warnings as errors all the time so as to address people who would ignore warnings is overtailoring the language for beginners at the price of ergonomics for experienced programmers.

i could understand this in 1995, but in 2015? We have peer reviews, and CI. Let's deny warnings there, and allow them during development.