HNHacker News
TopNewBestAskShowJobs

newpavlov

1,056 karma · joined January 13, 2021

submissionscomments
newpavlov··on Why choose async/await over threads?
Firstly, Rust does not have async Drop and it's unlikely to be added in the foreseeable future. Secondly, cancellation signals is a more general technique than async Drop, i.e. you can implement the latter on top of the former, but not the other way around. For example, with async Drop you can not ignore cancellation event (unless you copy code of your whole task into Drop impl). Some may say that it's a good thing, but it's just an obvious example of cancellation signals being more powerful than hypothetical async Drop.

As for implementation difficulties, I don't think so. For async Drop you need to mess with some fundamental parts of the Rust language (since Futures are "just types"), while fiber-based concurrency, in a certain sense, is transparent for compiler and implementation complexity is moved to executors.

If you are asking about how it would look in user code, then, yes, they would be somewhat similar. With cancellation signals you would call something like `let res = task_handle.cancell_join();`, while with async Drop you would use `drop(task_future)`. Note that the former also allows to get result from a cancelled task, another example of greater flexibility.

newpavlov··on Why choose async/await over threads?
With fibers you send cancellation signal to a task, then on next IO operation (or more generally yield) it will get cancellation error code with ability to get true result of IO operation, if there is any. Note that it does not mean that the task will sleep until the IO operation gets completed, cancellation signal causes any ongoing IO to "complete" immediately if it's possible (e.g. IIRC disk IO can not be cancelled).

It then becomes responsibility of the task to handle this signal. It may either finish immediately (e.g. by bubbling the "cancellation" error), finish some critical section before that and do some cleanup IO, or it may even outright ignore the signal.

With futures you just drop the task's future (i.e. its persistent stack) maybe with some synchronous cleanup and that's it, you don't give the task a chance to say a word in its cancellation. Hypothetical async Drop could help here (though you would have to rely on async drop guards extensively instead of processing "cancellation errors"), but adding it to Rust is far from easy and AFAIK there are certain fundamental issues with it.

With io-uring sending cancellation signals is quite straightforward (though you need to account for different possibilities, such as task being currently executed on a separate executor thread, or its CQE being already in completion queue), but with epoll, unfortunately, it's... less pleasant.

newpavlov··on Why choose async/await over threads?
I only scrolled the video, but it sounds similar, yes. Though, implementation details would probably vary significantly, since Rust is a lower-level language.
newpavlov··on Why choose async/await over threads?
I think a better question is "why choose async/await over fibers?". Yes, I know that Rust had green threads in the pre-1.0 days and it was intentionally removed, but there are different approaches for implementing fiber-based concurrency, including those which do not require a fat runtime built-in into the language.

If I understand the article correctly, it mostly lauds the ability to drop futures at any moment. Yes, you can not do a similar thing with threads for obvious reasons (well, technically, you can, but it's extremely unsafe). But this ability comes at a HUGE cost. Not only you can not use stack-based arrays with completion-based executors like io-uring and execute sub-tasks on different executor threads, but it also introduces certain subtle footguns and reliability issues (e.g. see [0]), which become very unpleasant surprises after writing sync Rust.

My opinion is that cancellation of tasks fundamentally should be cooperative and uncooperative cancellation is more of a misfeature, which is convenient at the surface level, but has deep issues underneath.

Also, praising composability of async/await sounds... strange. Its viral nature makes it anything but composable (with the current version of Rust without a proper effect system). For example, try to use async closure with map methods from std. What about using the standard io::Read/Write traits?

[0]: https://smallcultfollowing.com/babysteps/blog/2022/06/13/asy...

newpavlov··on Avoiding register spills in vectorized code with many constants
Compilers also may even spill data to stack from memory, even when the original location is still available, as can be seen in this issue: https://github.com/llvm/llvm-project/issues/53348

I vaguely remember that it could allow high-end CPUs to use something similar to register renaming, i.e. stack locations like [rsp + 96] could stay in physical registers during code execution (high-end CPUs often have more physical registers, than logical ones), but I couldn't find a good reference confirming that such optimization is indeed used in practice.

Unfortunately, I think more often than note it causes performance regressions and in some cases it may even cause unnecessary stack spilling of sensitive data: https://github.com/rust-lang/rust/issues/88930#issuecomment-...

newpavlov··on epoll: The API that powers the modern internet (2022)
epoll is a horrible API, which has grown from the even worse select and poll API. I believe its entrenchment, despite existence of APIs like IOCP, is one of the failings of Linux, which even has negatively influenced programming languages (I am looking at you Rust). Others have already posted the "fundamentally broken" article, but to add an anecdote, after closely working with io-uring, I simply can not go back to the horrors of epoll and I wish we, as an industry, will migrate from it as soon as possible.
newpavlov··on The Marvin Attack
The rsa crate is in the process of switching to https://docs.rs/crypto-bigint, which should more or less fix this vulnerability.

You can read more about it here: https://rustcrypto.zulipchat.com/#narrow/stream/260047-RSA/t...

newpavlov··on Thread-per-core
I believe that having the Send bound as a requirement to allow migration of tasks between executor threads is a clear deficiency of the Rust async system by itself, together with the fundamental issues around async Drop, which prevent implementation of scoped APIs. Similarly to threads it should be sufficient to have Send bound on functions like spawning and sending data through channels. The share-nothing approach is usually used as nothing more than a workaround to hide this deficiency..

Selectively pinning tasks to a thread/core has advantages and can be really useful in some circumstances, but it's a finer discussion, which has little to do with async users dissatisfaction related to Send.

newpavlov··on Safety and Soundness in Rust
Practically everything will be marked by it. Using `Vec`? Marked. Reading file? Marked. Printing something to stdout? Marked.

The idea behind Rust's unsafe is to cover unsafe bits in safe API which (assuming correctness of the unsafe code) can not be misused to cause memory issues in safe Rust.

newpavlov··on I use C when I believe in memory safety
>This means that I get many of the advantages of Rust

Arguably, you don't. The big difference is whether your compiler enforces and tracks your documentation, assumptions used by your code, often critically important for correctness. Unfortunately, Rust is far from being perfect in this regard (e.g. SPARK probably does a better job here), but it significantly improves on C/C++ in this regard.

>I Work Alone

Unfortunately, even if you are the only soul touching the code, it often quickly becomes false. On a big enough project, "alone" inevitably becomes "me and that guy who wrote this part of the code". This also can be formulated as "me and 2-years-ago-me". In other words, we quickly forget a lot of context used for writing a piece of code.

>Rust is too big for my small brain, unfortunately.

I think the lack of hubris, the understanding that small human brains are not enough to keep track of all necessary context required for writing correct software is the key element for liking Rust and similar languages. Rust is not about building ivory towers of abstractions for the sake of it (though some people try to walk in this direction), but about offloading as much as possible of the context on compiler. Unlike humans, the compiler does not get tired, it does not forget. Yes, it's limited and because of those limitations it requires to restructure our code and sprinkle it with additional information, but more often than note it's a good thing.

As sel4 and other formally verified software demonstrates, it absolutely can be done in C. But one could say that C used in sel4 is not a C language anymore. After all, you can not take a random C library and use it in sel4.

newpavlov··on Why does my SSH private key still work after changing some bytes? (2016)
Try to run a non-toy RSA by hand using pen and paper, it will be quite painful and impractical. Practicality of RSA largely depends on existence of computers. Even in the presence of all required theoretical knowledge, an application may not be practical, thus it does not get "invented" until the practicality issue gets resolved.

I would say it's more interesting that one-time pad encryption was not invented and used much earlier than 19th century. It has obvious military and intelligence uses and is simple enough to be executed by hand.

newpavlov··on Whatever happened to SHA-256 support in Git
One issue with SHA-3 is that it currently (unfortunately) lacks hardware acceleration support, while SHA-256 can be ridiculously fast on modern x86 and ARM chips. BLAKE3 is another potential alternative, it can be used as XOF and can be very fast without hardware support.
newpavlov··on Whatever happened to SHA-256 support in Git
>just truncate sha-256

A proper solution would be to define SHA-160, similarly to how SHA-224 is defined, i.e. you would use a different initialization constant and truncate output of the core SHA-256 algorithm. But I guess, it would be a bit more difficult to implement than simply truncating output of an existing SHA-256 implementation.

newpavlov··on More use of Rust is inevitable in open source software
>After trying rust only for a few hours i really wonder how come this language remain the most loved one on surveys

Arguably, it's because you tried it "only for a few hours". Rust has a relatively steep learning curve and it usually takes several months to become comfortable with it. But once you get over it, it often becomes quite hard to return to your previous languages, be it Python or C/C++.

Yes, there are tasks which are ill-fit for Rust and your IO-bound backend example is a good one. Not only Rust ecosystem is somewhat underdeveloped in this area (and don't get me started on Rust async and the heap of troubles it causes...), but also you have business requirements which change unpredictably and Rust's strictness introduces a certain friction in such environment.

But I think your mobile library can be a good project to properly learn Rust. If it's properly encapsulated, I don't think it will cause much trouble to your team outside of build system changes and risks associated with you being its sole developer.

newpavlov··on Rust in the 6.2 kernel
Are you talking about binaries compiled for bare-metal embedded? In such cases you usually simply do not have `std`. And how do you think dynamic linking would help here? If anything, it will make total size bigger since compiler will not be able to perform dead code elimination.

When people talk about big Rust binaries they often forget to strip debug information. Doing it can easily shed half of the binary size. And there are other relatively well known ways of reducing binary size (such as strategically using dynamic dispatch instead of monomorphization, using abort for panics, compiling with opt-level=z/s, etc.).

newpavlov··on Electric ferry uses a long extension cord [video]
The point is that on the current system motors which are powered by the electric cable are on the ferry, so it uses propellers to move. The proposed alternative is to use steel cables (which are already present) without any electric ones to pull the ferry, such system should be more efficient power-wise and easier to service.
newpavlov··on Electric ferry uses a long extension cord [video]
But it already does have two steel cables spanning the river all the time. IIUC they get relaxed when ferry does not move and sink to the bottom to allow other ships to pass.
newpavlov··on Electric ferry uses a long extension cord [video]
Why not simply pull the ferry using the steel cables? In other words, put traction systems on both riversides and move the ferry using them. It even should be more efficient than a propeller-based propulsion system.
newpavlov··on Next C compiler is a D compiler: Introducing DMD's ImportC
Because I am not knowledgeable about D runtime I can not say that it will not cause any conflicts with Rust assumptions (thus the use of "probably"). For example, IIRC there are serious issues with integrating Go code into a Rust project. If D code is compiled in the betterC mode, then there should not be any problems.
newpavlov··on Next C compiler is a D compiler: Introducing DMD's ImportC
Using Rust code in D should be relatively easy: expose C API, compile it into a shared library and use it on the D side as any other library. cbindgen (https://github.com/eqrion/cbindgen) can help with generation of header files. Static linking is possible as well.

Using D code in Rust probably will be much more painful and I am not sure if it can be done 100% safely...

newpavlov··on Why did the OpenSSL punycode vulnerability happen?
Rust code would probably return an error on output buffer overflow instead of siltently discarding data. Something like this: https://docs.rs/base16ct/latest/base16ct/mixed/fn.decode.htm...
newpavlov··on Is my package reproducible yet?
I think your definition of "reproducible" is somewhat different from the one usually used. Reproducibility means that by running build process you always get the same binary, nothing more, nothing less. Having a bunch of generated Rust code or blobs in a published crate makes it harder to review (which is why I consider it a bad practice), but such crate is still completely reproducible.
newpavlov··on Is my package reproducible yet?
But generated Rust code itself was included in the crate, correct? Otherwise, it wouldn't be possible to use such crate. If so, I don't see any problems for reproducibility.
newpavlov··on Is my package reproducible yet?
>code was generated from a Python script and some other data and neither of those things are in the crate

Yes, it's (IMO unfortunately) possible to have non-reproducible code generation during compile time. The most prominent example is code generation based on DB schema. But such cases are quite rare and "going with the VCS" does not help here in any way.

If you mean that Rust code gets generated by a Python script and then gets uploaded to crates.io without the "origin" scrips, then I don't see any problems here.

>Or crates where the tests were in VCS but not in the crate

Tests are not used in building the final application, so they are irrelevant for reproducibility.

>prebuilt files in the crate

While I think it's a bad practice (though I haven't seen such crates myself), it does not hinder build reproducibility.

newpavlov··on Is my package reproducible yet?
>Are there any other languages that focus on "reproducible builds" for packages?

I think you misunderstand what "reproducible" means in the context of OP, but I will try to answer your question in the context of Rust.

When you upload a crate to crates.io it gets uniquely identified by its cryptographic hash. It's not possible to replace code for an already uploaded crate version. Now, when you get dependencies for your build you may get newer (semver-compatible) version, but you always can retrieve the older version. To prevent such automatic updates, Rust applications often commit Cargo.lock to their repos, which contains hashes of all dependencies. So unless you manually run `cargo update` you will get exactly the same dependency tree, which with some minor build tweaks makes properly reproducible builds possible.

Arguably, it's a much better system than relying on VCS as Go does.

newpavlov··on The most secure cloud is your computer
>is there an ecryption that's actually secure against state-level actors

Any modern symmetric encryption is extremely likely to be safe against state-level actors for the observable future assuming your encryption key is not compromised and has enough entropy in it. Even with quantum computers you only need twice longer keys. Roughly, AES-256 would provide AES-128 level of security in the presence of practical quantum computers.

If you feel especially paranoid about potential backdoors in algorithms, you can chain several algorithms from different organizations (e.g. AES, Kuznyechik, ChaCha20, and SM4) initialized with different keys. Though such practice is frowned upon by cryptographers.

newpavlov··on Unreal Rust
>Nanite is a very clever data structure for meshes. It's brittle, and full of internal relative addresses within the data item, so it's very vulnerable to buffer overflows.

Using bounds checking is likely to kill performance on such data structures, while using `get_unchecked` or raw pointers will keep the issues.

But even then, I would love to see Nanite implemented in Rust.

newpavlov··on Mars is ‘irrelevant to us’ if Earth is doomed, says Kim Stanley Robertson
On Earth there are numerous second and third order effects to consider. For example, Sahara is considered a very important source of phosphorus for the Amazon rainforest [0] and making Sahara green could have a huge negative impact on it. Meanwhile on Mars impact of terraforming mistakes on population is much smaller because it would be already shielded heavily from external environment.

[0]: https://en.wikipedia.org/wiki/Saharan_dust

newpavlov··on Chaos: The real problem with quantum mechanics
Personally, I am firmly in the "superdeterminism" camp. To me it's the most elegant interpretation of QM without any mystical measurements magically dependent on "consciousness" and whole universes created on each interaction. If you want to learn basics about it, I recommend watching this video: https://www.youtube.com/watch?v=dEaecUuEqfc Do not mind the (joking) clickbait title, the talk itself is quite good even if you only vaguely remember QM.
newpavlov··on Accidentally Turing-Complete
Rust type system is also Turing complete: https://sdleffler.github.io/RustTypeSystemTuringComplete/

It has allowed us to emulate (numeric) const generics in crates like typenum (https://docs.rs/typenum) long before they were implemented as a proper language feature. Even today, typenum is more powerful in some regards than the stable version of const generics.

← PreviousPage 5 of 7Next →