HNHacker News
TopNewBestAskShowJobs

newpavlov

1,056 karma · joined January 13, 2021

submissionscomments
newpavlov··on All about automotive lidar
>we had about an inch of separation between our laser diodes and our photodiodes

Why can't you place them further away from each other using an additional optical system (i.e. a mirror) and adjusting for the additional distance in software?

newpavlov··on Zig's new plan for asynchronous programs
>`async` is in the type system.

No, it's not. `async` is just syntax sugar, the "effect" gets emulated in the type system using `Future`. This is one of the reasons why the `async` system feels so foreign and requires so many language changes to make it remotely usable. `const` is much closer to a "true" effect (well, to be precise it's an anti-effect, but it's not important right now).

Also, I think it's useful to distinguish between effect and type systems, instead of lumping them into just "type system". The former applies to code and the latter to data.

>That feels dense.

Yes. But why `async` is more important than `alloc`? For some applications it's as important to know about potential allocations, as for other applications to know about whether code potentially yields or not.

Explicitly listing all effects would the most straightforward approach, but I think a more practical approach would be to have a list of "default effects", which can be overwritten on the crate (or maybe even module) level. And on the function level you will be able to opt-in or opt-out from effects if needed.

>I think ideally it would be something readers could spot at first glance

Well, you either can have "dense" or explicit "at first glance" signatures.

newpavlov··on Zig's new plan for asynchronous programs
I agree. But it should be done with a proper effect system, not a pile of ad hoc hacks built on abuse of the type system.
newpavlov··on Zig's new plan for asynchronous programs
In my (Rust-colored) opinion, the async keyword has two main problems:

1) It tracks code property which is usually omitted in sync code (i.e. most languages do not mark functions with "does IO"). Why IO is more important than "may panic", "uses bounded stack", "may perform allocations", etc.?

2) It implements an ad-hoc problem-specific effect system with various warts. And working around those warts requires re-implementation of half of the language.

newpavlov··on Shai-Hulud Returns: Over 300 NPM Packages Infected
While I agree that dependency tree size can be sometimes a problem in Rust, I think it often gets overblown. Sure, having hundreds of dependencies in a "simple" project can be scary, but:

1) No one forces you to use dependencies with large number of transitive dependencies. For example, feel free to use `ureq` instead of `reqwest` pulling the async kitchen sink with it. If you see an unnecessary dependency, you could also ask maintainers to potentially remove it.

2) Are you sure that your project is as simple as you think?

3) What matters is not number of dependencies, but number of groups who maintain them.

On the last point, if your dependency tree has 20 dependencies maintained by the Rust lang team (such as `serde` or `libc`), your supply chain risks are not multiplied by 20, they stay at one and almost the same as using just `std`.

newpavlov··on Why is Zig so cool?
Great comment! I agree about comptime, as a Rust programmer I consider it one of the areas where Zig is clearly better than Rust with its two macro systems and the declarative generics language. It's probably the biggest "killer feature" of the language.
newpavlov··on Why is Zig so cool?
>often more performant than Rust with lower resource usage

[citation needed]

If we are to trust this page [0] Rust beats Zig on most benchmarks. In the Techempower benchmarks [1] Rust submissions dominate the TOP, while Zig is... quite far.

Several posts which I've seen in the past about Zig beating Rust by 3x or such all turned to be based on low quality Rust code with some performance pitfalls like measuring performance of writing into stdout (which Rust locks by default and Zig does not) or iterating over ..= ranges which are known to be problematic from the performance perspective.

[0]: https://programming-language-benchmarks.vercel.app/rust-vs-z...

[1]: https://www.techempower.com/benchmarks/

newpavlov··on Why is Zig so cool?
Have you tried the GNU toolchain? IIRC rustup provides the option to use it instead of the MSVC toolchain during the initial installation.
newpavlov··on Comparison Traits – Understanding Equality and Ordering in Rust
I agree. In my opinion NaNs were a big mistake in the IEEE 754 spec. Not only they introduce a lot of special casing, but also consume a relatively big chunk of all values in 32 bit floats (~0.4%).

I am not saying we do not need NaNs (I would even love to see them in integers, see: https://news.ycombinator.com/item?id=45174074), but I would prefer if we had less of them in floats with clear sorting rules.

newpavlov··on Futurelock: A subtle risk in async Rust
>This goes all the way back to the "futures are inert" design of async Rust

Yeap. And this footgun is yet another addition to the long list of reasons why I consider the Rust async model with its "inert" futures managed in user space a fundamentally flawed un-Rusty design.

newpavlov··on A years-long Turkish alphabet bug in the Kotlin compiler
In most cases locale is encoded in character itself, i.e. Latin "a" and Cyrillic "a" are two different characters, despite being visually indistinguishable in most cases.

The "language-sensitive" section of the special casing document [0] is extremely small and contains only the cases of stupid reuse of Latin I.

[0]: https://www.unicode.org/Public/UCD/latest/ucd/SpecialCasing....

newpavlov··on Some graphene firms have reaped its potential but others are struggling
IIUC graphitosis, silicosis, and black lung require to inhale ungodly amounts of dust. It's orders of magnitude more than we can expect from flaking-based trace contamination.

Why do you expect a different result from "tiny sheets of graphene flaking off everywhere from nanocoatings" compared to the same flaking from graphite smeared across paper?

newpavlov··on Some graphene firms have reaped its potential but others are struggling
>Try not to breathe any, studies are still pending but that stuff gets everywhere.

I would understand such comment in the context of carbon nanotubes or fullerenes, but graphene? Have you forgot that graphite is literally a bunch of stacked graphene?

Considering how much graphite pencils are used across the world, we would've seen hypothetical negative effects already with a high degree of confidence.

Yes, graphene production aims to produce larger sheets, but it only makes graphene less biologically active, not more.

newpavlov··on A years-long Turkish alphabet bug in the Kotlin compiler
I call BS. Without a series of MAJOR blunders Unicode was destined to succeed. When the rest of the world has migrated to Unicode, I am more than certain that Turks would've migrated as well. Yes, they may have complained for several years and would've spent a minuscule amount of resources to adopt the conversion software, but that's it, a decade or two later everyone would've forgotten about it.

I believe that even addition of emojis was completely unnecessary despite the pressure from Japanese telecoms. Today's landscape of messengers only confirms that.

newpavlov··on A years-long Turkish alphabet bug in the Kotlin compiler
>If they had invented a separate code point for I in Turkish, then when converting text from those existing ISO character encodings, you’d have to know whether the text is Turkish or English or something else, to know which Unicode code point to map the source “I” into. That’s exactly what Unicode was designed to avoid.

Great. So now we have to know locale for handling case conversion for probably centuries to come, but it was totally worth to save a bit of effort in the relatively short transition phase. /s

newpavlov··on A years-long Turkish alphabet bug in the Kotlin compiler
>Even modern languages like Rust did a crappy job of enforcing it

Rust did the only sensible thing here. String handling algorithms SHOULD NOT depend on locale and reusing LATIN CAPITAL LETTER I arguably was a terrible decision on the Unicode side (I know there were reasons for it, but I believe they should've bit the bullet here), same as Han unification.

newpavlov··on HTTP3 Explained
Thank you for a great overview! I wish HTTP3/QUIC was the "default option" and had much wider adoption.

Unfortunately, software implementations of QUIC suffer from dealing with UDP directly. Every UDP packet involves one syscall, which is relatively expensive in modern times. And accounting for MTU further makes the situation ~64 times worse.

In-kernel implementations and/or io-uring may improve this unfortunate situation, but today in practice it's hard to achieve the same throughput as with plain TCP. I also vaguely remember that QUIC makes load-balancing more challenging for ISPs, since they can not distinguish individual streams as with TCP.

Finally, QUIC arrived a bit too late and it gets blocked in some jurisdictions (e.g. Russia) and corporate environments similarly to ESNI.

newpavlov··on Redox OS Development Priorities for 2025/26
The kernel ABI is notoriously backwards compatible (the famous "we do not break userspace" and all). The primary reason why binaries rot on Linux is GLIBC and other shared library dependencies. I still can execute a MUSL binary compiled more than a decade ago without any issues.
newpavlov··on Redox OS Development Priorities for 2025/26
I still disagree with their decision to make libc THE system interface. I understand why it's important to provide a compatibility layer, bit, ideally, I would like to see a Linux-like (potentially semver-versioned) stable sycall API, or at the very least something like libsystem, i.e. a thin wrapper around technically unstable syscalls API.
newpavlov··on Folks, we have the best π
I wonder if there is a nice formula for the pi from n plot (extended to real n-s). It looks "nice enough" on the first glance.

As for n=0, can't you prove that pi=inf for n=0 using limits?

newpavlov··on Decentralized YouTube alternative adds livestream scheduling in new release
It's great to see PeerTube development, but I think it will continue to be extremely niche until a good solution for decentralized monetization is found. For both content creators and data hosters.

It's extremely hard to move users accustomed to "free" advertisement-fueled services to the micro-payment model. Content creators actively try to increase revenue by using things like Patreon and membership-only videos, but YouTube ads is still the dominant source for most.

And properly paying for the underlying infrastructure is even more difficult. Filecoin tries to do it, but it's hard to call it a succesfull project.

newpavlov··on A clickable visual guide to the Rust type system
I can not name an ISA with such instructions out of my head.

As for unsigned integers, as I mentioned in the other comment, we probably need two separate instruction sets for "wrapping" and NaN-able operations on unsigned integers.

newpavlov··on A clickable visual guide to the Rust type system
For absolute jumps you don't need extra logic, since CPUs could declare the last page always unmapped, so such jumps would always result in a page fault (similarly to the null page on most systems).

For relative non-immediate jumps the added logic is extremely simple (hardware exception on NaN) and should not (AFAIK) hinder performance of jumps in any way.

newpavlov··on AMD claims Arm ISA doesn't offer efficiency advantage over x86
- Handling of misaligned loads/stores: RISC-V got itself into a weird middle ground, ops on misaligned pointers may work fine, may work "extremely slow", or cause fatal exceptions (yes, I know about Zicclsm, it's extremely new and only helps with the latter, also see https://github.com/llvm/llvm-project/issues/110454). Other platforms either guarantee "reasonable" performance for such operations, or forbid misaligned access with "aligned" loads/stores and provide separate misaligned instructions. Arguably, RISC-V should've done the latter (with misaligned instructions defined in a separate higher-end extension), since passing unaligned pointer into an aligned instruction signals correctness problems in software.

- The hardcoded page size. 4 KiB is a good default for RV32, but arguably a huge missed opportunity for RV64.

- The weird restriction in the forward progress guarantees for LR/SC sequences, which forces compilers to compile `compare_exchange` and `compare_exchange_weak` in the absolutely same way. See this issue for more information: https://github.com/riscv/riscv-isa-manual/issues/2047

- The `seed` CSR: it does not provide a good quality entropy (i.e. after you accumulated 256 bits of output, it may contain only 128 bits of randomness). You have to use a CSPRNG on top of it for any sensitive applications. Doing so may be inefficient and will bloat binary size (remember, the relaxed requirement was introduced for "low-powered" devices). Also, software developers may make mistake in this area (not everyone is a security expert). Similar alternatives like RDRAND (x86) and RNDR (ARM) guarantee proper randomness and we can use their output directly for cryptographic keys with very small code footprint.

- Extensions do not form hierarchies: it looks like the AVX-512 situation once again, but worse. Profiles help, but it's not a hierarchy, but a "packet". Also, there are annoyances like Zbkb not being a proper subset of Zbb.

- Detection of available extensions: we usually have to rely on OS to query available extensions since the `misa` register is accessible only in machine mode. This makes detection quite annoying for "universal" libraries which intend to support various OSes and embedded targets. The CPUID instruction (x86) is ideal in this regard. I totally disagree with the virtualization argument against it, nothing prevents VM from intercepting the read, no one excepts huge performance from such reads.

And this list is compiled after a pretty surface-level dive into the RISC-V spec. I heard about other issues (e.g. being unable to port tricky SIMD code to the V extension or underspecification around memory coherence important for writing drivers), but I can not confidently talk about those, so it's not part of my list.

P.S.: I would be interested to hear about other people gripes with RISC-V.

newpavlov··on A clickable visual guide to the Rust type system
>Wouldn't this make CPU flags useless?

They would, but I agree with RISC-V here, CPUs should not rely on them in the first place.

I do not understand your argument about branches, how would it hinder the jump instructions?

We still would need separate "wrapping" instructions (e.g. for implementing bigints and cryptographic algorithms), but they probably could be limited to unsigned operations only.

>I can't wait to tell CPU to JMP NaN.

How is it different from jumping to null? If you do such jump, it means you have a huge correctness problem with your code.

newpavlov··on AMD claims Arm ISA doesn't offer efficiency advantage over x86
>RISC-V is a new-school hyper-academic hot mess.

Yeah... Previously I was a big fan of RISC-V, but after I had to dig slightly deeper into it as a software developer my enthusiasm for it has cooled down significantly.

It's still great that we got a mainstream open ISA, but now I view it as a Linux of the hardware world, i.e. a great achievement, with a big number of questionable choices baked in, which unfortunately stifles other open alternatives by the virtue of being "good enough".

newpavlov··on A clickable visual guide to the Rust type system
Tangential note: I sometimes wish that signed integers were symmetrical. i8 would represent the range of [-127 to 127] with 0xFF representing NaN. Any operation which can not be computed (division by zero, overflows, operation with another NaN, etc.) would result in NaN. For further symmetry we could do the same for signed integers as well.

Yes, it's possible to encode such types manually, but it will not be efficient since CPUs do not natively support such operations.

newpavlov··on The future of 32-bit support in the kernel
As well as in a number of widely spread cryptographic algorithms (e.g. SHA-2), which use BE for historic reasons.
newpavlov··on Io_uring, kTLS and Rust for zero syscall HTTPS server
>Is this really better than what we have now?

Depends on the metric you use. Memory-wise it's a bit less efficient (our tasks usually are quite big, so relative overhead is small in our case), runtime-wise it should be on par or slightly ahead. From the source code perspective, in my opinion, it's much better. We don't have the async/await noise everywhere and after development of the `std` fork we will get async in most dependencies as well for "free" (we still would need to inspect the code to see that they do not use blocking `libc` calls for example). I always found it amusing that people use "sync" `log`-based logging in their async projects, we will not have this problem. The approach also allows migration of tasks across cores even if you keep `Rc` across yield points. And of course we do not need to duplicate traits with their async counterparts and Drop implementations with async operations work properly out of the box.

>Can you go more in-depth into these limitations and which would be alleviated by having first class support for your approach in the compiler/std?

The most obvious example is thread locals. Right now we have to ensure that code does not wait on completion while having a thread local reference (we allow migration of tasks across workers/cores by default). We ban use of thread locals in our code and assume that dependencies are unable to yield into our executor. With forked `std` we can replace the `thread_local!` macro with a task-local implementation which would resolve this issue.

Another source of potential unsoundness is reuse of parent task stack for sub-task stacks in our implementation of `select!`/`join!` (we have separate variants which allocate full stacks for sub-tasks which are used for "fat" sub-tasks). Right now we have to provide stack size for sub-tasks manually and check that the value is correct using external tools (we use raw syscalls for interacting with io-uring and forbid external shared library calls inside sub-tasks). This could be resolved with the aforementioned special async ABI and tracking of maximum stack usage bound.

Finally, our implementation may not work out-of-box on Windows (I read that it has protections against messing with stack pointer on which we rely), but it's not a problem for us since we target only modern Linux.

newpavlov··on Io_uring, kTLS and Rust for zero syscall HTTPS server
You have an inevitable overhead of managing the owned buffer when compared against simply passing mutable borrow to an already existing buffer. Imagine if `io::Read` APIs were constructed as `fn read(&mut self, buf: Vec<u8>) -> io::Resul<Vec<u8>>`.

Parity with synchronous programming is an explicit goal of Rust async declared many times (e.g. see here https://github.com/rust-lang/rust-project-goals/issues/105). I agree with your rant about the illusion of synchronicity, but it does not matter. The synchronous abstraction is immensely useful in practice and less leaky it is, the better.

← PreviousPage 2 of 7Next →