HNHacker News
TopNewBestAskShowJobs

newpavlov

1,056 karma · joined January 13, 2021

submissionscomments
newpavlov··on Hate leap seconds? Imagine a negative one
FYI you can find an extensive discussion of this article on Reddit: https://www.reddit.com/r/programming/comments/s2mij2/
newpavlov··on In Defense of Async: Function Colors Are Rusty
Always? If you mean https://news.ycombinator.com/item?id=26406989#26407395, then I don't remember repeating this discussion with him on HN or anywhere else. He took it really personally, so I don't see a point in rehashing it. I stand by my words behind the link and I don't think that I stand "corrected". You can read the HN discussion and github and IRLO discussions linked in it to form your own opinion.
newpavlov··on In Defense of Async: Function Colors Are Rusty
I have several issues with Rust async and prefer to stay as far from it as possible, but "colors" is not one of them. The article correctly mentions that we can view other function properties, like fallibility, as "colors". For a language like Rust it's important to expose such properties (similarly to how it relies on Result instead of using exceptions) since FSM and sequential code are quite different from each other. If anything, I would say that Rust does not go far enough to expose such properties. I would love to have compiler-enforced properties like "can not not panic", "execution takes bounded time", "uses bounded amount of stack space", "does not use allocations", etc. Some of those properties could've been useful for async code as well and the bounded stack usage is in a certain sense already implicitly part of async.

One of my issues is that Rust continues to pile ad hoc solutions, without looking for more fundamental approaches. For example, in the async case, it was rushed in the current form instead of properly solving generators and linear types. Yes, it's probably helped with the language adoption, but it's a short-term win at the expense of the longer-term language health.

newpavlov··on Quaternions
In my opinion, Geometric Algebra and bivectors (https://bivector.net) in particular is a much better (both practically and pedagogically) approach compared to quaternions and significantly more elegant. It's a real shame that people continue to focus so much on quaternions in this day and age.
newpavlov··on A bug that doesn’t exist on x86: Exploiting an ARM-only race condition
Memory ordering gets somewhat easier after you understand that flat memory shared by execution units is a leaky abstraction desperately patched over decades by layer and layers of hardware and software. Memory ordering is one way to represent message passing and synchronization between different cores and RAM. This why I think that "lock-free algorithms" is a misnomer, you still have synchronization, but you simply rely on hardware for it.
newpavlov··on Fixed Point Arithmetic
Achieving reliable reproducibility for floating point calculations is... difficult, to say the least. There are minor differences between hardware (x87 vs SSE is the most famous one, but there are others). Changing compiler, its version or options may produce subtly different results (the most obvious example is the -ffast-math flag). And even the bigger problem is implementation of non-primitive (e.g. trigonometric) functions. Usually your program will use implementation from a system or vendor library, which probably have different underlying implementations.
newpavlov··on Scientists create matter from pure light, proving the Breit-Wheeler effect
You forget about one smaaaal issue: disposal of created antimatter. Also why bother creating matter from energy when even today we are able to create stockpiles of raw elements (although quite inefficiently and with almost no control over isotopic composition). Assuming that waste will be converted back to those elements, you don't need those stockpiles be too big and energy consumption will be modest when compared to creation of matter directly from energy.
newpavlov··on Rust programs written entirely in Rust
I really hope that we will get a libc-free Linux target. Mustang/rsix looks like a great step in this direction, similarly to steed (https://github.com/japaric/steed) before it. But I think that ideally it should be done properly in Rust proper, not in outside projects. Here a relevant issue, but there is not much progress on it: https://github.com/rust-lang/rfcs/issues/2610
newpavlov··on The problem of effects in Rust (2020)
>I am not sure how to ensure your Rust function being panic safe.

You could use the linking trick in which your panic handler uses non-existent extern fn. For example, this approach is used in the no-panic crate. Of course, this approach is nothing more than a clever hack with several significant limitations.

>Future cancellation

I would say it's a more general problem of Rust lacking linear types.

newpavlov··on Hiding Images in Plain Sight: The Physics of Magic Windows
While the described window is not a hologram per se and I think the hologram-like effect can easily break for other images (i.e. it can not reliably represent depth), it should be noted that it has a certain relation to holograms. Holograms modify plane wave in a way which reproduces wavefront from a recorded scene. Usually it's done by capturing interference pattern, but alternatively hologram can introduce phase shifts by varying thickness (ideally it's worth to use both phase shift and absorption). I think it should be possible to create hologram with the same setup by changing optimization function. Although I am not sure if you will be able to perform precise enough carving with your setup for reproducing complex enough scenes. Try to start with representing several dots at different distance from hologram plane each radiating radial wave. Thinking about diverging lens as of hologram representing one such point should help with intuition.
newpavlov··on Decoding AVIF: Deep dive with cats and imgproxy
Thank you! Yeah, the ability to handle pixel data in chunks of 4 bytes can be useful in certain context.
newpavlov··on Optical adversarial attack can change the meaning of road signs
I don't quite follow you. Yes, we can't prevent keys from leaking here and there, which could allow malicious actors to cause some damage, but we can reduce its impact by making keys short-term (as we do today with TLS certificates) and capability-restricted, i.e. you would not be able to change highway speed limit with a road worker key and you will not be able to use a key generated for city A in a neighboring city B. And you probably will be able to use a leaked key only once, since after an illegal usage is detected it will be quickly revoked. Plus it will be clear whose negligence has caused the leak, thus responsible people/organizations can be held accountable, which eventually should improve cybersecurity level (at least a bit).
newpavlov··on Decoding AVIF: Deep dive with cats and imgproxy
I do understand [1] why 4:2:0 sampling is used, my question was strictly about 4:2:2.

[1]: https://news.ycombinator.com/item?id=28204865

newpavlov··on Optical adversarial attack can change the meaning of road signs
I would assume that "smart" signs would transmit their state via Vehicle-to-Infrastructure (V2I) network, same for roadworks (i.e. workers would mark affected zone and this state will be transmitted via a mobile battery-powered beacon). Such transmitted states will be signed, with road authority key in the root of the certificate tree, so it will be hard to spoof them (at the very least, it will be easy to identify responsible for key leaking). Of course, it assumes a proper digitalization of infrastructure and appropriate regulation changes, which is not given... But in my opinion massive adoption of full self-driving cars will be difficult without them.

>Here in AU, we also have "school zones" with speed limit signs that turn on only during certain times and dates.

Such conditions are usually fixed and do not suddenly change. Even OSM handles them today more or less fine.

newpavlov··on Optical adversarial attack can change the meaning of road signs
Realistic impact of such issues is usually quite exaggerated: https://xkcd.com/1958/

I think that any massively deployed true autopilot system will maintain a global semantic map of roads. Most sudden traffic sign changes, especially those which may influence car behavior, will be manually verified and committed to the global map. Any consistent discrepancies between the model and reality will be investigated as well, especially if it causes customer complaints. And it can be followed by releasing map patches telling software to prefer the model over detection for the problematic sign. Considering that at this stage such maps will be probably integrated with government services (i.e. the map will be compared against official road data), I think such attacks on the autopilot systems will be found quite fast.

newpavlov··on Decoding AVIF: Deep dive with cats and imgproxy
I wonder what are modern use-cases for 4:2:2 sampling? Is it simply a historic relic, which gets ported from codec to codec?
newpavlov··on Decoding AVIF: Deep dive with cats and imgproxy
It makes sense not only from biological point of view as noted in the article, but also from technological as well. Almost all color cameras use Color Filter Arrays [1], meaning that for WxH resolution you don't get WxHx3 pixel values as you would expect from RGB images which you usually consume, but only WxH (i.e. 2/3rds of RGB image data is generated, not measured). With 4:4:4 sampling you have 12 values per 2x2 block, even though only 4 values have been measured by camera for it. Meanwhile with 4:2:0 sub-sampling you have 6 values, which is still bigger than 4, but quite convenient for processing in Y-based color spaces.

[1]: https://en.wikipedia.org/wiki/Color_filter_array

newpavlov··on Are there flaws in some ARM64 instructions?
>In most robotics and AI applications, calculations are unlikely to suffer greatly from inaccuracy but benefit from speed.

FMA instructions are more accurate and faster compared to a sequence of separate multiply and add/sub instructions, not less. The article literally talks about it in the beginning. The issue is reproducibility, which with floats can be quite bad even without FMA.

Also both robotics and neural networks (I dislike the tendency to call it "AI") can be quite susceptible to errors. Gradient-based optimization algorithms can suffer noticeably from calculation errors. This is why you usually want to train ANNs using 64-bit floats.

newpavlov··on Does Rust's array bounds checking affect performance (2017)?
In performance sensitive code without getting into unsafe you would write this function like this: https://godbolt.org/z/PTsnTMq7M

Effectively it results in the same assembly as for the completely unsafe C++ code, apart from the 3 instructions which test slice length. During normal operation they are trivial for branch predictor, so their impact will be negligible in the most cases (but not always, e.g. if this function is part of a hot loop, but then inlining should come to the rescue) and in many cases this check will be removed if compiler is able to prove that the condition is always true. So in the worst case scenario you will lose several cycles on the test instructions and the panic code may slightly increase pressure on I-cache, which I think is a good price to pay for the safety.

newpavlov··on Why asynchronous Rust doesn't work
It does not work in the current version of Rust, but it's not given that a backwards-compatible solution for it could not have been designed, e.g. by using a deeper integration of async tasks with the language or by adding proper linear types, thus all the discussions around reliable async Drop. The linked blog post takes for given that we should be able to drop futures at any point in time, which while being convenient has a lot of implications.
newpavlov··on Why asynchronous Rust doesn't work
Thank you for the advice. Yes, I should've been more careful in my previous comment.

I have noticed this comment only after engaging with him in https://news.ycombinator.com/item?id=26410565 in which he wrote about me:

> You do not know what you are talking about.

> You are confused.

So my reaction was a bit too harsh partially due to that.

newpavlov··on Why asynchronous Rust doesn't work
>I don’t agree with this analysis TBH

No worries, I like when someone disagrees with me and argues his or her position well, since it's a chance for me to learn.

>async drop has been revisited multiple times recently with no luck

The key word is "recently", meaning "after the stabilization". It's exactly my point: this problem was not sufficiently explored in my opinion prior stabilization. I would've been fine with a well argued position "async Drop is impossible without breaking language changes, so we will not care about it", but now we try to shoehorn async Drop on top of the stabilized feature.

>Async drop would still not address the single-allocation-per-state-machine advantage of the current design that you’ve mostly not engaged with in this thread.

I don't think you are correct here, please see this comment: https://news.ycombinator.com/item?id=26408524

newpavlov··on Why asynchronous Rust doesn't work
>In an completion based design the future itself tells the runtime that the value is ready

I think there is a misunderstanding. In a completion-based model (read io-uring, but I think IOCP behaves similarly, though I am less familiar with it) it's a runtime who "notifies" tasks about completed IO requests. In io-uring you have two queues represented by ring buffers shared with OS. You add submission queue entries (SQE) to the first buffer which describe what you want for OS to do, OS reads them, performs the requested job, and places completion queue events (CQEs) for completed requests into the second buffer.

So in this model a task (Future in your terminology) registers SQE (the registration process may be proxied via user-space runtime) and suspends itself. Let's assume for simplicity that only one SQE was registered for the task. After OS sends CQE for the request, runtime finds a correct state transition function (via meta-information embedded into SQE, which gets mirrored to the relevant CQE) and simply executes it, the requested data (if it was a read) will be already filled into a buffer which is part of the FSM state, so no need for additional syscalls or interactions with the runtime to read this data!

If you are familiar with embedded development, then it should sound quite familiar, since it's roughly how hardware interrupts work as well! You register a job (e.g. DMA transfer), dedicated hardware block does it, and notifies a registered callback after the job was done. Of course, it's quite an oversimplification, but fundamental similarity is there.

>I think your design ultimately requires futures to be implemented as a language feature, rather than a library

I am not sure if this design would have had a Future type at all, but you are right, the advocated approach requires a deeper integration with the language compared to the stabilized solution. Though I disagree with the opinion that it would've been impossible to do in Rust 1.

newpavlov··on Why asynchronous Rust doesn't work
Yes, it's a real possibility. But the problem is that the other route was not properly explored, so we can not compare advantages and disadvantages. Instead Rust went all-in on a bet which was made 3 years ago.
newpavlov··on Why asynchronous Rust doesn't work
I do not think that impossibility of a reliable async Drop in Rust 1 is a proven thing (prior to the stabilization of async in the current form). Yes, it may require some unpleasant additions such as making Futures and async fns more special than they are right now and implementing it with high probability would have required a lot of work (at least on the same scale as was invested into the poll-based model), but it does not make it impossible automatically.
newpavlov··on Why asynchronous Rust doesn't work
I am not mad, it was nothing more than an attempt to urge a more civil tone from boats. If you both think that such tone is warranted, then so be it. But it does affect my (really high) opinion about you.

I do understand the pain of your dear work to be harshly criticized. I have experienced it many times in my career. But my critique intended as a tough love for the language in which I am heavily invested in. If you see my comments as only "extremely inflammatory"... Well, it's a shame I guess, since it's not the first case of the Rust team unnecessarily rushing something (see the 2018 edition debacle), so I guess such attitude only increases rate of mistake accumulation by Rust.

newpavlov··on Why asynchronous Rust doesn't work
I have tried to raise those issues in the stabilization issue (I know, quite late in the game), but it simply got shut down by the linked comment with a clear message that further discussion be it in an issue on in a new RFC will be pointless.

Also please note that the article is not mine.

newpavlov··on Why asynchronous Rust doesn't work
I meant the callback based approach described in the article, for example take this line from it:

>Unfortunately, this approach nevertheless forces allocation at almost every point of future composition, and often imposes dynamic dispatch, despite our best efforts to avoid such overhead.

It clearly does not apply to the model which I've described earlier.

Of course, the described FSM state transition functions can be rightfully called callbacks, which adds a certain amount of confusion.

I can agree with the argument that a proper async Drop can not be implemented in Rust 1.0, so we have to settle with a compromise solution. Same with proper self-referential structs vs Pin. But I would like to see this argument to be explicitly stated with sufficient backing of the impossibility statements.

newpavlov··on Why asynchronous Rust doesn't work
Please, calm down. I do appreciate your work on Rust, but people do make mistakes and I strongly belive that in the long term the async stabilization was one of them. It's debatable whether async was essential or not for Rust, I agree it gave Rust a noticeable boost in popularity, but personally I don't think it was worth the long term cost. I do not intend to change your opinion, but I will keep mine and reserve the right to speak about this opinion publicly.

In this thread [1] we have a more technicall discussion about those models, I suggest to continue that thread.

>I assure you, the problems it present are not related to Rust's polling model AT ALL.

I do not agree about all problems, but my OP was indeed worded somewhat poorly, as I've admitted here [2].

>Pin is not a hack any more than Box is. It is the only way to fit the desired ownership expression into the language that already exists

I can agree that it was the easiest solution, but I strongly disagree about the only one. And frankly it's quite disheartening to hear from a tech leader such absolutistic statements.

>It is simply FUD - frankly, a lie - to say that it will block "noalias,

Where did I say "will"? I think you will agree that it at the very least will it will cause a delay. Also the issue shows that Pin was not proprely thought out, especially in the light of other safety issues it has caused. And as uou can see by other comments, I am not the only one who thinks so.

>the content your consuming is usually from one of these kinds of people:

Please, satisfy my curiosity. To which category do I belong in your opinion?

[1]: https://news.ycombinator.com/item?id=26410359

[2]: https://news.ycombinator.com/item?id=26407565

newpavlov··on Why asynchronous Rust doesn't work
Thank you for the link! But immideately we can see the false equivalence: completion based API does not imply the callback-based approach. The article critigues the latter, but not the former. Earlier in this thread I've described how I see a completion-based model built on top of FSMs generated by compiler from async fns. In other words, the arguments presented in that article do not apply to this discussion.

>The performance issues it presented had nothing to do any sort of impedence mismatch between a completion based future and epoll

Sorry, but what? Even aturon's article states zero-cost as one of the 3 main goals. So performance issues with strong roots in the selected model is a very big problem in my book.

>You do not know what you are talking about.

>You are confused.

Please, tone down your replies.

← PreviousPage 6 of 7Next →