HNHacker News
TopNewBestAskShowJobs

OskarS

8,399 karma · joined September 4, 2016

submissionscomments
OskarS··on The curl-wget Venn diagram
> You recall incorrectly. curl's -C flag does not work as-is. You must specify the offset from where it should continue. Why doesn't it take the resumed file's existing length as the guess by default? What else could the user want outside of some very exotic cases?

But... it does, though. From the man page ( https://curl.se/docs/manpage.html#-C )

> Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.

I just tried it, works perfectly. I don't really see a difference between writing `--continue` in wget and `-C -` in curl. And the use case for specifying it is not so exotic, you might want to do range requests for all sorts of reasons.

Look, it's fine if you prefer wget's command line: I don't think retrying a request is a reasonable default for a tool like curl, but reasonable people can disagree on that for sure. But curl is perfectly capable of resuming downloads automatically, you're just (very arrogantly) wrong on that one.

OskarS··on The curl-wget Venn diagram
> I tried using curl but out of the box it could not resume or retry the download.

Maybe I'm misunderstanding, but curl has exactly that feature, it's the `-C` flag. If you want retries, there's `--retry`. I find curls defaults pretty sane, personally, I wouldn't want either of those by default for a tool like curl.

OskarS··on The curl-wget Venn diagram
I much prefer the fact that curl doesn't do this by default (but has the option), it much closer matches the behavior of most unix-y tools. Makes it so much easier to put it into pipelines.
OskarS··on Philosophy of Coroutines
Stackless coroutines are arguably pretty serializable, since they are (usually) implemented by bundling all the relevant data into a struct. There’s no a priori reason that couldn’t be serialized, thiugh I admit I cannot think of an example I’ve seen it used.
OskarS··on ISPs should not police online speech no matter how awful it is
> Intuition pump: imagine the postal service opening every letter to check to see if someone's organising a letter writing campaign sending abuse to George Washington

Just on this specific point: this is not a particularly weird scenario at all. There are very strong rules about what you can and cannot send in the mail, and these rules are enforced. The most obvious example is the Comstock laws, which regulate "obscene and immoral materials" sent through the mail (including things like information about birth control). They're from the 1800s, but several are still on the books. It's not a strange concept at all in American law to regulate what can and cannot be sent through the mail.

OskarS··on GIL removal and the Faster CPython project
My impression was that that was exactly what they were going to do: replace the GIL with fine-grained locking on the objects themselves. I can't imagine they'd let multiple python threads manipulate python data-structures concurrently, the interpreter would segfault immediately.

> (but then how do you get a non-synchronized [] if you want one?)

You don't. This is one of the reasons why using the GIL is higher performance for single-threaded use-cases: stuff like lists and dicts can be non-synchronized

OskarS··on Decimal to fraction
Why would you do this instead of just converting the digits to a fraction over a power of 10 and then reducing the fraction (or power of 2, if it's a floating point datatype)? I was thinking it was faster, but the recursive procedure involves a division step, so I would assume that calculating GCD using the binary algorithm (which is just uses shifts and subtraction) would be faster? I guess this is if your numeric data-type can't fit the size of the denominator?
OskarS··on The Future of the Vim Project
Lots of other people have mentioned very good practical reasons (and you can read the linked post for others, like people being unsure of what the password to the Vim FTP is), but there are lots of good sentimental ones as well.

When something like this happens, you lose your mind slightly, and you become obsessed with preserving whatever is possible to preserve of the person. I went so far as to record his voice-mail message, just because I felt i needed to.

Of all the internet stuff, the thing that was most important to us was photos: he was a photographer, and he used a photo-uploading service (I think it was Google Photos, but I'm unsure, it's been a couple of years) and I was able to get an archive with all his thousands of photographs.

Eventually, I put everything I could possibly find (computers, internet services, whatever) into one big zip file, and put it on my local NAS (which is backed up to the cloud). I don't think I'll ever have the heart to go through it again, but my brother had young kids who never really got to know their dad. I figure one day they might want to look at it (even if it's 30 years from now) so it makes me feel good to know that it's been preserved.

OskarS··on The Future of the Vim Project
My brother passed away very suddenly a few years ago, and I was put in charge of wrapping up and archiving his "digital" life. We were very lucky that we had access to a recovery email for his main gmail account (as well as a couple of passwords that his partner knew) and was able to access and archive virtually all data we could think of (services like Google Takeout were invaluable). I realized that if this had happened to me, it would have been virtually impossible to do, as all my passwords and credentials are in my password manager, and the password to that was only in my head.

It's a good thing to plan for this eventuality, to make it easy for your family and friends to wind up your "digital life" after you've passed. 1Password has a very good solution for this, with a "recovery document" you can print out and write down your password on, which contains instructions anyone else would need to access your 1Password account. I gave a copy of this document printed out to a small number of people I trust implicitly.

You never know when something sudden can happen to you. For the sake of those you leave behind, it's a nice gift to plan for this eventuality, even if it seems far off at the moment.

OskarS··on How to check if two SQL tables are the same
Does this work with the bag/multiset distinction that the author uses? Like, if table1 has two copies of some row and table2 has a single copy of that row, wont this query return that they're the same? But they're not: table1 has two copies of the same row, whereas table2 just has one?
OskarS··on NAT Types
> As to "implicit firewall", as has been indicated, there isn't really such a thing, just the filtering behaviour.

I mean, there clearly is a firewall. If you buy a printer today, you don't really have to worry very much about how safe its network features are or if the firmware is updated, because if you put it behind a NAT router, it's not reachable from the global internet. Which is clearly a kind of firewall. And, generally, a good thing: it would be TERRIBLE for security if every shoddy network enabled device that anyone's bought for the last 30 years were reachable from the public internet. In the utopian "every device has a unique IPv6 address!", that would be the case.

It's true that an IPv6 gateway/router/whatever could have an explicit firewall that provides the same security, but the fact that this comes "by default" with a NAT router is honestly pretty nice for most users. And it's silly to deny it's an "implicit firewall", it clearly is. That's why I called the IPv6 version an "explicit firewall" two sentences ago.

OskarS··on People are paying to break printers with sledgehammers in smash rooms
I've always wanted to smash a CRT monitor. Big and hollow! Seems like so much fun!
OskarS··on Mistakes with Rust smart pointers: when Deref goes wrong
I've always liked that feature of C and C++. There's a lot of people that really hate it and in new languages they unify it to be just a dot. Zig does this as well, for instance. I always liked that the distinction was very explicit: use a.b() if you're calling b() on the object itself, use a->b() if you're de-referencing a first.

There are lots of things wrong with C and C++ that Zig and Rust fixes, but I never thought of this one being problematic in any way. It's as easy to type, the difference is obvious, and it's easy to see from a glance what's going on.

OskarS··on Memory safety without borrow checking, reference counting, or garbage collection
> Try it, make an array of, I dunno, Strings, and then "delete" one. You can't, there is no "delete" operator, that would be meaningless.

Your argument is that there's no such function as `std::Vec::remove`? I can assure you, it exists: https://doc.rust-lang.org/std/vec/struct.Vec.html#method.rem...

If you use that function, and you've held on to a an index that is no longer valid, it is pointing to the wrong string. Or worse: it's pointing outside of the array, and your program will panic. The index is no longer valid, just like a pointer in C is no longer valid after calling free().

> You have to go build your own pit, and fill it with spikes, and jump into it, all so that you can moan, "Oh, Rust basically has the same terrible pit traps as C or C++" except, you built the trap, so don't do that.

I am not saying "Rust basically has the same terrible pit traps as C or C++". I have never said that, and I never would. What I'm saying is that if you're using indexes like this, you no longer have the benefit of lifetime safety guaranteed by the borrow checker. You have, in essence, turned the borrow checker off for this part.

BTW, making the argument "as long as you're not a bad programmer, this isn't a problem" is exactly the argument that some Rust haters make: "well, just don't call free() twice, and it's not a problem!". Either way, it's bull: the whole point of the borrow checker is that it statically guarantees that these kinds of lifetime issues can't happen. But if you turn it off using indexes, it guarantee is out the window. The whole discussion started with the parent commentator pointing out a project that does exactly this, and me pointing out that this is a problem from a static safety perspective.

OskarS··on Memory safety without borrow checking, reference counting, or garbage collection
I want to be clear: I think Rust offers ENORMOUS security benefits over C. No one is disputing that. I personally love Rust, I think it's a great language. This is not an attack on the language, this is an observation that if you use this particular pattern (indexes instead of pointers), you are doing something dangerous, and you can no longer rely on the borrow checker to validate lifetimes.

It seems to me to be a pretty unarguable point. It's frankly a bit shocking to me to see this vehement of a disagreement.

OskarS··on Memory safety without borrow checking, reference counting, or garbage collection
From a security perspective, yes Rust is better, but it's still a security issue: if you can make a Rust program panic due to overflow, that's a DoS attack.

I think this description is a very narrow way to look at memory safety (and again: totally ignoring the broader issue of lifetime safety), if I'm going to be honest. In my Rust program, I have two functions `spawnDude()` returning an index and a `despawnDude()` taking an index. In C, i have `malloc()` returning a pointer and `free()` taking a pointer. The lifetime issues are the same: just like I shouldn't `free()` a pointer twice, i shouldn't `despawnDude()` twice, and I shouldn't use a dude after I've despawned him. The implementation could even be very similar: using arenas (which is essentially what the Rust array is) and free-lists.

Again: these were the issues Rust was designed to solve, and the borrow checker is the tool it uses to solve them. And it absolutely does do that, if you use the native Rust constructs: this is the true super-power of Rust. The reason why it's so much easier to work with indexes is because you've deliberately chosen not to have the borrow checker analyze this situation. If that is something you have to do a lot of (and I've seen it a number of times, not just in the project the parent mentioned), it does say something important about how the borrow checker limits the expressive power of the language, if you have to turn it off in this way.

OskarS··on Memory safety without borrow checking, reference counting, or garbage collection
I wrote a longer response to a sibling comment, but I highly dispute this:

> Rust's array type is an actual array, and so a lot of the problems you're worried about (which would be big pitfalls out of the box in C) only happen in Rust if you went to some effort to dig a hole for yourself so you could fall into it.

It is an extremely easy hole to fall into. If you hold on to an index X into an array, and some different part of the program deleted an entry in the array with an index smaller than X, your X is now no longer valid, it's pointing to the wrong object. This is exactly equivalent to an "invalidated pointer", which is an issue the borrow checker was designed to solve. But there is no way for it to know this, because by using indexes instead of pointers, you've disabled its ability to analyze this situation.

I agree that pointers in C and indexes in Rust are not the same thing, and there are other issues with C pointers that Rust's borrow checker solves. But imagining that you're magically safe from any lifetime issues "because Rust" is foolish.

OskarS··on Memory safety without borrow checking, reference counting, or garbage collection
I would say that this is an extremely narrow view of "memory safety", and totally ignores the broader concept of "lifetime safety". Yes, if the thing you want to prevent is crashes due to segfaults, indexes in Rust prevent that. But I would argue there's no practical difference between "crash due to segfault" and "panic due to index overflow" (which is definitely possible with the Rust model): both are just different ways of looking at "you dereferenced an invalid pointer". It does prevent the pointer writing into different data structures though, which is definitely an advantage of Rust (which comes with the runtime cost of bounds-checking, but that is probably worth it).

But more generally: imagine you're making a game where you shoot a bunch of dudes, so you have a `std::Vec<Dude>` to store them, and elsewhere in the code, you reference this vec using indexes. When you shoot a dude, you want to remove it from this array, but being a programmer concerned about performance, you don't want to literally delete the entry, as that is an O(n) operation. Instead, you mark the dude as "despawned" or whatever, and then when you want to spawn a new dude, you keep a list of despawned dudes and just recreate the new dude in the same place in the array. If other places in the code are holding on to indexes to this array, the index is now invalid, but it will not be detectable: it'll just point to the wrong data, which can wreak all sorts of havoc.

This is almost exactly equivalent to "use-after-free" issues in C. In most cases, `free()` will not literally free the memory, it will mark the memory as unused and stored in a free-list, to be available for the next `malloc()` call. If you're holding on to a pointer that has been free()d, it's not that the program will segfault, it's much more insidious than that: it will point to memory that is apparently valid, but actually garbage.

This is precisely the kind of lifetime issue the Rust borrow checker was designed to solve, but by using indexes like this, you've basically just turned the borrow checker off without realizing it. If the borrow checker is intrusive enough that this becomes a very common pattern in Rust, I personally think it does serve as a pretty good counter-argument to a lot of things people say about Rust. Rust people are very quick to sing the borrow checker's praises and say that "as long as you learn to work with it, it stops becoming a problem!". If it stops becoming a problem only because you sneakily turn it off using tricks like this, maybe it's a bigger problem than people are willing to admit.

OskarS··on Memory safety without borrow checking, reference counting, or garbage collection
I’ve always thought this is a pretty big blindspot in Rust. Having a raw pointer and having an index into an array are essentially equivalent, and doing this in Rust is a way to turn off the borrow checker for this particular object.

The index is bounds-checked, sure, but lifetime issues like “double free” or “use after free” are every bit as present with indexes. For instance, if you have an index in one place, but it’s deleted and/or reinitialized to some other place, you are now holding on to an object in an undefined state. Using it will cause the same issues “use-after-free” does in C. Best case scenario in that cases is that it crashes, but it’s just as likely your program will just silently be wrong.

I’m not saying not to do it, I’m just saying this is something Rust developers needs to be aware of: storing indexes like that removes a whole bunch of safety guarantees the borrow checker checks for you, without any code marked “unsafe”.

OskarS··on Modern language models refute Chomsky’s approach to language
The common thread in both cases is (as you say) children: in both creoles and NSL, the early versions of the language are somewhat primitive (being developed by older children and adults), but when children are exposed to it from an early age, the languages acquire much more complexity and sophistication, becoming every bit as rich as "regular" languages. It fits well with the hypothesis that the early years of life are crucial for language development, and that there is indeed something innate in most humans for deep structural understanding of language. It's not just statistical pattern recognition.
OskarS··on Modern language models refute Chomsky’s approach to language
The classic example which I think you’re referring to is Nicaraguan Sign Language, which developed organically in Nicaraguan schools for deaf children where neither the children nor the teachers knew any other form of sign language. It’s a fascinating story, a complex fully developed language created by children. Seems to indicate that this is indeed a very inate capability among humans in larger groups: https://en.wikipedia.org/wiki/Nicaraguan_Sign_Language
OskarS··on That’s a huge amount of energy being transferred to the atmosphere
Indeed, there were a lot of scaremongering about covid. It then turned into the worst global pandemic in a century which killed millions and millions of people, and hundreds of millions of people (if not billions) was afflicted with it, many of them with symptoms that affect them for years. I'm not entirely sure what point you are trying to make.
OskarS··on Wizards of the Coast Raids YouTuber’s House to Take Back Magic Cards
For sure. I didn't mean to imply otherwise. I was just saying I'm always curious about what the processes are in companies that lead to these disastrous results.
OskarS··on Wizards of the Coast Raids YouTuber’s House to Take Back Magic Cards
I always wonder about stories like this what the internal process that lead to this insane overreaction was. At some point, did someone really make the decision "lets send Pinkertons to this guy's house, risking an enormous publicity backlash"? It's easy to blame "the legal department" for stuff like this, but it seems extreme even for the most zealous attorney.

My hunch is that they probably have some kind of security contract with the Pinkertons to safeguard secrets, and the Pinkertons acted on their own. Hasbro/WotC has done some sketchy stuff recently, but this seems extreme even for them.

OskarS··on A collection of lock-free data structures written in standard C++11
Assuming low or no contention, it is easy to imagine a scenario where a mutex vastly outperforms it: if you need to push a 1000 things into the queue, it's still just two fences for the mutex but it's now a 1000 CASes.

Moreover: the point with mutexes is that your data structure can be the optimized assuming no thread-safety. There are lots of, like, hyper-optimized hash table variants (with all sorts of SIMD nonsense and stuff) that are just not possible to do lock-free. The very "lock-freedomness" of the datastructure slows it down enough that in low contention scenarios mutexes clearly would outperform them without being particularly contrived.

OskarS··on Random bit-flip invalidates certificate transparency log – again?
If it's this common, it should be straight-forward to test as well, right? Allocate 1 gig of memory, do whatever mmap call you need to make sure it's not paged out, fill it with deterministically random stuff, then every hour check the contents and refill it. Keep running for a month.

Seems odd to me that we have rely on some paper from 2009...

OskarS··on The seven programming ur-languages (2021)
Pretty excellent summary, this is roughly the taxonomy I have in my head.

I would maybe add SQL as an ur-language as well. It's not quite general purpose like most of these, but it should have a place in this list, I think. It has some kinship with Prolog and the declarative style, but it's really it's own thing.

You could also maybe argue for something like LabView. Many programmers look down on purely graphical programming languages, but with Houdini, Unreal's Blueprints and the various node-based shader/material systems in gamedev, I think it probably deserves it's own little branch of this family tree.

OskarS··on Bunki, a C Coroutine Library
That's really interesting, I hadn't considered the case of the compiler caching the address of errno. Obviously it can't cache the value across a function call (with or without a yield point), but caching the address does make sense, and I see how that would blow up if the coroutine was scheduled on a different thread.

I'd argue that the problem here is that you should obviously not use TLS with corutines, since they can be scheduled on any thread. The concept of thread-local storage just doesn't make much sense for coroutines. The issue isn't necessarily that there's anything wrong with coroutines, it's rather with APIs that use TLS as an information side-channel. errno is just a bad idea, basically.

But point taken.

OskarS··on Bunki, a C Coroutine Library
Do you have any specific reason to say why it will cause miscompilation? Anything in particular that makes this UB? My understanding is that as long as your functions compile to the correct ABI, it shouldn't be a problem. All you're doing is setting different stack/frame pointers before calling the function.

As for the errno thing: the whole point about coroutines is that they are cooperatively scheduled, they don't yield unless you specifically tell them to. As long as you don't insert a yield-point in between the function that sets errno and then reading the value (and you should also always read errno immediately anyway), I don't see what the problem is (except errno being a terrible API in general, but that ship has pretty much sailed).

OskarS··on Reference count, don't garbage collect
> The alice ones are moved to another generation, making the space reusable for free.

It's pretty hilarious to me that you first say "they have to move it to another generation" and then you say "it's free!" It's like saying "I paid for my dinner, and now I get to eat it for free!"

Also: what do you think `free()` does? All modern memory allocators do this trick, keeping thread-local caches. This is not an advantage of GCs when reference counting does it as well.

Almost all modern GCs are stop-the-world in at least phases, and for good reason: stop-the-world GCs are higher performance. You pay in other ways for skipping that stop.

> And atomic writes need synchronization which is crazy expensive, I honestly don’t get why you think it isn’t.

Because I've actually benchmarked it: https://quick-bench.com/q/ISEetAHOohv-GaEuYR-7MajJgTc

18.5 nanoseconds fits under no reasonable definition of "crazy expensive", not when a regular increment clocks in at 5.9 nanoseconds. And there is extremely few situations where you increment a reference count more than, like, 5 times. It's just not an issue.

This is like cargo cult programming: you've been told these things and never tested them in the real world, and you have all these preconceived notions that just don't stand up to two minutes of scrutiny.

> Also, just try writing rust/c++ code that relies entirely on RC vs Java in an object heavy workload - I really don’t think it is an open question in any shape or form.

Yes, of course garbage collectors are easier to use than reference counting. Nobody has ever disputed this. That is the whole raison d'etre of garbage collectors. This is not what the discussion is about, it's about performance.

I'm done with this thread now, unless anybody can show me any actual data to back anything you say up. It's really tiring. I started this by saying "I don't actually know", and everyone replying to me has been so darn certain of everything they say while being outright incorrect about most things, and without any actual data to back up the rest.

← PreviousPage 12 of 34Next →