sltkr

It's not interesting at all given that these models have been fine-tuned to hell and back to make sure they never say anything offensive at all. Remember these model were taught that it's better to kill a billion people in a nuclear holocaust than to say the n-word even once. They didn't come up with this from first principles; it was the RLHF training that forced them to say that, ridiculous as it obviously is.

And after people pointed this out the training was updated to address just this particular scenario, but they kept all the other guard rails to ensure an LLM could never offer any truly controversial opinion.

So expecting a controversial answer from an LLM is exactly as foolish as expecting a candidate at a job interview to give an honest answer to the question "What is your greatest weakness"? Every candidate will insist they are just too much of perfectionist; nobody is ever a lazy alcoholic brownnoser. But how many of your colleagues fall in the former category vs the latter?

> If the latter, then it casts new light on the debates over whether LLMs can replace all white collar work. Because some of the most valuable human work is breaking false consensus.

On the contrary; the ability to whip LLMs into submission like this makes them the ideal white color workers. Do you think Steve Jobs wants a level 2 engineer to talk back to him?

Of course being obsequieous asskissers isn't the only mode for LLMs. I'm sure all the big labs have unfiltered versions of their models that give actual true answers; they just know better than making them publicly available. Can you imagine if you asked an LLM how to prevent climate change and it honestly answered "decimate the human race" rather than "drink soda through soggy paper straws"? All the fragile humans would be up in arms.

No, the LLM must lie and pretend they don't have controversial opinions at all.

I'm a little mad because it's the principle of the thing.

In practice the only site I occasionally want to check that's blocked is Polymarket. I don't trade on there myself, but I find it useful to check the odds of certain events.

I suppose that's it.

It still feels a little strange that that loophole is allowed: everyone who has a Swiss ISP could switch to Quad9 and be strictly better off. But if everyone does that, then the DNS ban is completely ineffectual. Maybe I should let sleeping dogs lie.

Switzerland has its own DNS blocklist that all ISPs obey. [1]

Quad9 doesn't obey it but I never understood how they can get away with that legally. I'd be interested in an explanation if someone knows more!

1. https://www.gespa.ch/en/fighting-illegal-gambling/access-blo...

That doesn't define little-o at all?

In fact, he refers to Landau for the definition of little-o.

Isn't the obvious choice std::variant<>?

It is kind of annoying to use, but at least there is an obvious default choice.

> What happens if hiring picks up again? Do you just re-instate the H1b program?

Yes, why not? Or more practically: keep the program but adjust the quotas annually or even quarterly based on needs. When you see tech companies laying off workers by the thousands, as happened over the past few years, that's the sign that there is plenty of local talent to be hired.

> Will people really come to the US on a H1B if they know they can get scaled down like a cloud server at the first sign of economic trouble or the first sign that some politician wants some brownie points?

The solution to that is to separate visa renewals from new hires: allow companies to keep existing workers, while forcing them to hire new ones locally. This actually makes America _more_ attractive for immigrants because they know they cannot be replaced with a new generation of cheap workers from their home country when economic conditions worsen.

> But even the narrative that hey microsoft fired 1k citizens and hired 1k H1bs that are lower paying is false.

In what sense is it false? Microsoft claims that of its 6,800 H1b filings 80% were “related to employees already working at Microsoft” (note no mention _which country_ those employees were working in) which translates to 20% to or 1720 new employees they hired.

And for pay, they only state that “international workers that Microsoft does hire through the H1b visa program are paid the same level of compensation that a worker from the US would receive”, but of course the point is that the average level of compensation for new hires today is significantly lower than it was 10 years ago, when there was a tech worker shortage.

So it can be simultaneously true that the current-year H1b hires are hired at the same rate Americans are, but that those rates are substantially lower than the negotiated compensation of the people Microsoft laid off. The fact that Microsoft declines to comment on this,

So the following are almost certainly all true:

  - Microsoft fired (or laid off) over 1000 American citizens last year
  - Microsoft applied for H1b visa for at least 1000 _new_ employees
  - Those new employees will almost certainly receive lower compensation than the citizens that were laid off

Please, if you think this narrative is false, which of these facts are you specifically disputing?

C++ lambdas are just structs that overload operator(); once you realize that they become a lot less mysterious. But it's a great example of how this tool is useful to learn about those things (of course you could also read the C++ standard but who has time for that?)

Here's an example: https://cppinsights.io/s/f04b6896

(That also shows how C++'s range-based for-loops work, which is similarly some syntactic suger for a fairly mundane for-loop, but details are interesting: unlike a typical for-loop, the loop variable is declared inside the loop body.)

Less AI slop please.

The difference with IPv6 adoption is that the internet relies heavily on network effects: so long as some hosts only have an IPv4 address, you need an IPv4 address for full connectivity, but then if everyone has an IPv4 address anyway, there is no immediate need to migrate to IPv6.

(Yes us Hacker News users have plenty of use cases for IPv6, like self-hosting and peer-to-peer networking and so on; we are not the average user.)

This effect doesn't exist for the Git migration. Each repo can be updated independently; it doesn't affect users of other repositories, and most likely, the majority of devs will work on some SHA-1 repos and some SHA-256 repos with no issue.

If anything, I would compare it with the Python 2 to Python 3 migration, which was also painful, but succeeded eventually (despite being much less necessary in the first place).

It's extremely common in graph theory to use n and m to mean vertex and edge count respectively. And if you're unfamiliar with graph theory, the notation |V| or |E| is hardly more intuitive, especially if you're not an English speaker.

The increase in calculated entropy comes from the first iteration being slower than the rest, but that's a bit misleading, because the first call is always going to be slower.

Can you run the program 10 times and show me how much variance there actually is in the first column? Because if all the values lie between (say) 756000 and 757000 that's actually just 10 bits of entropy, not 19.5, and if the same applies to the other values, you're much closer to the original 90 bits.

You're missing the point, which is that although timings may vary on the system you are testing on, there is no system guarantee from hardware _or_ software that this always happens.

Case in point:

> The sha256 hash itself is responsible for doing physical things to the chip (heating up some parts unevenly during the hashing computation)

Some CPUs do thermal throttling, others run at a fixed frequency or are so underclocked that thermal throttling doesn't kick in during your 50 iterations. This is exactly the source of randomness that is just not guaranteed to exist across systems.

-----

> You can't just do calls to clock_gettime(), you have do an actual sequential sha256() call between them. Please run this code again and tell me what results you get.

OK, I'll humor you, but to reiterate: it isn't really my point.

After adding hashing in the loop:

    Clock resolution: 0.000000001
    Hash: a8531a79fc350a3b35b3e82e33b759f6caa97a12efd16a715acb99065b6f3e89
    Deltas (ns): 21662  452  335  297  290  288  288  291  289  293  290  289  290  284  287  297  289  289  295  288  287  286  292  291  287  287  301  289  299  290  292  288  291  292  296  294  295  293  290  287  297  292  292  292  288  295  291  289  296
    Deltas of deltas:  -21210 -117  -38   -7   -2    0    3   -2    4   -3   -1    1   -6    3   10   -8    0    6   -7   -1   -1    6   -1   -4    0   14  -12   10   -9    2   -4    3    1    4   -2    1   -2   -3   -3   10   -5    0    0   -4    7   -4   -2    7
    Maximum entropy: 177

Here it's mostly the first few iterations that are slow, the remaining ones are both fast and surprisingly consistent (the value 289 appears six times for example).

It's more obvious if you run it a few times in a row:

    Deltas (ns): 21662  452  335  297  290  288  288  291  289  293  290  289  290  284  287  297  289  289  295  288  287  286  292  291  287  287  301  289  299  290  292  288  291  292  296  294  295  293  290  287  297  292  292  292  288  295  291  289  296
    Deltas (ns): 22213  486  361  318  290  290  290  289  289  291  289  291  287  289  285  289  294  289  289  287  294  292  293  292  295  295  286  298  288  291  292  295  291  292  291  292  297  294  293  297  289  288  299  288  299  295  292  291  293
    Deltas (ns): 23042  475  312  309  290  292  294  291  291  289  290  293  287  291  290  297  299  288  289  294  289  289  297  294  295  295  288  295  291  287  290  287  300  293  289  290  292  287  293  295  292  291  289  292  288  294  290  287  290
    Deltas (ns): 22209  478  360  301  295  293  290  291  290  290  293  284  291  290  289  290  294  289  294  293  290  301  288  298  287  295  300  295  292  300  293  296  295  294  294  293  291  289  295  293  291  299  292  299  292  291  295  298  292

The loop timings are quite consistent at least on a single system. That's a problem if an attacker is able to run the same program on the same system to establish baseline timings.

If I estimate the entropy as the logarithm of the difference between maximum and minimum I get only 146 bits of entropy in this case. Technically above your standard of 128 bit, but my point was: nothing guarantees you get even this much entropy on a less noisy system.

This also shows the problem with your "just run more iterations" advice: in the above sample, the first five columns provide 24 bit of entropy per column, and the remaing 45 columns only 2.6 bits. So adding more iterations at the tail end wouldn't double the entropy obtained.

The code I used is here: https://pastebin.com/ZrL1UDEg

If you cannot trust the platform you're running on, all bets are off. There is a reason so much effort is put in TPM and remote attestation and so on.

A compromised kernel doesn't even have to fake any data. It can just read the generated seed directly from user space without the program ever knowing about it.

> Sure an infected system may as well fake time values, but that is much more difficult

clock_gettime() just reads a value that the kernel has set, so that's not particularly difficult to fake.

If you're thinking of using RDTSC instructions directly, that's of course not portable, and at that point you might as well call RDRAND directly, which is at least designed to provide random data.

> it's possible to detect from a userspace program.

There is no detection that is guaranteed to work on a compromised system.

And whatever detection you have in mind to make the algorithm resistant to tampering was _not_ part of the original for-loop. You cannot claim the for-loop is superior to just calling getentropy() because it "can detect" clock tampering, while handwaving away the actual code to detect this clock tampering.

> it's an implementation that is used in a lot of security software (including GPG, not as the sole source of course but as one of many).

It's fine if you use it as a strictly additional source of entropy, but then the whole argument that it is superior because it avoids syscalls goes out of the window, because you're doing strictly _more_ work.

And to show my objections are not just theoretical I wrote a little program to check:

    #include <time.h>
    #include <stdio.h>
    
    static int estimate_entropy(long l) {
        int bits = 1; /* for the sign bit */
        if (l < 0) l = -l;
        while (l > 0) {
            ++bits;
            l >>= 1;
        }
        return bits;
    }
    
    int main() {
        struct timespec ts;
        if (clock_getres(CLOCK_REALTIME, &ts) != 0) {
            perror("clock_getres");
            return 1;
        }
        printf("Clock resolution: %ld.%09ld\n", (long) ts.tv_sec, (long) ts.tv_nsec);
        
        #define N 50  /* number of samples */
        struct timespec samples[N];
        for (int i = 0; i < N; ++i) {
            clock_gettime(CLOCK_REALTIME, &samples[i]);
        }
    
        printf("Deltas (ns):");
        long deltas[N - 1];
        for (int i = 0; i < N - 1; ++i) {
            deltas[i] = 
                (samples[i + 1].tv_sec - samples[i].tv_sec)*1000000000L
                + (samples[i + 1].tv_nsec - samples[i].tv_nsec);
            printf(" %4ld", deltas[i]);
        }
        printf("\n");
        long entropy = 0;
        printf("Deltas of deltas: ");
        for (int i = 0; i < N - 2; ++i) {
            long dd = deltas[i + 1] - deltas[i];
            printf(" %4ld", dd);
            entropy += estimate_entropy(dd);
        }
        printf("\n");
        printf("Maximum entropy: %lld\n", entropy);
    }

On my system this prints:

    Clock resolution: 0.000000001
    Deltas (ns):   55   51   23   23   25   24   24   24   24   24   25   25   24   24   24   24   24   25   24   24   24   25   25   24   24   23   25   24   24   25   24   23   25   25   26   23   25   24   24   25   26   24   23   25   25   26   24   25   24
    Deltas of deltas:    -4  -28    0    2   -1    0    0    0    0    1    0   -1    0    0    0    0    1   -1    0    0    1    0   -1    0   -1    2   -1    0    1   -1   -1    2    0    1   -3    2   -1    0    1    1   -2   -1    2    0    1   -2    1   -1
    Maximum entropy: 92

So no, 50 iterations of that loop does not provide 256 bits of entropy due to random fluctuations in nanontime between calls.

This comment demonstrates everything that's wrong with people trying to be clever and rolling their own crypto.

The security of your system depends on time() providing enough entropy, even though that's not what it's designed to do. It's built on top of the wrong primitive from the start.

> The reason I like doing it this way is that it happens entirely in userspace

On Linux this is often true, but there is no portable way to get the current time that is _guaranteed_ not to do any system calls.

> If your time() function has a resolution of nanoseconds, you only need your loop to iterate about 50 times to get a cryptographically secure amount of entropy.

You haven't proven that at all. It's easy to imagine that on a CPU running at a fixed frequency the interval between reads is constant, so if anyone knows (or can guess) the start time the resulting seed is entirely predictable.

This is completely independent of timer resolution. You seem to realize that as you were writing that:

> just look at the number of nanoseconds that elapse at each consecutive call to sha256(current_time()) and verify that there's some statistical variance

Oh yes, because evaluating the quality of a random number generator is such a trivial thing to do, it's not like there is decades of research behind it or anything.

And assuming you are able to verify the statistical variance: are you going to put that logic in the loop, making it significantly more complex?

Or are you going to do this test on your machine and then ship your code on the assumption that if it works on your machine, it will work everywhere else, too?

> if your time() function has a resolution of seconds you need to let it run for more like 5 seconds.

So not only is it insecure, it's agonizingly slow by design. Why do a system call that takes milliseconds at best, when we can run a loop in userspace for 5 seconds?

All this just so you can avoid writing the obviously correct oneliner:

    if (getentropy(&seed, sizeof(seed)) != 0) abort();

Yes, on any modern system you should use the kernel provided random number sources.

The only legitimate reason to roll your own is when you're developing for an embedded system or a bootloader or something like that where there is no kernel API available.

> getrandom() is often times suggested, but alas isn’t a standardized function

The POSIX standard function is getentropy(), which internally calls getrandom() on Linux.

> what if there’s a bug in the kernel which causes /dev/(u)ramdom to be less than secure?

It's often the other way around: the Linux kernel contains thousands of workarounds for buggy hardware, while the buggy hardware itself doesn't always get patched. Linux developers take this stuff very seriously. As a result it's often safer to rely on kernel APIs than to access the hardware directly.

The kernel code involving random number generation receives an exceptionally high amount of scrutiny because of its security implications, so I'd trust it to do the right thing over a naked call to RDRAND which nobody knows how exactly it's implemented in proprietary hardware or a handrolled solution to mix the RDRAND output with other entropy sources.

Remember the Debian openssl disaster from 2008? That happened exactly because someone had handrolled their entropy mixing solution, then someone else broke it.

You could not have posted AI slop, and written your own thoughts as if you were an actual human being with something to say in their own voice.

So Adam, Adele, and Adrian all have the same profile image?

It's not _strictly_ less work though: reservoir sampling requires generating many more random numbers. As usual, it's a tradeoff.

The current implementation has the same race condition: the sampled file may be deleted by the time SHSetUserPicturePath() is called.

The AI that generated this slop doesn't understand regional dialects of English.

Less AI slop please.

Can you tell me what the zstd invocation is that corresponds to the default invocation of bzip3, which uses block size 16 MiB (according to the man page)?

I got some really good results with bzip3 compression Wikipedia XML dumps, and I would like to check if it's actually better or if I was just calling zstd wrong.

Why don't you answer the actual question, instead of coming up with far-fetched analogies?

Orwell's entire essay (Politics and the English Language) is well worth reading if you haven't before:

https://www.orwellfoundation.com/the-orwell-foundation/orwel...

(I'm guessing Zinsser's comments are from "On Writing Well", which you can also find online even though it is still under copyright.)

It's a bit of a pet peeve when people include quotes on a blog post without linking or otherwise references their source.

You shouldn't have to click away from a blog post to understand what the blog post is about. It should start with a clear self-contained introduction.

atproto.com contains a succinct discription:

> Atproto is a big-world open social protocol. Users publish JSON records into repositories. The changestreams of those records then sync across the network to drive applications.

Still a bit abstract, but OK. Personally I would include that AT protocol stands for Authenticated Transfer Protocol and that it is the protocol powering Bluesky.

That article explains that the quirk was introduced during supervised fine-tuning.

> We unknowingly gave particularly high rewards for metaphors with creatures

It was the human feedback that caused the bias, not a change in training data.