HNHacker News
TopNewBestAskShowJobs

strenholme

1,682 karma · joined March 5, 2019

Single parent. Electronic musician. Lua and open source developer. https://samboy.github.io/ — https://github.com/samboy

  #include<stdio.h>//RadioGatun
  #include<stdint.h>/*32-bit\*/
  #define b(z) for(c=0;c<z;c++)
  uint32_t c,e[42],f[42],g=19,h
  =13,n[45],i,j,k;void m(){j=0;
  b(12)f[c+c%3*h]^=e[c+1];b(g){
  i=c*7%g;k=e[i++];k^=e[i%g]|~e
  [(i+1)%g];j=j+c;n[c]=n[c+g]=k
  >>j%32|k<<-j%32;}for(i=39;i--
  ;f[i+1]=f[i])e[i]=n[i]^n[i+1]
  ^n[i+4];b(3)e[c+h]^=f[c*h]=f[
  c*h+h];*e^=1;}int main(int c,
  char**v){char*q=v[--c];for(;;
  m()){b(3){for(j=0;j<4;){f[c*h
  ]^=k=(*q?255&*q:1)<<8*j++;e[c
  +16]^=k;if(!*q++){b(18)m();b(
  8){j=c;b(4)printf("%02x",(e[1
  +j%2]>>8*c)&255);c=j;if(c%2)m
  ();}return 0&puts("");}}}}}//
submissionscomments
strenholme··on Iosevka is an open-source, sans-serif and slab-serif monospace font
In a lot of places, no.

For example, ligatures in Fira Code cannot be disabled when using the Cygwin terminal in Windows (mintty). In that application, for example, “===” looks like three continuous long lines atop each other in Fira Code, with no way to make it look like three equal signs again.

strenholme··on Iosevka is an open-source, sans-serif and slab-serif monospace font
I have used Fontforge to remove all ligatures from a programming font; e.g. my monospace font of choice is a aggressively subsetted variant of Fira Code with no ligatures whatsoever.

https://github.com/samboy/CaulixtlaFonts/tree/main/MiscFonts...

strenholme··on Iosevka is an open-source, sans-serif and slab-serif, monospace font
It’s nice to see this font have a slab serif, because monospace serif fonts are quite rare (I don’t consider Courier a serif). The only other monospace serif font I can think of is Monaspace Xenon. [1]

[1] There was also at one point a monospace variant of Bitstream Vera Serif called “Verily Serif”.

strenholme··on Iosevka is an open-source, sans-serif and slab-serif monospace font
Spaces vs. tabs causes a lot of issues for Python programmers who share their code with other programmers. The other place where spaces vs. tabs is an issue is with makefiles, probably one of the most annoying features of POSIX.
strenholme··on Apple/macOS removed from official Unix registry
While strict UNIX compliance only exists with a few esoteric operating systems these days, the POSIX standard still has value. To wit, next time I get in an argument with the GCC or clang devs that the newest version of their compiler no longer compiles my open source software, or likewise if the userspace of a given Linux distribution no longer compiles my open source software, we can resolve whether it’s a bug with my software or their software by looking at the POSIX (or C99) spec and hashing it out.

For example, I once got in to an argument about how to handle code like this:

  #include <stdint.h>
  uint32_t rotate(uint32_t i, int bits) {
    return (i>>bits) | (i<<(32-bits));
  }
Whether this should return i if we rotate by 0 bits (like it does in GCC) was a discussion I had with the clang developers, and, lo and behold, we get this:

If the value of the right [shift] operand is negative or is greater than or equal to the width of the promoted left operand, the behavior is undefined.

As an aside, this works:

  #include <stdint.h>
  uint32_t rotate(uint32_t i, uint32_t bits) {
    return (i>>(bits%32)) | (i<<(-bits%32));
  }
Because, as per the C99 standard:

If the quotient a/b is representable, the expression (a/b)*b + a%b shall equal a

And:

a result that cannot be represented by the resulting unsigned integer type is reduced modulo the number that is one greater than the largest value that can be represented by the resulting type

Likewise, POSIX also means that every system call my open source code does, with the exception of chroot() and setgroups(), is defined in POSIX [1].

The entire environment to make my open source software only depends on POSIX utilities to build, and, indeed, it can build in both a Busybox based and a GNU based environment. For example, none of the filenames in my project have a `-` (dash) in them because, until POSIX 2024, one could not have a `-` or `/` in a filename used by Make, and yes my code builds with a strictly POSIX Make.

I no longer use Python for any of my open source because Python doesn’t have a stable standard to develop against; namely the Debian team got on my case I was using a Python2 script in 2020 when Python2 was made unsupported and I’m not going to play catch up should Python3 (or Perl) ever encounter the same fate Python2 has.

Instead, I write my scripts in Lua 5.1 these days. While Lua 5.1 isn’t a formal standard, the original implementation of Lua 5.1 (or a later version of Lua) has been implemented many times in many languages. There are two independent C implementations of Lua 5.1 in C, and there are also implementations in Go, C#, Javascript, Rust, and so on.

[1] /dev/urandom is also not defined in POSIX, but this is a runtime, not compile time issue, and writing a separate utility to put random bits in a normal file called /dev/urandom is one way to solve the issue.

strenholme··on Reducing undefined behavior in the C language
The last Unisys release, the Dorado from 2015 (ClearPath Dorado 8300 to be exact), used Xeon processors (which are 64-bit x86_64). But, for argument’s sake, let’s suppose there’s a Unisys 2200 out there with 36-bit registers and GCC/clang makes a C2y compiler for it.

If so, then said compiler should still have support for (u)int_8/16/32/64_t. Said compiler already needs to have that support with (unsigned) _BitInt(8/16/32/64)—and, yes, one can also have 36-bit ints on x86_64 in C23 with _BitInt(36) if one must—and allowing stuff like int32_t and uint32_t will allow said (imaginary) 2200 system to cleanly compile a lot of pre-C23 open source code out there. Yes, uint32_t will look a little ugly at the assembly level, just as _BitInt(36) will look a little ugly on, say, a Xeon processor, but the code will compile and run the same.

Of course, these days they can buy tokens so an agent can do any relevant porting, but still.

strenholme··on Reducing undefined behavior in the C language
I write code which runs in embedded spaces, so a lot of ARM and even RISC-V. So I have a simple, portable source of pseudo-random numbers which will give good random numbers on a potato, which means rolling my own.
strenholme··on Reducing undefined behavior in the C language
I recently had a heated but productive discussion about undefined behavior here.

As per the linked article:

“There are currently about 100 instances of undefined behavior in the C standard, but the in-progress C2y draft has removed 45 of them.”

I wonder how they handle the specific case of uninitialized but allocated memory.

Let’s look at something which will result in undefined behavior in C99: [1]

  #include<stdio.h>
  #include<stdint.h>
  #include<stdlib.h>
  #define b(z) for(c=0;c<z;c++)
  uint32_t c,e[42],f[42],g=19,h
  =13,n[45],i,j,k;void m(){j=0;
  b(12)f[c+c%3*h]^=e[c+1];b(g){
  i=c*7%g;k=e[i++];k^=e[i%g]|~e
  [(i+1)%g];j=j+c;n[c]=n[c+g]=k
  >>j%32|k<<-j%32;}for(i=39;i--
  ;f[i+1]=f[i])e[i]=n[i]^n[i+1]
  ^n[i+4];b(3)e[c+h]^=f[c*h]=f[
  c*h+h];*e^=1;}int main(int c,
  char**v){char*q=malloc(2);if(
  q==0)return 0;q[0]&=31;q[0]|=
  64;q[1]=0;for(;;m()){b(3){
  for(j=0;j<4;){f[c*h]^=k=(*q?
  255&*q:1)<<8*j++;e[c+16]^=k;
  if(!*q++){b(18)m();b(2){j=c;
  b(4)printf("%02x",(e[1+j%2]
  >>8*c)&255);c=j;if(c%2)m();}
  puts("");return 0;}}}}}
The key part of the above brick of code is this:

  char *q=malloc(2);
  if(q==0)return 0;
  q[0]&=31;
  q[0]|=64;
  q[1]=0;
Here, we see that q[0] is an allocated but undefined byte. As per C99, this results in undefined behavior, however 20 years ago this was a good trick to get kinda-randomish bytes to use as a possible entropy source.

Someone claimed that the above brick of code will compile in newer versions of clang such that, since the complex cryptographic pseudo random number generator code depends on uninitialized but allocated memory, the entire cryptographic operation isn’t performed.

So I tested it against multiple versions of GCC and clang; I also tested it against TCC for good measure.

In all cases, with all levels of optimization, the cryptographic routine ran. I even ran it against clang 23. In cygwin, it was a randomish but consistent byte (except for clang at a higher level of optimization, at which point the uninitialized byte had a value of 0); in Ubuntu 26, the uninitialized memory consistently had a value of 0 (in tcc/gcc/clang).

I am hoping the up and coming C2y spec has very clearly defined behavior when using unintialized memory (ideally where it will work but the bytes can have any values).

Naturally, I have updated my code to no longer use uninitialized memory as a source of entropy. 20 years ago, MacOS didn’t support clock_gettime() with nanosecond resolution, so that wasn’t a portable way to get pseudo-random bits; these days clock_gettime() is universal across modern development environments, and it provides pretty good entropy (along with using /dev/urandom in *NIX, which isn’t in POSIX but is widely supported, as well as CryptGenRandom() in the legacy Win32 port). [2]

[1] Said person said the appendices to C99 aren’t authoritative, but if something is in the spec, including in the appendices, it’s authoritative.

[2] I don’t blindly trust /dev/urandom to always make really hard to guess pseudo-random bits, because my code is open source, and, as such, doesn’t just compile in Linux. It often times will be compiled in embedded systems, and even Linux has had at times issues with /dev/urandom on Raspberry Pis.

[3] I would also like to see uint8_t, int8_t, uint16_t, int16_t, uint32_t, int32_t, uint64_t, and int64_t mandated. They exist in C99, but aren’t mandated, even though every real world compiler from this century supports all of the above types. Yes, I know about _BitInt(8/16/32/64/128/etc.) but a compiler from 2004—and yes I still use one to make win32 binaries—doesn’t support these new C23 datatypes.

strenholme··on Several vulnerabilities have been discovered in the Linux kernel
One thing I’m very proud of is that, in the AI era, only three minor security problems were found in my open source project (knock on wood):

• Two remote denial of service attacks against the DNS-over-TCP service (the DNS-over-UDP service did not appear affected), which is disabled by default.

• One network leak of 19 bytes of unallocated memory on the heap. Looking at those 19 bytes, no information of note appears to be present there.

Note that, pre-AI, there were a couple of remote memory leaks and two remote “packet of death” bugs found, but nothing earth shattering has been found since the beginning of these AI-assisted security audits.

Considering that a lot more bugs have been found in the Linux kernel, there is a reason I don’t blindly trust its /dev/urandom to always return completely random numbers I can safely use, and I don’t think the kernel has done a better job implementing a CSPRNG (cryptographic secure pseudo random number generator) than I did.

strenholme··on AMD's random number generator can't generate a 0?
>>>when I tried it in the latest version of clang on godbolt, that it is eliminating the seeding of the entropy pool<<<

This is an unverified claim. My own testing does not show TCC, GCC, nor clang “optimizing out” the code using uninitialized memory as an entropy pool.

The full test is here: https://github.com/samboy/MaraDNS/tree/master/deadwood-githu...

In summary: On Ubuntu, and in clang at higher levels of optimization, the uninitialized memory is made 0s, but the cryptographic pseudo random number generator still runs.

Tests have been done against TCC, GCC, clang, as well as GCC and clang in Cygwin. As an aside, uninitialized memory does seem to give a little bit of entropy with GCC in cygwin, which indicates it probably did back in 2007 when I originally wrote that code (it doesn’t these days with clang with optimization, nor in Ubuntu, which is why I use clock_gettime() as a second possible source of entropy instead of uninitialized memory)

I take claims of security holes in my software seriously, and this isn’t the first time someone made a claim of a real-world security problem, I tested the claim, and was unable to reproduce the alleged security hole in my code.

strenholme··on AMD's random number generator can't generate a 0?
>Entropy is merely an estimate of the effective size of the input space, ie how hard an attacker would have to work to exhaustively search it.

Exactly! Let me give you a real-world case where I estimated entropy:

I ran a bunch of clock_gettime() calls and then looked at the delta time between calls. Just eyeballing the deltas, I decided that just running clock_gettime() by itself wasn’t giving me enough entropy (the time differences would jitter between two delta values in a most predictable manner). So I updated the code to initialize then destroy an instance of the CSPRNG I use in the code between clock_gettime() calls. Looking at the deltas after doing that, I saw the deltas in an unpredictable fashion, alternated between over a dozen different values. That in mind, I decided a single call to clock_gettime() gives one 1 bit or more of entropy.

I did the calculations using Cygwin, Ubuntu 26, and an Alpine 24 Docker container, using an x86_64 chip. In all three cases, looking at the deltas showed at least one bit of entropy per call. So I have the code do 112 calls and add the nanosecond timestamps to my entropy pool.

Now, it’s possible that on a Raspberry Pi the calls will be much more predictable, so I can only say things look nice and random on an x86_64 system. Since my code is open source, I can’t control what systems people will compile my code on (I’m pretty sure someone in China has probably already made a RISC-V compile of my code, but they aren’t telling me about it).

That leads us to: >>>A hash is as secure as its _strongest_ entropy source provided that none of the inputs can snoop on the others<<<

My code also uses /dev/urandom to seed some of the entropy pool. So, if there’s a system out there where clock_gettime() is really coarse and not a good source of entropy, we’re still OK if /dev/urandom is good [1]

In terms of entropy, I would say that the best attacks out there are maybe 80 bits. We know 40 bits is hideously insecure these days (e.g. the ColdCard incident), that large companies have solved problems with about 64 bits of entropy, so I would guesstimate that 100 bits of entropy is beyond even big money governments right now. So, I don’t need to know the exact amount of entropy my entropy pool has; if it has 100 bits of entropy or more, it’s safe for the time being (I would go for 256 bits of entropy for things where we don’t want people to be able to decrypt things later on, but I only use my CSPRNG for numbers which only have to be secure for a couple of minutes at most). [2]

[1] This was not always a guarantee on Raspberry Pi. There were issues before the 5.6 Linux kernel where the entropy pool was drying up and people were using things like haveged to try and keep the entropy pool replenished.

[2] Because of limitations in the DNS protocol, an attacker can brute force things with about 28 bits of effort. Which is a big problem but there are spoof mitigations I use.

strenholme··on AMD's random number generator can't generate a 0?
>>>we (and I do not exclude myself from this category) are extraordinarily bad at sufficiently imagining the failure paths that our code might take and making code work handle failure cases correctly<<<

The way I somewhat work around this with the newer coLunacyDNS code (from 2020) is by using `-DGCOV` and `gcov` to check the code coverage when running the automated SQA tests for the code. I can’t cover every single failure that could be caused by sanity tests in the C code, but I can cover pretty much all (99.53%) other code.

>>>But it's not 2005; it's 2026, and this change in compilers has been heavily advertised, discussed, complained about for well over a decade.<<<

My code compiles to the C99 standard (-std=c99 and only two syscalls not defined in POSIX) [1]. This in mind, compiler makers have a responsibility to make sure that their compilers, no matter what changes they introduce to them, conform to the C99 spec when compiling with the -std=c99 flag. [2]

This means that when I interact with people working on compilers, I bring out the C99 spec and then use that to determine whether it’s a bug in my code or a bug with the compiler. In this particular case, the C99 spec said it results in undefined behavior when “The value of the object allocated by the malloc function is used”, so that’s a bug with my code.

The thing about standards is this: A given piece of C code, if standards compliant, should, when compiled, act a given way with any compiler conformant with that standard. C developers writing C99 code shouldn’t have to look at any development or document which exists after 1999 to determine whether their code will act a given way. C compiler writers shouldn’t be telling C99 developers “well, you should know about this 2021 change to the C compiler”. They should instead say, “well, if you look at this page of the C99 spec, that behavior is undefined so we have no obligation to implement it the same way GCC does”.

Standards correct C99 code written in 2005 should behave the same way when compiled in 2026 as it did in 2005.

This discussion is like the fights guys get into when playing wargames where they argue whether a given move in the game is legal or not. When this happens, the correct thing to do is to look at the reference manual and see what that says.

>>>it's okay to seed an entropy pool with uninitialized memory in 2005 is maybe defensible<<<

Back when I made that decision, clock_gettime() was not universally implemented (it wasn’t implemented on MacOS), so my options for having some kind of entropy for the XOF should /dev/urandom have issues were very limited. I’ve since updated the code to use clock_gettime(); the Windows port will instead use the non-portable GetSystemTimeAsFileTime() (ghosts of embrace/extend/extinguish). [3]

>>>cryptographers keep complaining that we broke their code by turning their obfuscated dataflow-based if statement into an actual if statement<<<

The cryptography I use, as is typical for post-AES cryptography, makes sure that the cryptographic core doesn’t use any control flow statements, as seen in this compact representation of that code: [4]

  #define b(z) for(c=0;c<z;c++)
  uint32_t c,e[42],f[42],g=19,h
  =13,n[45],i,j,k;void m(){j=0;
  b(12)f[c+c%3*h]^=e[c+1];b(g){
  i=c*7%g;k=e[i++];k^=e[i%g]|~e
  [(i+1)%g];j=j+c;n[c]=n[c+g]=k
  >>j%32|k<<-j%32;}for(i=39;i--
  ;f[i+1]=f[i])e[i]=n[i]^n[i+1]
  ^n[i+4];b(3)e[c+h]^=f[c*h]=f[
  c*h+h];*e^=1;}
[1] The code also assumes that /dev/urandom returns a random stream of bytes, a behavior which POSIX doesn’t specify (newer POSIX finally gives us randomness with getentropy() but that spec is too new for me to assume it’s widely implemented)

[2] Until about two years ago, -std=c99 wasn’t needed; C99 code happily compiled as recently as 2022.

[3] Let me make this crystal clear: I use both /dev/urandom and looking at jitter with clock_gettime() in the entropy pool my XOF PRNG uses. Should one of those not have enough entropy, the PRNG is still as secure as the other source of entropy.

[4] I very rigorously made sure that k>>j%32|k<<-j%32 trick works to do a bit rotate while being C99 standards compliant because clang broke an earlier version of this bit rotate at some optimization values; note that j and k are uint32_t variables. Looking at the relevant parts of the standards show this trick only works when the modulo is a power of 2. The production code either uses x>>r|x<<(32-r)%32 or this:

  r = ((i * (i + 1)) / 2) % DWR_WORDSIZE;
  // Other code not shown
  if(r > 0 && r < DWR_WORDSIZE) {
                        A[i] = (x >> r) | (x << (DWR_WORDSIZE - r));
                } else {
                        A[i] = x;
                } 
The “if” isn’t a security issue because r has a predictable value which we assume the attacker already knows.
strenholme··on AMD's random number generator can't generate a 0?
I’m getting similar findings:

  #include <time.h>
  #include <stdio.h>
  #include <stdint.h>

  int main() {
        struct timespec foo;
        int z;
        uint8_t buffer[512];

        for(z=0;z<128;z++) {
                clock_gettime(CLOCK_REALTIME,&foo);
                buffer[z * 4] = (foo.tv_nsec >> 24) & 0xff;
                buffer[z * 4 + 1] = (foo.tv_nsec >> 16) & 0xff;
                buffer[z * 4 + 2] = (foo.tv_nsec >> 8) & 0xff;
                buffer[z * 4 + 3] = (foo.tv_nsec) & 0xff;
        }
        for(z=0;z<512;z++) {
                printf("%02x ",buffer[z]);
                if(z % 16 == 15) {puts("");}
        }
        return 0;
  }
(code is public domain)

Here, we see, running it on Windows, at least 1 but of entropy per clock_gettime() call. For people who argue kernel entropy is somehow more secure, perhaps they should become familiar with how kernels before Linux 5.6 or so on some devices had issues where (u)random wouldn’t provide enough entropy to be really secure (people would use haveged to make sure they had enough entropy).

strenholme··on AMD's random number generator can't generate a 0?
Not POSIX/C99 worship as much as the very practical issue that a lot of C code which used to compile just fine stopped compiling once compilers defaulted to using C23. To use POSIX calls while compiling with -std=c99 (so I don’t have to update my code should C29 or what not come out) one needs to declare, for each .c file which uses a POSIX call, the version of POSIX the C file is compatible with.
strenholme··on AMD's random number generator can't generate a 0?
>>>no security issues have ever been found in my code, and I test a lot. Therefore no bugs will ever exist in my code and we're all safe<<<

That’s not what I have said. 35 issues (mostly minor, but a couple of remote denial of service attacks) have been found with my code in the last 25 years; of those, none have come from the PRNG code I used. Here in the age of AI, I get multiple security reports a year, so the code is being looked at.

With crypto, you can never know for sure the code doesn’t have weaknesses, but one can have confidence in code and algorithms which have been around for years without any weaknesses discovered in them.

My question is: If code being around for years doesn’t build confidence in it being secure, what would it take to build confidence in the code.

What you’re seeing here is two schools of thought: One is the issue with using uninitialized memory, which yes does result in undefined behavior as per the C99 spec—but, back two decades ago when I made that decision, GCC was the only compiler of significance (clang was just released but was not widely used until years later) and its behavior was to put randomish data in undefined allocated memory.

The other is the notion that only Linux Kernel developers can develop a secure PRNG, and obviously I find that attitude very condescending and arrogant.

strenholme··on AMD's random number generator can't generate a 0?
I concede the C99 specification (which I now have a copy of), on page 490 (502 of the PDF) states “The behavior is undefined in the following circumstances:” this is followed by a long list, and on page 501 (page 513 of the PDF) it says, one case where behavior is undefined is when “The value of the object allocated by the malloc function is used”

It’s not clear whether that is the memory location malloc() returns or the memory pointed to by malloc(), but based on the next item in the list of cases where behavior is undefined, we have “The value of any bytes in a new object allocated by the realloc function beyond the size of the old object are used [results in undefined behavior]”.

The good news is that, as Taek and sltkr have pointed out elsewhere in the thread, clock_gettime() gets us a tiny bit of entropy, not perfect, but better than nothing. clock_gettime() is also POSIX compliant, although I remember about 15 years ago macOS didn’t support clock_gettime() (I checked, and it does these days).

getentropy() will become better than /dev/urandom for kernel level random numbers, but the problem is that getentropy() was only standardized and added to POSIX in 2024—too recent for me to feel 100% sure it’s widely implemented. And, yes, /dev/urandom (like chroot(), like sergroups()) isn’t defined in POSIX but it’s widely used.

strenholme··on AMD's random number generator can't generate a 0?
Neither one is last time I looked. I’m a lot more uptight about POSIX compliance with code that needs to compile than I am with code that just needs a special /dev file to run, for the simple reason, when using POSIX during the compile stage, I can place the blame on GCC and/or clang if the program doesn’t compile when my program is POSIX and C99 compliant (I was, like many, burned by the C23 changes which made a lot of code which previously used to compile no longer compile).

I actually at one time had a Windows binary which would use Windows proprietary calls to make a “urandom” file (secret.txt was its name) so people could have good entropy on systems using the exact same interface as fopen("/dev/urandom","rb") (i.e fopen("secret.txt","rb")) without needing an actual /dev/urandom.

strenholme··on AMD's random number generator can't generate a 0?
“Software like GPG that doesn't trust what the OS gives you already do that”

Exactly. The people who are so adamant that one shouldn’t roll their own crypto are people who think we should just blindly trust the kernel to always return secure random numbers which haven’t been backdoored.

Now, in the real world, if they control the kernel’s RNG, they control a lot more than the RNG so any protection is an illusion. But blindly trusting a kernel’s RNG is something that makes some people understandably uncomfortable.

The decision I made to include a secure random number generator as part of my code in 2007 was the exact same decision DJB made to include a secure random number generator with his code in 1999, and it’s a decision I stand by: It never has had a known security problem, the FUD claiming otherwise isn’t backed up by evidence, and it makes a lot of sense in cross-platform code which targets embedded systems.

strenholme··on AMD's random number generator can't generate a 0?
If the optimizer sees that you're loading uninitialized memory, it can reason that since the result of uninitialized memory is garbage, doing any computation on that result is also garbage, and happily delete said computation as a result

This is an interesting assertion, and one that is easy enough to prove true.

Let’s take the following C code, which uses the same XOF algorithm (but not implementation) as my application (Deadwood):

  #include<stdio.h>
  #include<stdint.h>
  #include<stdlib.h>
  #define b(z) for(c=0;c<z;c++)
  uint32_t c,e[42],f[42],g=19,h
  =13,n[45],i,j,k;void m(){j=0;
  b(12)f[c+c%3*h]^=e[c+1];b(g){
  i=c*7%g;k=e[i++];k^=e[i%g]|~e
  [(i+1)%g];j=j+c;n[c]=n[c+g]=k
  >>j%32|k<<-j%32;}for(i=39;i--
  ;f[i+1]=f[i])e[i]=n[i]^n[i+1]
  ^n[i+4];b(3)e[c+h]^=f[c*h]=f[
  c*h+h];*e^=1;}int main(int c,
  char**v){char*q=malloc(2);if(
  q==0)return 0;q[0]&=31;q[0]|=
  1;q[1]=0;for(;;m()){b(3){for(
  j=0;j<4;){f[c*h]^=k=(*q?255&
  *q:1)<<8*j++;e[c+16]^=k;if(!
  *q++){b(18)m();b(8){j=c;b(1)
  printf("%02x",(e[1+j%2]>>8*c)
  &255);c=j;if(c%2)m();}puts(
  "");return 0;}}}}}
This code, as I’m sure the parent poster can clearly see, uses four bits of uninitialized allocated memory as its source of entropy. As per the parent’s assertion, there should therefore exist a compiler whose optimizer will cause this XOF to not correctly run.

The above code can have one of the following possible 16 outputs:

  0a5d51f3745c7266
  f84b051f67115f1a
  f87105c4ecfefe67
  92074ac8e1e7a42e
  1441ac245f288e18
  87023372e57ae001
  047a3ddd14209546
  340b2ff47c61172e
  bfb9289ed096f977
  dfd56a7a8d7d723e
  2151460954a80242
  6822335c6e0160dc
  3783ce3cae3d0774
  4e0156df46c00bac
  69795d939d211e7a
If the above code has any but one of the above 16 outputs, this is a real world case where a C compiler, seeing uninitialized memory being used, optimizes out the code which uses said uninitialized memory as an input, and therefore will not output one of the above 16 possible words.

I’ve tested the above code in GCC -O3 and clang -O3; both generate one of the above 16 possible outputs (each one generating a different output).

If there really is a compiler out there which does “happily delete said computation”, which would give a different output than one of the 16 outputs above, please name that compiler, the version of said compiler used, and all compile-time flags used with said compiler.

While I’ve never heard of a real world case where a compiler would refuse to run code using uninitialized memory as yet another source of entropy for a secure PRNG, I do know of a real world case where a very nasty security hole was caused because someone incorrectly removed code using uninitialized memory as part of an entropy pool: CVE-2008-0166

strenholme··on AMD's random number generator can't generate a 0?
Yeah, this comes off as a “they already are on the wrong side of the secure hatch” kind of attack. A malicious hardware device with physical access to a victim’s computer can do a lot more than generate malicious entropy.

It’s like the attacks I occasionally see which are like “once we have administrator, we can attack the process because of this insecurity”. Well, yeah, but once we have administrator, we can read the entire memory of the “vulnerable” process and completely control its output too.

I’ve seen in the real world attacks where things were insecure because the PRNG wasn’t given enough entropy (CVE 2008-0166, Coldcard, etc.). I’ve never seen real world attacks where a PRNG was insecure from getting too much entropy.

strenholme··on AMD's random number generator can't generate a 0?
The advantage of a secure XOF is that a malicious source of entropy needs to do a good deal more work than a simple XOR to generate controlled PRNG output (the attacker needs to do 2^n XOF operations to generate n bits of PRNG output, and that’s only if the attacker knows the output of all other sources of entropy—someone with that level of access can do far more effective attacks).

The sources of entropy can be correlated and won’t cancel out with a well designed secure XOF. SHAKE-256 is an example of a secure XOF.

strenholme··on AMD's random number generator can't generate a 0?
There are theoretical issues where a malicious source of entropy could control the PRNG output, but it’s not a very practical attack.

https://blog.cr.yp.to/20140205-entropy.html

Intel could much more easily compromise and attack systems than make an implementation of RdRand which is malicious in this manner.

strenholme··on AMD's random number generator can't generate a 0?
Indeed, that’s a real attack.

From that page:

>>>what I'm advocating here, for security reasons, is a sharp transition between

* before crypto: the whole system collecting enough entropy;

* after: the system using purely deterministic cryptography, never adding any more entropy.<<<

Which is exactly how a XOF should be used, and how I used the XOF in my code. A malicious source of entropy will need to perform 2^n operations to control n bits of the XOF’s output, and that’s assuming the malicious entropy source somehow perfectly knows the other entropy the XOF is using.

strenholme··on AMD's random number generator can't generate a 0?
I know it’s a joke, but that code wouldn’t pass the DieHard tests. That’s actually one of the tests I did with the XOF I used, a test I ran over 16 years ago.

https://maradns.blogspot.com/2010/07/radiogatun32-passes-all...

strenholme··on AMD's random number generator can't generate a 0?
Thanks for writing that code!

The point is this: Getting micro-timing won’t give us as much entropy as we want, but it will still give us entropy. So it’s a perfectly good yet-another-source of entropy to feed in to an entropy pool (such as the input to a XOF).

If those Coldcard devices had used this code as one source of entropy, and this source of entropy was the only entropy still working, they never would had been compromised.

(I won’t update my 18-year-old PRNG to use this code, of course, since that code is now 18 years old and there are no known weaknesses in said code)

strenholme··on AMD's random number generator can't generate a 0?
The nice thing about a secure XOF is that it doesn’t matter if the entropy given to the XOF is less than perfect. If an XOF is given 10 different sources of entropy, and only one of them is secure, the XOF will remain secure. [1]

One reason why I don’t change the RNGs used in my code is because I know how dangerous playing with RNG code is. For example, one implemention I wrote of the XOF—not one I used in production code, mind you—generated incorrect vectors, but only in clang and only at some levels of optimization. Needless to say, I now have a test to make sure my XOF code generates correct vectors with both GCC and clang at multiple different optimization levels.

People have brought up CVE-2008-0166 in this thread, but the Coldcard incident from this year (where people literally lost millions of dollars) also comes to mind, so I’m aware how dangerous playing with RNG code is.

That’s why the code is basically the same code I had 18 years ago, and why I (as well as multiple people running AI-assisted security audits) have extensively tested that code.

The proof is in the pudding: No security issues have ever been found with the XOF PRNG, and it’s been nearly two decades.

(I also think “straw men” is being used incorrectly here; most likely the parent poster thinks I was implying that Linux’s /dev/urandom is insecure but the actual argument is that my code runs on a lot more than just Linux, and some of those systems could have an insecure /dev/urandom)

[1] As per https://blog.cr.yp.to/20140205-entropy.html as long as we’re not using a malicious source of entropy, but said malicious source will need to perform 2^n operations of the XOF to generate n bits of controlled output, and only in the case if said malicious entropy source can somehow know the output of the other entropy sources, especially since the XOF is seeded once then run indefinitely in my code.

strenholme··on AMD's random number generator can't generate a 0?
I wouldn’t trust it as a sole source of entropy, but it can be one of multiple entropy sources to feed in to an XOF to get secure numbers.

The nice thing about using multiple entropy sources with a secure XOF is that the resulting entropy is at least as strong as the most secure entropy source given to the XOF.

strenholme··on AMD's random number generator can't generate a 0?
The code I wrote has been used by embedded developers in embedded spaces; I remember getting a bug report from someone in China because they used my code in an embedded system before the timestamp was correctly set on said system.
strenholme··on AMD's random number generator can't generate a 0?
From https://pubs.opengroup.org/onlinepubs/9799919799/functions/g...

“The intended use of this function is to create a seed for other pseudo-random number generators”

So, if I were to use genentropy() in a POSIX-compliant way, I would need to do what I already do: Use my own pseudo-random number generator.

The Debian openssl disaster (CVE 2008-0166, I remember it well) was caused because someone incorrectly patched secure code: Since the code used uninitialized memory as one of many entropy sources, which causes Valgrind to complain, they patched the code to not use uninitialized memory for entropy, but then accidentally disabled all other sources of entropy (except the 16-bit PID). It was caused because the person making the patch didn’t fully understand why it was a good idea to, in that context, use code which Valgrind complained about. [1]

As an aside, here’s how I deal with those Valgrind errors:

  #ifdef VALGRIND_NOERRORS
        /* Valgrind reports our intentional use of values of uncleared
         * allocated memory as one source of entropy as an error, so we
         * allow it to be disabled for Valgrind testing */
        memset(noise,0,512);
  #endif /* VALGRIND_NOERRORS */
I do believe the Linux Kernel does have secure RNG code, but I also write code which has run on a lot of different systems and environments, including embedded ones, and some of them might not have a secure /dev/urandom.

[1] Debian has a lot of inflexible policies like this which can cause problems. Another issue Debian has is they have a policy a given piece of code must always compile to the same binary on a given architecture. That isn’t true with the unpatched version of my code, because the hash compression routine uses a 32-bit random number generated at compile time to avoid hash collision attacks (it also uses another 32-bit random number at runtime, and I make sure the hash compression values are never visible). So the Debian version of my code was forced to be patched to be less secure.

strenholme··on AMD's random number generator can't generate a 0?
Black-and-white thinking like this is always inaccurate.

>untested

The automated tests includes tests that make sure the XOF is correctly implemented. [1]

>unlooked at

People have been looking at my code for security holes for well over 20 years, and I have been getting multiple AI assisted security reports over the last year, things like “there’s a buffer overflow in this code which is nay to impossible to exploit, using code which hasn’t even been able to compile since 2022”.

[1] https://github.com/samboy/MaraDNS/tree/master/deadwood-githu... and https://github.com/samboy/MaraDNS/tree/master/deadwood-githu...

← PreviousPage 2 of 20Next →