boulos

I kind of love that this continues to haunt you :).

Depends on what you're doing. The main issue here is reductions / accumulations.

That is, if you have a bunch of floats like:

  float sum = 0.f;
  for (int i = 0; i < N; i++) {
    sum += x[i];
  }

and that you vectorize that to something like (typing in this comment, errors are likely):

  __mm256 sum_8wide = _mm256_setzero_ps();
  for (int i = 0; i < N/8; i++) {
    sum_8wide = _mm256_add_ps(sum_8wide, _mm256_load_ps(x[8*i]);
  }
  // Now sum up the 8 values to get the final sum
  float sum = _mm256_hadd_ps(...);

that will result in a different accumulation than if you did those are 4-wide and then a reduction. The usual solution is to either use the lowest common denominator (e.g., use AVX instead of AVX2) or more performance oriented, use the 4-wide SIMD units on ARM to "emulate" an 8-wide virtual vector (~15 years since I wrote NEON... and again, this is in a comment):

  float32x4_t sum_lo = ; // zero;
  float32x4_t sum_hi = ; // zero;
  for (int i = 0; i < N/8; i++) {
    sum_lo = vaddq_f32(sum_lo, vload1q_f32(x[8*i));
    sum_hi = vaddq_f32(sum_hi, vload1q_f32(x[8*i + 4]));
  }
  // Reduce the sum in the same order

You would want to write a "virtual SIMD" wrapper library, so you don't do this manually in lots of places.

The same reasoning applies though. The compiler is just another program. Outside of doing constant folding on things that are unspec'ed or not required (like mmsqrtps and most transcendentals), you should get consistent results even between architectures.

Of course the specific line linked to in that GH issue is showing that LLVM will attempt constant folding of various trig functions:

https://github.com/llvm/llvm-project/blob/faa43a80f70c639f40...

but the IEEE-754 spec does also recommend correctly rounded results for those (https://en.wikipedia.org/wiki/IEEE_754#Recommended_operation...).

The majority of code I'm talking about though uses constants that are some long, explicit number, and doesn't do any math on them that would then be amenable to constant folding itself.

That said, lines like:

https://github.com/llvm/llvm-project/blob/faa43a80f70c639f40...

are more worrying, since that may differ from what people expect dynamically (though the underlying stuff supports different denormal rules).

Either way, thanks for highlighting this! Clearly the answer is to just use LLVM/clang regardless of backend :).

That blog post is now a decade old, but includes an important quote:

> The IEEE standard does guarantee some things. It guarantees more than the floating-point-math-is-mystical crowd realizes, but less than some programmers might think.

Summarizing the blog post, it highlights a few things though some less clearly than I would like:

  * x87 was wonky

  * You need to ensure rounding modes, flush-to-zero, etc. are consistently set

  * Some older processors don't have FMA

  * Approximate instructions (mmsqrtps et al.) don't have a consistent spec

  * Compilers may reassociate expressions

For small routines and self-written libraries, it's straightforward, if painful to ensure you avoid all of that.

Briefly mentioned in the blog post is IEEE-754 (2008) made the spec more explicit, and effectively assumed the death of x87. It's 2024 now, so you can definitely avoid x87. Similarly, FMA is part of the IEEE-754 2008 spec, and has been built into all modern processors since (Haswell and later on Intel).

There are still cross-architecture differences like 8-wide AVX2 vs 4-wide NEON that can trip you up, but if you are writing assembly or intrinsics or just C that inspect with Compiler Explorer or objdump, you can look at the output and say "Yep, that'll be consistent".

Maybe ask them to make you "an Americano, but with cold brew coffee"? Trick them into it being an on-the-menu item.

Agreed. I should have said so, too.

At the very least, I assume .+ at the start is a better choice.

The RFC says:

     addr-spec   =  local-part "@" domain        ; global address
     atom        =  1*<any CHAR except specials, SPACE and CTLs>
     word        =  atom / quoted-string

so I think the bit in front of the @ has to be non-empty.

> If you already have a network and a decent platform to communicate with people, as well as the motivation to go at it for yourself, you're giving up a huge chunk of the company for a pittance.

But that's literally the point. You suddenly get a huge network that would help you get through the early stages.

It's certainly true that many founders would find better terms elsewhere. However, it's also true that most companies fail. So negotiating better terms usually doesn't matter nearly as much as making the company more successful. (The same rationale applies to employee equity grants)

Like you said, for people with no connections, whether just out of school or not, it's valuable.

Example: https://e-estonia.com/solutions/ease_of_doing_business/e-bus...

This is also directly available from MIT Press as Open Access: https://direct.mit.edu/books/oa-monograph/5598/Balkan-Cyberi...

That isn't what that blog post is saying.

It's saying that the author's invented metric may indicate that the studies within each chapter may not replicate. No actual replication studies were done to produce the table in that post.

How about "weights available" as similar to the "source available" moniker?

JIT'ed code is sadly poorly supported, but LLVM has had great hooks for noting each method that is produced and its address. So you can build a simple mixed-mode unwinder, pretty easily, but mostly in process.

I think Intel's DNN things dump their info out to some common file that perf can read instead, but because the *kernels* themselves reuse rbp throughout oneDNN, it's totally useless.

Finally, can any JVM folks explain this claim about DWARF info from the article:

> Doesn't exist for JIT'd runtimes like the Java JVM

that just sounds surprising to me. Is it off by default or literally not available? (Google searches have mostly pointed to people wanting to include the JNI/C side of a JVM stack, like https://github.com/async-profiler/async-profiler/issues/215).

I'm not sure which external or internal product you're talking about, but there are no networks involved for Local SSD on GCE: https://cloud.google.com/compute/docs/disks/local-ssd

Are you referring to PD-SSD? Internal storage usage?

It took me a while to realize digital twin got autocorrected to digital town here :).

Queen Elizabeth II oversaw a lot of the final decline of the British Empire and Commonwealth. She seemed okay with it.

I'm still (poorly) maintaining it!

Eh, c6a is also an AMD Rome which has worse memory bandwidth at the tails and weaker per thread performance than Cascadelake (c5). I don't understand anything about this particular benchmark, but I wouldn't compare them simply as "older vs newer".

Do you have a version using the same instance type? Seems weird to mix them.

You could imagine sanctions on Brazilian beef. Either a blanket ban or something targeted at particular producers. I doubt it would happen, but that's a different plan.

https://www.publicpower.org/system/files/documents/Americas_... via https://www.publicpower.org/resource/americas-electricity-ge... disagrees with you even in the last decade. And the future seems mostly solar:

> This report also analyzes prospective generation capacity in four categories — under construction, permitted, application pending, and proposed. More than 466,000 MW of new generation capacity is under development in the United States — a 13% increase over 2022. Sixty-one percent of capacity most likely to come online, permitted plants and plants that are under construction, are in solar.

China's growth in power capacity is non-trivially due to increasing demand. If the US or Europe or wherever suddenly wanted to build XXX GW per year, they could (modulo bureaucracy, which is very real).

It's unfortunate that no math is ever done in these stories.

If you take the "350,000" H100s that Facebook wants by EOY, each of those can do 700W, which gives you almost 250 MW for just the GPUs. That sounds like a lot, until you realize that a single large power plant is measured in Gigawatts. All of Google's data centers combined are O(10 GW) which are matched with renewable power offsets [1].

Importantly, the world installed >500 Gigawatts of renewable energy in 2023 [2], mostly driven by PV Solar in China. The amount of potential solar and wind and other renewable-ish (hydro) outstrips even a 10x'ing of a lot of these numbers. But even for a single site, dams like Three Gorges are >20 GW.

There are real efficiency and scale challenges in doing AI in a single, large site. But existing power generation systems deliver plenty of power.

[1] https://www.gstatic.com/gumdrop/sustainability/google-2023-e...

[2] https://www.iea.org/reports/renewables-2023/executive-summar...

Most of the world lacks this product. In the UK, for example, I think the most popular product is what we'd call a 2/1 ARM!

I made a simple bash alias for this:

  mknote () {
   emacs -nw $HOME/notes/`date "+%Y-%m-%d-"`${1}.txt
  }

so that I can do:

  $ mknote FooTopic

and start recording things.

I also have enote:

  enote () {
   local newest=$(ls -Ft $HOME/notes/\*.txt | grep "$1" | head -n 1) 
   emacs -nw $newest
  }

to go back and edit them by name.

Movies yes, but often a single episode of a show is say $2. That means if you watch a season in a month, you're up to $16-20 pretty quickly with 8-10 episodes.

I like the term parsley code!

I do suspect though that both the hashing and 6-bit weight examples are just extremely rare in the corpus. It wasn't confused about loops, or hashing generally, but just didn't do as well as antirez would have liked. The description of the 6-bit to "why don't I just cast this to 8-bits" thing is definitely a problem. And worse, it's a problem a more junior engineer might not understand. But I suspect that a model trained on a corpus with lots more bit manipulation would have been fine, as it wasn't complex.

Clearly we just need a fine tuned one :).

I really like the argument about misinformation vs testing. I'm not totally sold on "you can just see it", but I do think something like TDD could suddenly be really productive in this world.

I've found autocomplete via these systems to be improving rapidly. For some work, it's already a big boost, and it's close to a difference in kind from the original IntelliSense. Amusingly though, I primarily write in an editor without any autocomplete, so I don't experience this often. But I do, precisely for the throwaway code and lower-value changes.

Finally, it's not clear to me that the distinction is between systems programming and scripting. My sense is that Chat GPT and similar are (a) heavily influenced by the large corpus of Python, so it's better at it than C and (b) the examples here involved more clever bit manipulation than most software engineers ever interact with.

Let's call it "production order" then. Firefly was famously broadcast out of order.

> I believe Waymo has had issues where several AVs occupied the same street after an event and blocked traffic because they couldn't decide what to do - they were waiting on each other to behave like a human.

You're thinking of some of the Cruise incidents like https://m.youtube.com/watch?v=kyAX28dapps .

> And a minor but still important caveat is that SF and Phoenix have modern linear grids

San Francisco has some amusing areas as well. If you look around the streets near Buena Vista park or even Twin Peaks, you'll see plenty of funky curves, six way "intersections" and so on. Lots of two-ways that are only lane wide.

Like the older parts of Boston and Cambridge, it's a holdover from the days of horses and the geography. That's where the paths were, people built houses, and now you've got to deal with it :).

I linked to it in https://news.ycombinator.com/item?id=38722356 .