Characterizing gaming workloads on Zen 4

On the other hand, while in all the previous years there has been a delay of around one year between the appearance in laptops of some Ryzen model and its appearance in small NUC-like computers, this year a large number of companies have introduced small computers with AMD Phoenix (Ryzen ? 7?40) immediately after its launch.

I am happy about this, because instead of carrying a big laptop, I prefer to carry a NUC-like computer, a 17" portable monitor and a compact keyboard, which all together are both lighter and much cheaper than a mobile workstation laptop, while being faster and more comfortable for working and also having more peripheral interfaces.

> The ones I found also have a discrete GPU which defeats the point. Having a low power draw, thin and light laptop that can also run fairly recent AAA games - amazing!

So, disable it? Most Zen4 laptops I've seen so far still fit the general thin and light category (albeit, at 15" vs 11-13") and is paired with a mid-range Radeon or GeForce (usually 4060/4070). Buy the laptop and put it in Eco mode in Windows (doesn't multiplex with discrete) or disable it in bios.

The Framework comes with a Zen4 cpu. It's thin and light and the GPU can be disabled:

https://youtu.be/L9Ww_7wwNkM?si=Zke0hCcubnwEl6oe

As someone who owns an AMD laptop (with Radeon discrete), I should point out that AMD's muxing algorithm is pretty conservative, unless you put it into performance mode. Video playback, basic GL/VK/DX operations, desktop usage/web browsing, etc sticks to the integrated GPU. I've only seen the discrete kick in once obvious heavy 3d operations were running (video games, benchmarks, 3d modeling, etc).

Yeah but Zen 5 is going to widen the front end anyway, which is one of the things they had just said was a good choice not to do ;-)

It’s definitely a thought I’ve had from time to time as well. I’m not a chip designer but my hunch is that it’s because one of the branches is likely to result in even more branches, likely very soon after that initial branch on one of the paths taken (ie you won’t get far before hitting another branch). Example is a condition that causes you to leave the loop - on every iteration of the loop you’d be trying to speculatively execute what happens after the loop.

That means you’ve got an exponentially increasing number of speculative execution happening and it’s almost all worthless (ie 1/n^2) while competing for resources with the one real path. This is ignoring the practical challenges of managing all that speculation and any fun additional speculative attacks that might arise again.

The best case for something like this is where you have a bunch of unpredictable branches. In those cases, it’s better for the compiler to convert it to “branchless” code (eg using the x86 cmov instruction).

There’s also major performance problems unrelated to branches which are data dependencies. I think those are even more impactful these days as they completely inhibit the length of a deep pipeline. Those can only really be solved by fixing code.

Some CPUs have attempted to do this, but the cost is enormous.

Conditional branches are very frequent, so a modern CPU that has much more than one hundred instructions in flight might have ten or twenty branches past which it executes instructions speculatively.

So it will not have two sides executed in parallel, but each side will be split in two at the next branch, then again at the next branch. The number of branches that must be executed speculatively in parallel grows exponentially, together with the ratio between executed instructions that must be discarded and useful instructions whose results are retained after the conditions of the branches are resolved.

So such a CPU would have a huge power consumption due to doing mostly useless work, which would reduce its performance much more than what is lost during branch mispredictions when only one side is executed.

For unpredictable branches, all modern CPUs offer conditional move or conditional select instructions, where both sides of a conditional expression are executed and then the result for the true condition is selected.

You need to double the hardware... which fair enough modern CPUs are super scalar so they already do that. The issue is that there are a lot of branches in the code, the CPU would need to keep a lot of redundant hardware. All that extra hardware comes with increased power usage and consequently heat dissipation. On top of that modern branch predictors are pretty amazing, so you would need to get a lot of benefit to make this worth it...

So the trade is, you can get slightly better latency due to misprediction masking by executing both branches at the cost of massively decreased throughput (b.c. you are using extra hardware to execute branches of the same thread vs different threads), increased power and heat dissipation, and increased cost due to additional hardware. Note that cost, power, and usage are generally the constraints you want to satisfy, so you will generally get a significantly slower CPU that probably has worse latency due to lower clocks, less cache, and whatever other tradeoffs you need to make to fit into the power/heat/cost budget.

Had the same thought, like hyperhreading but for branches. But then I read in the article that one of the games had a branch every 4-5 instructions.

Let’s look at the problem numerically!

Under your proposed scheme, from the time between a branch being issued and resolved around half the subsequent instructions executed by your CPU will be retired and the other half will be discarded. So, all else being equal, performance is 50% compared to non-branching code.

With branch prediction, after a single branch all the speculatively-executed instructions are either retired or discarded. If a branch predictor randomly guesses for each branch, it will achieve 50% accuracy...so, averaged across many branches, 50% of speculatively executed instructions are retired.

So the worst possible branch predictor has equivalent performance to your proposed scheme, and it’s quite a bit simpler to implement (which translates to increased performance). If your branch predictor is any better than blind guessing, it will achieve better performance (by the benchmark of % of correctly-speculated instructions). Given that modern branch predictors achieve 95%+ accuracy on typical workloads, it’s clear why processor designers choose branch prediction rather than executing both pathways.

As a game developer, you should consider reducing branches in shaders. Though if you're CPU bound, then sure, it's something to think about.

Ideally try to design your loops so you're accessing memory sequentially (better cache locality) and predictably (i.e. don't branch on a condition that changes in every iteration). Using a branchless algorithm is only really beneficial if the branch is constantly being mispredicted.

As always, profile first so you don't fall into the premature optimization hole.

This isn't necessarily game-specific. I'm not up to speed with current gen, but in the past the simple answer was "it depends", as usual. :-) There are times where doing some bit-twiddling hacks outperform branching, or where (partial) loop-unrolling is faster than the higher code density of a firm loop, but in the end for trivial cases, the compiler often, but not always, would do these things behind the scenes if you tell it what CPU you want to target. And if you really want the best performance in a particularly hot section, you just have to benchmark every possible implementation and pick on a case-by-case basis, or even provide two or three different implementations and pick the best one at runtime.

Yes, here's an example of a case where eliminating a branch will save you as a game developer:

    while (money > 0) {
      if (iOS) {
        // ios revenue
        money += 5;
        // ios costs
        money -= 2;
      } else if (android) {
        // android revenue
        money += 1;
        // android costs
        money -= 5;
      }
    }

Anyway, to your real question, an entity component system-based middleware, hopefully in a language that isn't C++, forces you to author your code in a way where branches can be "moved" to remove the problematic memory access and execution pattern. So maybe it's not about specific C++ method branches, which may be beneficial to your employer but never beneficial to you or your career, but it's about the architecture of the thing that still has innovation opportunities that could really matter.

You probably aren't interested in the even bigger picture conversation, which is that some games are so ill specified that it is impracticable to use an ECS middleware. Games that are really well specified (i.e. clones & sequels) might thrive on ECS, but the ones we end up playing are authored by companies with real budgets and take years to develop, so the compute platform will get faster than your micro-optimization.

TLDR: worry about branches in your hot loop kinda last and only if the profiler indicates you have a random walk through your branches AND you’ve verified the compiler emitted branches. There are more impactful optimization techniques to worry about first and often with PGO and LTO the compiler is going to be a lot better about making relevant code branchless on your behalf without you having to think about it.

I’d say that’s something you should only do once you understand your architectural bottlenecks. These are bottlenecks that appear because of how you’re doing computation and moving data around. They won’t show up on any profiler because the profiler is telling you “this is where an implementation is slowest” and not “there’s a better implementation altogether” - the latter is art as it requires a good working understanding of computer architecture and playing around with algorithms that aren’t taught in text books (or adapting them more optimally for your problem domain).

If you’ve done all other major architectural optimizations and you’ve identified a hot loop where the compiler is inserting branch code which is mispredicted, an explicitly branchless version will help. But remember - compilers are very good often at recognizing the opportunity for a branchless version and changing your code to that, so if you’re in AOT land with a major compiler (clang/llvm, gcc, msvc, Intel) there’s a good chance your algorithm might already be branchless. In fact, if you use something like PGO on a realistic workload, the compiler is even more likely to emit the correct machine code in way more places without you having to manually alter code by hand.

More impactful low level optimizations often are:

Optimizing for cache locality through something like entity component systems (ECS). Basically using a struct of arrays and no polymorphism instead of an array of polymorphic structs. This is a popular one for games and works really well. Similarly, make sure that data being accessed in a hot path is all linearly laid out in processing order if possible.

Applying SIMD to your hot loop processing. The compiler does have autovectorization that can work well with ECS but requires you to write your scalar code carefully to trigger it (ie your scalar code has to look similar to the vector version in important ways). Check the assembly to see if the compiler is doing it and rewrite using compiler intrinsics when it’s not. Similarly try to minimize data dependencies between your loops. Eg instead of

    sum=0
    foreach x in arr
      sun += x

Try

    sum0 = 0, sum1 =0, … sum7 = 0
    foreach (x0, x1, …, x7) in arr
      sum0 += x0
      …
    sum = sum0 + sum1 +…

That’s very very similar to what a SIMD loop would look like and a compiler is likely to recognize it. Even if it doesn’t autovectorize, you’re now doing 8 summations “concurrently” because there’s no data dependency for those 8 additions on every loop iteration, so the CPU will execute them without waiting for the result of any other summation. Make it unrolled enough to exhaust the execution units and you’ve guaranteed that the beginning of the loop will start after the previous data dependency has been resolved (ie the CPU will be executing that loop at maximum speed). Again, this kind of stuff is silly to worry about until you figure out where your hot loop is for a given implementation.

Knute writes “premature optimization is the root of all evil”. Implementation optimizations can get you very small improvements once you’ve exhausted low hanging fruit (basically implementation mistakes). Switching implementations can often net you order of magnitude improvements.

Looking into branchless is probably often premature. Organizing your data efficiently and using good algorithms is not. Data dependencies in loops is probably in the middle there - they optimize your current implementation but the improvement for that hot loop can be drastically more significant than a branchless design (fixing data dependencies can easily 10x your performance, making branchless for a mispredicted loop can net you maybe 20% or so for that hot loop).

You should run your critical loops through perf or a similar tool and get statistics for branch misprediction. If you have a lot of misprediction, you should think about doing branchless stuff. If you don't have a lot of misprediction, the branchy code will be faster.

Sometimes you can guess whether the data the loop will hit will be predictable or not. The same applies; if you expect it to be predictable, then don't try to remove branches. If you expect it to be unpredictable, try to remove branches.

Note that you will need to teach yourself your expectations; the branch predictor might be better or worse than you expect it to be. Modern branch predictors are extremely complex, often taking more die space than everything except cache. Don't assume that just because you can't see a pattern in the data, the branch predictor won't be able to.

Branches are still valuable in many cases but it can be valuable, situationally, to replace them with things like cmovs instead. You'll need to benchmark it to be sure - it's not like the old days where branches were always worse.

Branch hoisting, on the other hand, is a win like 90% or more of the time (I can imagine it being a loss if it causes stuff to fall out of icache), which is why most compilers will try to do it automatically.

Worse than branches (generally) are indirect calls (i.e. virtual methods, function pointers) that don't predict well, so you especially want to avoid those in tight loops and if you can't avoid them you want to sort your data so that the call target doesn't change frequently.

Only if they are poorly predicted.