SASS2MLIR findings – ~20% to 100%+ GPU performance [Nvidia] improvements

The project explores GPU optimization at a layer below conventional framework- and compiler-level tuning, including analysis and transformation of the machine code ultimately executed by the GPU. Across our testing so far, we have observed ~20% to 100%+ performance improvements, depending on the architecture, kernel, workload, and execution conditions.

Testing has included: - NVIDIA architectures spanning Ampere through Blackwell - Jetson Orin Nano, Orin NX, and AGX Orin - Individual instruction and microbenchmark testing - Kernel-level benchmarking - Model and workload-level testing - Comparisons against conventional execution paths, including CUDA Graphs in applicable tests

One of the areas we are particularly interested in is the optimization opportunity that exists after traditional compilation has already taken place.

Our broader work looks at analyzing the final GPU machine code, identifying architectural and execution inefficiencies, and dynamically modifying the execution path while maintaining numerical correctness.

This includes areas such as instruction scheduling, memory behavior, register utilization, execution dependencies, architecture-specific instruction behavior, and increasingly runtime kernel optimization and dynamic kernel fusion.

The interesting result for us is that the performance opportunity is not limited to a single GPU generation or workload type. We are seeing measurable opportunities across both datacenter-class GPUs and constrained edge platforms such as Jetson, although the magnitude of the improvement varies considerably with workload characteristics and hardware limits.

These are still initial findings, and we are continuing to expand the benchmark coverage and validate the methodology across additional models, architectures, and workloads.

For anyone interested in the deeper engineering details, we have published a technical explanation of the discoveries, methodology, and underlying work here: https://mbuchel.github.io/projects/sass2mlir/

For a Blackwell and Jetson testing you can go to https://github.com/mbuchel/sass2mlir-bench and watch https://youtu.be/cBfBWGG3Vas

Jetson Orin Nano — Qwen3.5 4B Standard baseline: 10 → 21 tok/s (+110%) CUDA Graphs baseline: 16 → 21 tok/s (+31.25%)

Jetson AGX Orin — Nemotron 3 Nano 4B 31.2 → 40.5 tok/s (~30%)

Jetson AGX Orin — Qwen3.5 4B 25.0 → 31.0 tok/s (+24%)

Technical feedback, criticism, and discussion are very welcome. Michael Buchel, hacker news user “mbuchel-hn”, will be available for live questioning for an hour; 10:00AM Eastern to 11:00AM Eastern.