Assembly & Low-Level GPU Systems

The compiler is the final say on what your code actually costs — loop unrolling, inlining, vectorization, and register allocation all happen below the source level, and two source lines that look equally cheap can compile to wildly different instruction counts. Reading assembly (or GPU ISA) is a debugging tool for exactly one question: "did the compiler do what I assumed it would?"

godbolt.org (Compiler Explorer) is the fastest iteration loop for this: paste source, see assembly update live, and diff two optimization levels or two compilers side by side.

CUDA C++ compiles to PTX (a virtual, forward-compatible intermediate assembly), which the driver then JITs to SASS — the real, GPU-generation-specific machine code. HIP/ROCm compiles through LLVM straight to the AMDGPU backend's native ISA (RDNA or CDNA instructions) with no separate virtual-ISA hop. In both cases this is the layer where you can check whether tensor/matrix-core instructions were actually emitted, or whether the compiler fell back to regular ALU math.

GPU ISAs split registers into vector registers (one value per thread/lane, the common case) and scalar registers (one shared value for the whole warp/wavefront, e.g. a loop bound or a uniform pointer base). Code the compiler can prove is uniform across the warp gets promoted to scalar registers and scalar instructions, freeing vector register pressure and instruction slots — this is one of the reasons hoisting loop-invariant, thread-independent computation out of a kernel body can measurably speed it up even though "it's just moving one line."

A single hot for-loop can be simultaneously affected by:

• cache misses — a load that isn't already in L1/L2 stalls the pipeline for hundreds of cycles
• memory alignment — unaligned accesses can split into two transactions instead of one
• prefetching & latency hiding — whether the hardware (or software prefetch instructions) started the next load early enough to overlap its latency with useful work
• instruction-level parallelism (ILP) — whether independent operations in the loop body can issue back-to-back instead of waiting on each other
• pipeline stalls — a dependent instruction issued before its input is ready bubbles the pipeline
• false sharing — two threads writing different variables that happen to share one cache line ping-pong that line between cores/CUs on every write

None of these show up by reading the source code alone — they show up in a profiler's stall-reason breakdown or in the assembly/ISA itself.