GPU Architecture Explained: GPU Cores, Shader Units, CUDA Cores, Stream Processors, Compute Units, SIMT, Ray Tracing, VRAM and Modern Graphics Processing

Meta Description

Understand GPU architecture and how modern graphics processors deliver massive parallel computing performance. Explore GPU cores, shader units, CUDA cores, stream processors, compute units, streaming multiprocessors, SIMT, warps, wavefronts, GPU registers, shared memory, L1 and L2 cache, VRAM, memory bandwidth, rasterization, shaders, texture units, ray-tracing hardware, AI and tensor units, integrated and discrete GPUs, GPU power efficiency and performance metrics. Learn how GPU architecture affects gaming, graphics, AI, video, scientific computing and high-performance workloads across smartphones, tablets, laptops, desktops and data centers.

In One Sentence

A GPU is a massively parallel processor designed to execute large numbers of similar operations simultaneously, using thousands of lightweight arithmetic resources organized into compute or shader units and supported by specialized caches, high-bandwidth memory, texture and rasterization hardware, ray-tracing engines, AI accelerators and graphics pipelines.

Introduction: Why Do We Need a GPU?

A CPU is designed primarily for:

general-purpose computing
complex control flow
low-latency execution
sequential and lightly parallel workloads

A GPU is designed primarily for:

massively parallel computation
graphics rendering
image processing
video workloads
scientific computation
AI and machine learning

The fundamental difference is architectural.

CPU
→ Fewer powerful cores
→ Complex control
→ Low latency
→ General-purpose workloads

GPU
→ Many parallel execution resources
→ High throughput
→ Massive data parallelism
→ Graphics + parallel compute

This distinction is the foundation of GPU architecture.

1. Full Definition: What Is a GPU?

A Graphics Processing Unit (GPU) is a massively parallel processor designed to execute large numbers of computational operations concurrently, originally developed primarily for graphics rendering but now widely used for general-purpose parallel computing, AI, scientific workloads, simulation, video processing and other highly parallel tasks.

A modern GPU is therefore much more than a graphics chip.

It is a highly parallel computing architecture.

2. CPU vs GPU

Characteristic	CPU	GPU
Primary objective	General-purpose computing	Parallel throughput
Core/resource count	Relatively few complex cores	Large number of parallel resources
Control logic	Highly sophisticated	Distributed across many compute resources
Branch-heavy workloads	Strong	Generally less efficient
Parallel workloads	Good	Excellent
Graphics	Limited/general	Specialized
AI	Capable	Highly capable
Memory bandwidth	Moderate to high	Often extremely high
Latency optimization	Major priority	Throughput optimization
Typical role	System control	Graphics + parallel acceleration

Neither is universally “better.”

They are optimized for different types of computation.

3. Why GPUs Are So Parallel

Consider an image containing millions of pixels.

Each pixel may require calculations involving:

color
lighting
texture
geometry
shadows

Many of those calculations can be performed independently.

Instead of:

Task 1
 ↓
Task 2
 ↓
Task 3
 ↓
Task 4

a GPU can execute:

Task 1 ─┐
Task 2 ─┤
Task 3 ─┤
Task 4 ─┤
Task 5 ─┤
Task 6 ─┤
Task 7 ─┘

simultaneously or in overlapping groups.

That is the essence of GPU computing.

4. GPU Throughput vs CPU Latency

A CPU is often optimized to finish an individual task quickly.

A GPU is often optimized to finish a huge number of similar tasks efficiently.

Conceptually:

CPU
→ Optimize individual task latency

GPU
→ Optimize total workload throughput

This distinction explains why GPUs excel at graphics and many AI workloads.

5. The Basic GPU Architecture

A simplified modern GPU can be represented as:

                    GPU
                     │
       ┌─────────────┼─────────────┐
       ↓             ↓             ↓
 Graphics       Compute        Memory
 Pipeline       Engines        System
       │             │             │
 Rasterization   Shader/ALU      VRAM
 Textures        Units           Cache
 Ray Tracing     AI Units        Controller

The exact organization varies significantly by GPU architecture.

6. GPU Architecture Is Not the Same as GPU Product

This distinction is important.

An architecture defines the underlying design.

A commercial GPU product combines:

GPU architecture
number of compute units
memory
clocks
power limits
process technology
cooling
software

Therefore:

GPU Architecture
        ↓
GPU Design
        ↓
GPU Product
        ↓
Graphics Card / SoC

7. Shader

A shader is a program executed by GPU hardware to perform graphics or compute operations.

Modern graphics pipelines use different shader stages for different tasks.

Examples include:

vertex shaders
pixel/fragment shaders
compute shaders
geometry shaders
mesh/task shaders

8. Shader Core

A shader core is a programmable execution resource capable of performing numerical operations required by GPU workloads.

Different manufacturers use different terminology.

For example:

NVIDIA → CUDA cores
AMD → Stream Processors
Intel → Execution Units / vector engines depending on architecture

These terms should not be compared as if they were equivalent numerical units.

9. CUDA Cores

NVIDIA uses the term:

CUDA Core

for individual arithmetic execution resources within its GPU architecture.

A GPU may advertise thousands of CUDA cores.

However:

CUDA core count alone does not determine GPU performance.

Architecture, frequency, memory, cache, workload and other resources matter.

10. AMD Stream Processors

AMD commonly refers to its general-purpose GPU arithmetic resources as:

Stream Processors.

Again:

Stream Processor count is not directly comparable to CUDA Core count.

The underlying architecture and execution model differ.

11. Compute Units

AMD GPUs traditionally organize execution resources into:

Compute Units : CUs.

A CU contains multiple execution resources and supporting hardware.

Conceptually:

GPU
│
├── CU
│   ├── ALUs
│   ├── Registers
│   ├── Cache
│   └── Other resources
│
├── CU
│
└── CU

Modern architectures can differ substantially from this simplified model.

12. Streaming Multiprocessors

NVIDIA commonly organizes GPU execution resources into:

Streaming Multiprocessors : SMs.

An SM includes:

CUDA cores
registers
shared memory
scheduling hardware
specialized execution units

Conceptually:

GPU
│
├── SM
├── SM
├── SM
├── SM
└── SM

A GPU may contain many SMs.

13. Execution Units

GPU execution resources perform operations such as:

integer arithmetic
floating-point arithmetic
vector operations
matrix operations

Different architectures organize these resources differently.

14. SIMT

Modern GPUs commonly use a model related to:

SIMT : Single Instruction, Multiple Threads.

The GPU executes groups of threads together while allowing each thread to operate on different data.

Conceptually:

Instruction
     ↓
┌────┼────┬────┬────┐
Thread 1  Thread 2  Thread 3  Thread 4
   ↓         ↓         ↓         ↓
 Data A     Data B    Data C    Data D

This is extremely effective for data-parallel workloads.

15. SIMD vs SIMT

Feature	SIMD	SIMT
Meaning	Single Instruction, Multiple Data	Single Instruction, Multiple Threads
Typical concept	Vector lanes	Groups of GPU threads
Primary use	Parallel data operations	Massive GPU thread execution
Control flow	More constrained	Thread-oriented execution model

The exact hardware implementation varies.

16. GPU Warps and Wavefronts

GPU architectures group threads for execution.

NVIDIA commonly uses:

warps

AMD historically uses:

wavefronts

These groups allow the GPU to execute many threads efficiently.

If threads within a group follow different control paths, execution efficiency can decrease.

17. Branch Divergence

Consider:

Thread 1 → IF
Thread 2 → IF
Thread 3 → ELSE
Thread 4 → ELSE

When threads in the same execution group take different branches, the GPU may need to execute different paths separately.

This is called:

branch divergence.

It can reduce parallel efficiency.

18. GPU Occupancy

Occupancy broadly refers to how effectively GPU execution resources are populated with active work.

High occupancy can help hide latency.

But:

Maximum occupancy does not automatically mean maximum performance.

A workload may be limited by:

memory bandwidth
registers
shared memory
instruction dependencies
execution throughput

19. GPU Registers

GPU threads require registers to store temporary values.

Registers are extremely fast but limited.

If a kernel requires many registers per thread, fewer threads may fit simultaneously on a compute unit.

This can affect occupancy.

20. Shared Memory / Local Data Storage

Many GPUs provide fast on-chip memory that can be shared among threads within an execution group or compute block.

It can be used to:

reuse data
reduce global memory traffic
coordinate computation

This is particularly useful in optimized GPU kernels.

21. GPU Cache

Modern GPUs contain multiple cache levels.

A simplified model:

GPU Compute Unit
       ↓
L1 / Local Cache
       ↓
L2 Cache
       ↓
VRAM

Cache hierarchy reduces the need to repeatedly access external graphics memory.

22. VRAM

VRAM = Video Random-Access Memory.

Dedicated GPUs commonly use high-bandwidth memory technologies such as:

GDDR
GDDR6
GDDR6X
HBM in some high-performance applications

VRAM stores:

textures
frame buffers
geometry
shaders
compute data
AI model data

23. GPU Memory Bandwidth

GPU workloads can require enormous data movement.

Memory bandwidth is therefore critical.

A simplified relationship:

Bandwidth
≈
Memory Data Rate
×
Bus Width
÷
8

For example, a wider memory interface can provide significantly greater bandwidth at the same data rate.

24. Why GPU Memory Bandwidth Matters

Imagine:

GPU Compute
      ↓
Needs enormous data
      ↓
Memory cannot supply it fast enough
      ↓
GPU resources become underutilized

This creates a:

memory bandwidth bottleneck.

25. GPU Compute vs Memory Bottleneck

A GPU workload may be:

Compute-bound

Execution units are the limiting factor.

Memory-bound

Memory bandwidth or latency is the limiting factor.

Compute-bound:
ALUs → Busy

Memory-bound:
ALUs → Waiting

Understanding this distinction is essential when analyzing GPU performance.

26. Graphics Pipeline

Traditional real-time graphics processing involves a sequence of stages.

A simplified modern pipeline:

Application
    ↓
CPU
    ↓
Geometry
    ↓
Vertex Processing
    ↓
Primitive Assembly
    ↓
Rasterization
    ↓
Fragment / Pixel Processing
    ↓
Depth / Blending
    ↓
Frame Buffer
    ↓
Display

Modern APIs and architectures can add significant complexity.

27. Vertex Processing

Vertices describe geometric objects.

A vertex shader can transform:

position
normals
texture coordinates
other attributes

Conceptually:

3D Model
 ↓
Vertex Processing
 ↓
Transformed Geometry

28. Rasterization

Rasterization converts geometric primitives into fragments/pixels that can ultimately become part of an image.

Conceptually:

Triangles
   ↓
Rasterizer
   ↓
Fragments
   ↓
Pixel Processing

Rasterization remains fundamental to real-time graphics.

29. Pixel / Fragment Shading

Pixel or fragment shaders calculate properties such as:

color
lighting
texture
material effects

This stage can require enormous amounts of parallel computation.

30. Texture Units

Texture units perform specialized operations related to texture sampling.

They can handle:

texture addressing
filtering
sampling

This offloads specialized graphics operations from general arithmetic units.

31. Raster Operations

Graphics processors also include hardware for operations such as:

depth testing
blending
render-target operations

These resources help complete the rendering pipeline.

32. Ray Tracing

Modern GPUs increasingly include dedicated hardware for:

ray tracing.

Ray tracing simulates the paths of light rays to calculate realistic:

reflections
shadows
global illumination

Conceptually:

Light
 ↓
Ray
 ↓
Object
 ↓
Reflection / Shadow
 ↓
Pixel

33. Ray-Tracing Cores / Units

GPU vendors use different terminology.

NVIDIA:

RT Cores

AMD:

Ray Accelerators

These specialized resources accelerate ray-tracing operations.

34. AI / Tensor Hardware

Modern GPUs increasingly include specialized matrix and AI execution resources.

NVIDIA uses:

Tensor Cores

Other GPU architectures have their own AI/matrix acceleration hardware.

These units accelerate workloads involving:

matrix multiplication
AI inference
AI training
image enhancement
upscaling

35. GPU and AI

Modern GPUs have become important AI processors because AI workloads are highly parallel.

AI Matrix Operations
        ↓
Massive Parallelism
        ↓
GPU

However, dedicated NPUs can be more power-efficient for certain AI inference workloads.

36. GPU Architecture Specification Table

Component	Function
Shader / ALU Resources	General GPU arithmetic
Compute Units / SMs	Organize execution resources
Registers	Fast per-thread storage
Shared / Local Memory	Fast on-chip data sharing
L1 Cache	Local high-speed cache
L2 Cache	Larger GPU-wide cache
Texture Units	Texture sampling/filtering
Rasterizer	Converts primitives into fragments
RT Units	Accelerate ray tracing
AI / Matrix Units	Accelerate matrix/AI operations
Memory Controller	Connects GPU to external memory
VRAM	Stores graphics/compute data
Display Engine	Generates display output

37. Integrated GPU

An integrated GPU (iGPU) is incorporated into the same processor or SoC platform as the CPU.

Examples are common in:

smartphones
tablets
laptops
desktop APUs

Integrated GPUs often share system memory.

38. Discrete GPU

A discrete GPU is a separate processor, typically with its own dedicated memory.

Conceptually:

CPU
  │
  └── PCIe / Interconnect
          │
        GPU
          │
         VRAM

Discrete GPUs generally provide much greater graphics and compute resources.

39. Integrated vs Discrete GPU

Feature	Integrated GPU	Discrete GPU
Physical design	Integrated	Separate GPU
Memory	Often shared system memory	Dedicated VRAM
Power	Usually lower	Usually higher
Performance	Lower to moderate	Moderate to extremely high
Physical space	Compact	Larger
Cooling	Easier	More demanding
Common devices	Phones, tablets, thin laptops	Gaming PCs, workstations, servers

40. GPU Clock Speed

Like CPUs, GPUs have operating frequencies.

But GPU performance depends on:

architecture
number of execution resources
clock frequency
memory bandwidth
cache
workload

Therefore:

Higher GPU GHz
≠
Automatically Faster GPU

41. GPU Compute Performance

GPU compute throughput is often expressed in:

FLOPS

or:

TFLOPS.

FLOPS means:

Floating-Point Operations Per Second.

TFLOPS means:

Trillions of Floating-Point Operations Per Second.

42. Why TFLOPS Is Not Enough

Two GPUs can have similar theoretical TFLOPS but different real-world performance because of:

memory bandwidth
cache
architecture
software
scheduling
instruction efficiency
workload characteristics

Therefore:

TFLOPS is a theoretical throughput indicator, not a universal performance score.

43. GPU Graphics Performance

Gaming performance is typically evaluated using:

frames per second : FPS.

But FPS depends on:

resolution
graphics settings
ray tracing
game engine
CPU performance
driver
GPU architecture

Therefore benchmark conditions must be specified.

44. Resolution and GPU Workload

Higher resolution means more pixels.

For example:

1080p
→ ~2.1 million pixels

1440p
→ ~3.7 million pixels

4K
→ ~8.3 million pixels

The GPU therefore has substantially more pixel-related work at higher resolutions.

45. Frame Rate

Frame rate measures how many frames are produced per second.

Examples:

30 FPS
60 FPS
120 FPS
144 FPS
240 FPS

Higher frame rates can improve perceived motion smoothness, assuming the display supports them.

46. GPU Memory Capacity

VRAM capacity affects how much graphics and compute data can remain resident.

It can become important for:

high-resolution textures
high-resolution gaming
large AI models
professional workloads
large datasets

But:

More VRAM does not automatically make a GPU faster.

47. GPU Memory Bandwidth vs Capacity

These are different.

Capacity

How much data can be stored.

Bandwidth

How quickly data can be transferred.

A GPU may have:

large VRAM capacity + insufficient bandwidth

or:

high bandwidth + limited capacity.

Both can affect workloads differently.

48. GPU Architecture and Power

GPU performance is strongly connected to power.

A GPU can increase performance through:

more execution units
higher frequency
wider memory
larger cache
specialized hardware

But these generally involve additional:

silicon area
power
heat

49. GPU Performance Per Watt

Modern GPUs increasingly prioritize:

performance per watt.

This is especially important for:

smartphones
laptops
data centers

A GPU that delivers high throughput within a constrained power budget can be more useful than one with greater peak performance but excessive power consumption.

50. GPU Bottlenecks

GPU performance can be limited by:

Compute

Not enough arithmetic throughput.

Memory

Not enough bandwidth or capacity.

Rasterization

Graphics pipeline limitations.

Ray tracing

Ray-tracing workload exceeds specialized hardware capability.

CPU

The CPU cannot supply the GPU with work quickly enough.

Software

Drivers or applications do not efficiently utilize the GPU.

51. CPU Bottleneck vs GPU Bottleneck

A gaming workload can look like:

CPU → 100%
GPU → 60%

The CPU may be limiting performance.

Or:

CPU → 50%
GPU → 99%

The GPU is likely the primary performance limiter.

This is why gaming benchmarks need both CPU and GPU context.

52. GPU Architecture and APIs

GPUs interact with software through graphics and compute APIs such as:

DirectX
Vulkan
OpenGL
Metal
CUDA
OpenCL

Different APIs expose different capabilities.

Software support can significantly affect real-world GPU performance.

53. GPU Drivers

Drivers translate application/API commands into operations suitable for the specific GPU architecture.

Driver quality can affect:

performance
compatibility
stability
feature support

Therefore hardware specifications alone cannot describe the complete GPU experience.

54. GPU Architecture Comparison Framework

For Digital Plaza GPU analysis, evaluate:

Category	What to Examine
Architecture	Underlying GPU design
Execution Resources	CUs, SMs, shader resources
Clock	Operating frequency
Compute	Theoretical arithmetic throughput
VRAM	Capacity
Memory Bus	Width
Bandwidth	Data transfer capability
Cache	Capacity and hierarchy
Ray Tracing	Dedicated acceleration
AI	Matrix/AI acceleration
Software	Drivers and APIs
Power	Typical and peak consumption
Thermals	Sustained performance
Real Workloads	Games, rendering, AI, compute

55. GPU Core Count Is Not Universal

This deserves special emphasis.

Do not directly conclude:

10,000 CUDA cores
>
5,000 Stream Processors

That comparison is invalid.

Different vendors use different:

execution-unit designs
instruction models
clock characteristics
scheduling
arithmetic resources

Therefore:

GPU core counts should primarily be compared within the same architecture family or with appropriate architectural context.

56. GPU Architecture vs GPU Performance

A strong GPU architecture balances:

Compute
+
Memory
+
Cache
+
Scheduling
+
Specialized Hardware
+
Software
+
Power

Weakness in any major subsystem can become a bottleneck.

57. GPU Architecture in Smartphones

Mobile GPUs prioritize:

performance per watt
compact silicon area
shared memory efficiency
thermal efficiency
sustained graphics performance

Mobile GPUs are integrated directly into SoCs.

They often share system memory with:

CPU
NPU
ISP
media engines

58. GPU Architecture in Tablets

Tablets can provide greater thermal and power envelopes than smartphones.

This enables:

larger GPU resources
higher sustained performance
higher-resolution displays
more demanding creative workloads

59. GPU Architecture in Laptops

Laptops can use:

integrated GPUs

or:

discrete GPUs.

Integrated solutions prioritize efficiency.

Discrete GPUs prioritize substantially greater graphics and compute throughput.

60. GPU Architecture in Desktops

Desktop discrete GPUs can use:

large GPU dies
wide memory interfaces
large VRAM pools
high power budgets
advanced cooling

This allows very high graphics and compute performance.

61. GPU Architecture in Data Centers

Data-center GPUs increasingly focus on:

AI training
AI inference
HPC
scientific computing
virtualization
large-scale parallel workloads

These GPUs can have enormous:

compute throughput
memory bandwidth
memory capacity
specialized AI resources

62. The Modern GPU Is More Than a Graphics Processor

The evolution can be summarized:

Graphics Processor
       ↓
Programmable GPU
       ↓
General-Purpose Parallel Processor
       ↓
AI / HPC Accelerator

Modern GPUs are therefore major computing platforms.

63. CPU + GPU + NPU

Modern systems increasingly combine:

             SoC / Processor
                  │
        ┌─────────┼─────────┐
        ↓         ↓         ↓
       CPU       GPU       NPU
        │         │         │
   General     Graphics     AI
   Compute     Parallel     Inference

Each processor specializes in different workloads.

This is the foundation of heterogeneous computing.

64. GPU vs NPU

Feature	GPU	NPU
General parallel compute	Excellent	Limited
Graphics	Excellent	No
AI	Excellent	Highly specialized
Flexibility	High	More specialized
AI efficiency	High	Potentially extremely high
Typical role	Graphics + compute + AI	AI inference

The distinction will be explored fully in the next article.

65. GPU Performance Formula

A simplified conceptual model is:

GPU Performance
≈
Execution Resources
×
Frequency
×
Work Efficiency

But actual performance also depends heavily on:

GPU Performance
+
Memory Bandwidth
+
Cache
+
Software
+
Workload
+
Power
+
Thermals

66. The GPU Architecture Stack

A complete GPU stack can be represented as:

Application
     ↓
Graphics / Compute API
     ↓
Driver
     ↓
GPU ISA / Instruction Model
     ↓
GPU Microarchitecture
     ↓
Shader / Compute Units
     ↓
Memory System
     ↓
Transistors
     ↓
Process Technology

This mirrors the CPU architecture hierarchy while introducing GPU-specific execution models.

67. Common GPU Misconceptions

Myth 1: More GPU cores always means faster.

False.

Different architectures have different execution resources.

Myth 2: Higher TFLOPS always means faster gaming.

False.

Gaming performance depends on the entire graphics pipeline.

Myth 3: More VRAM automatically increases performance.

False.

VRAM capacity and compute throughput are different.

Myth 4: GPU clock speed alone determines performance.

False.

Architecture and parallel resources matter.

Myth 5: Integrated GPUs are always weak.

False.

Modern integrated GPUs can be highly capable for their power envelope.

Myth 6: GPUs are only for graphics.

False.

Modern GPUs are major parallel-computing and AI accelerators.

68. How to Properly Compare GPUs

A strong comparison should include:

Architecture

→ What design generation?

Execution Resources

→ How much parallel hardware?

Frequency

→ At what clocks?

Memory

→ Capacity, type, bus and bandwidth?

Cache

→ How much and how organized?

Ray Tracing

→ Dedicated hardware?

→ Matrix/AI acceleration?

Software

→ Driver and API support?

Power

→ What power envelope?

Benchmarks

→ What real workloads?

This prevents misleading “core count” comparisons.

69. Final Takeaway

A GPU is fundamentally a massively parallel processor.

Its strength comes from combining:

many execution resources + parallel scheduling + high memory bandwidth + specialized graphics hardware + increasingly specialized AI and ray-tracing engines.

The key concepts are: