SoC Architecture Explained: CPU, GPU, NPU, ISP, DSP, Modem, Memory Controller, Media Engine, Interconnect and Modern System-on-Chip Design

Meta Description

Understand modern System-on-Chip (SoC) architecture and how CPU, GPU, NPU, ISP, DSP, modem, media engine, memory controller, security hardware, display engine and I/O work together inside a single integrated computing platform. Explore heterogeneous computing, shared memory, cache, interconnects, Network-on-Chip, power management, chiplets, monolithic SoCs, AI processing, camera pipelines, video engines and connectivity. Learn how SoC architecture affects smartphones, tablets, laptops, PCs, edge devices and other modern computing systems.

In One Sentence

A System-on-Chip (SoC) integrates multiple computing, memory, connectivity, media, security and I/O functions into a highly coordinated semiconductor platform, allowing modern smartphones, tablets, laptops and other devices to combine CPU, GPU, NPU, ISP, DSP, modem and memory technologies within a single chip or tightly integrated package.

Introduction: A Modern Processor Is No Longer Just a CPU

When people hear:

“processor”

they often imagine the CPU.

Modern computing hardware is much more complicated.

A smartphone SoC can contain:

CPU
GPU
NPU
ISP
DSP
modem
media engine
memory controller
security processor
display engine
image/video accelerators
interconnect
cache
I/O controllers

All of these components work together.

This is why the term:

SoC : System-on-Chip

is so important.

1. Full Definition: What Is an SoC?

A System-on-Chip (SoC) is an integrated semiconductor system that combines multiple processing, memory-control, connectivity, security, I/O and specialized acceleration functions onto one chip or closely integrated silicon platform.

An SoC can contain several different processor types, each optimized for a particular workload.

Conceptually:

                 SoC
                  │
 ┌────────────────┼────────────────┐
 ↓                ↓                ↓
CPU              GPU              NPU
 ↓                ↓                ↓
General         Graphics           AI
Compute         Compute         Processing

But that is only the beginning.

2. Why Were SoCs Developed?

Historically, computers used many separate chips.

For example:

CPU
 ↓
GPU
 ↓
Memory Controller
 ↓
I/O Controller
 ↓
Modem
 ↓
Other Chips

Modern integration places many of these functions together:

              ┌─────────────────────┐
              │         SoC         │
              │                     │
              │ CPU  GPU  NPU       │
              │ ISP  DSP  Modem     │
              │ Media  Security     │
              │ Memory Controller   │
              └─────────────────────┘

This can reduce:

physical space
interconnect distance
power consumption
latency
motherboard complexity

3. SoC vs CPU

This distinction is fundamental.

CPU

A processor designed primarily for general-purpose computation.

SoC

A complete integrated computing platform containing a CPU plus many other components.

Therefore:

A CPU can be part of an SoC, but an SoC is not simply a CPU.

9. GPU

The GPU handles highly parallel workloads.

Typical tasks include:

graphics rendering
gaming
image processing
compute workloads
some AI operations

The GPU communicates with the rest of the SoC through the internal interconnect and memory system.

10. NPU

The NPU specializes in AI and machine-learning workloads.

Examples:

object recognition
voice processing
image enhancement
generative AI
local inference

AI Workload
     ↓
NPU
     ↓
AI Result

This reduces the need to use the CPU or GPU for every AI task.

11. ISP

ISP = Image Signal Processor.

An ISP processes data from cameras.

It can perform tasks such as:

demosaicing
noise reduction
exposure processing
white balance
HDR processing
autofocus-related processing
image pipeline operations

Conceptually:

Camera Sensor
      ↓
     ISP
      ↓
Processed Image

4. SoC vs Processor

The word processor is often used loosely.

A modern SoC may contain several processors:

Therefore, the system can contain:

Multiple Processing Engines
          ↓
Integrated SoC

This is called:

heterogeneous computing.

5. Heterogeneous Computing

Heterogeneous computing means using different processing engines for different types of work.

Workload
   │
   ├── General computation → CPU
   ├── Graphics → GPU
   ├── AI → NPU
   ├── Image processing → ISP
   ├── Signal processing → DSP
   └── Video → Media Engine

This is one of the defining characteristics of modern SoCs.

6. The Core SoC Architecture

A simplified modern SoC looks like:

                         SoC
                          │
       ┌──────────────────┼──────────────────┐
       ↓                  ↓                  ↓
      CPU                GPU                NPU
       │                  │                  │
       └──────────────┬───┴──────┬───────────┘
                      ↓
                 Interconnect
                      │
       ┌──────────────┼──────────────┐
       ↓              ↓              ↓
      ISP             DSP          Media
       │              │           Engine
       └──────────────┼──────────────┘
                      ↓
               Memory Controller
                      ↓
                    DRAM

Additional blocks provide:

connectivity
security
display
storage
I/O

7. CPU Complex

The CPU remains the primary general-purpose processing engine.

It handles:

operating-system tasks
application logic
control flow
background processing
system management

A modern SoC may contain multiple CPU cores.

8. CPU Core Clusters

Mobile SoCs frequently use heterogeneous CPU cores.

A simplified arrangement:

CPU Complex
│
├── Performance Cores
├── Performance Cores
├── Efficiency Cores
└── Efficiency Cores

Different core types can balance:

performance ↔ power efficiency.

9. GPU

The GPU handles highly parallel workloads.

Typical tasks include:

graphics rendering
gaming
image processing
compute workloads
some AI operations

The GPU communicates with the rest of the SoC through the internal interconnect and memory system.

10. NPU

The NPU specializes in AI and machine-learning workloads.

Examples:

object recognition
voice processing
image enhancement
generative AI
local inference

AI Workload
     ↓
NPU
     ↓
AI Result

This reduces the need to use the CPU or GPU for every AI task.

11. ISP

ISP = Image Signal Processor.

An ISP processes data from cameras.

It can perform tasks such as:

demosaicing
noise reduction
exposure processing
white balance
HDR processing
autofocus-related processing
image pipeline operations

Conceptually:

Camera Sensor
      ↓
     ISP
      ↓
Processed Image

12. Why the ISP Matters

Modern smartphone photography depends heavily on computational image processing.

The camera sensor provides raw or partially processed information.

The ISP helps transform that data into a usable image.

Sensor
 ↓
Raw Data
 ↓
ISP
 ↓
Image Processing
 ↓
Final Image

AI hardware can then further enhance the result.

13. DSP

DSP = Digital Signal Processor.

DSPs are optimized for repetitive mathematical operations involving signals.

Common workloads include:

audio
voice
sensors
communications
signal filtering

A DSP can execute these tasks more efficiently than a general-purpose CPU in suitable workloads.

14. DSP vs NPU

Feature	DSP	NPU
Primary purpose	Signal processing	Neural-network processing
Audio	Excellent	Limited
Sensors	Excellent	Possible
AI inference	Possible	Specialized
Matrix processing	Architecture-dependent	Major focus
Power efficiency	High for signal workloads	High for AI workloads

Modern SoCs can contain both.

15. Media Engine

A media engine is specialized hardware for processing video and/or audio.

It can accelerate:

video encoding
video decoding
compression
decompression
format conversion

Instead of using CPU cores:

Video
 ↓
CPU
 ↓
High CPU Load

dedicated hardware can perform the workload:

Video
 ↓
Media Engine
 ↓
Efficient Processing

16. Hardware Video Decoder

A video decoder converts compressed video into frames that can be displayed or processed.

Examples of video formats include:

H.264
H.265 / HEVC
VP9
AV1

Hardware decoding can significantly reduce CPU workload.

17. Hardware Video Encoder

A video encoder compresses raw video into a storage or transmission format.

It is used when:

recording video
streaming
video conferencing
exporting video

Dedicated encoding hardware can improve performance and efficiency.

18. Modem

A modem provides cellular communication.

A modern cellular modem handles technologies such as:

4G LTE
5G

It manages complex radio communication functions.

A smartphone SoC may integrate the modem directly or pair it closely with a separate modem.

19. SoC With Integrated Modem

A highly integrated mobile platform can look like:

               SoC
 ┌──────────────────────────────┐
 │ CPU  GPU  NPU  ISP  DSP      │
 │                              │
 │ Media Engine                 │
 │ Cellular Modem               │
 │ Security                     │
 │ Memory Controller            │
 └──────────────────────────────┘

Integration can reduce:

board space
power
interconnect complexity

20. Memory Controller

The memory controller manages communication between the SoC and system memory.

SoC
 ↓
Memory Controller
 ↓
LPDDR / DRAM

It determines important characteristics such as:

supported memory type
memory channels
bandwidth
addressing
memory timing

21. Shared Memory

Many SoCs use a shared system memory architecture.

              LPDDR
                │
       ┌────────┼────────┐
       ↓        ↓        ↓
      CPU      GPU      NPU

This allows different processing engines to access common data.

However, they also compete for memory bandwidth.

22. Memory Bandwidth Sharing

Imagine:

CPU → Memory
GPU → Memory
NPU → Memory
ISP → Memory

If several engines are active simultaneously, total memory demand can become very high.

The SoC therefore needs sophisticated:

arbitration
scheduling
caching
bandwidth management

23. Interconnect

The interconnect is the communication network connecting components within the SoC.

Conceptually:

CPU ─┐
GPU ─┤
NPU ─┤
ISP ─┼──→ Interconnect
DSP ─┤
I/O ─┘

It allows different blocks to exchange:

data
instructions
memory requests
control information

24. SoC Fabric

Large SoCs often use sophisticated on-chip communication fabrics.

These can include:

buses
crossbars
networks-on-chip
coherent interconnects

The goal is to provide efficient communication among many components.

25. Network-on-Chip

NoC = Network-on-Chip.

A NoC uses network-like communication concepts to connect different IP blocks.

Conceptually:

CPU ─────┐
GPU ─────┤
NPU ─────┤
ISP ─────┼── NoC
DSP ─────┤
Memory ──┘

This architecture becomes increasingly important as SoCs become more complex.

26. Cache Coherence

When multiple processing engines share memory, maintaining consistent views of data can become complicated.

CPU cores often use cache-coherence mechanisms.

For example:

Core 1 Cache
      │
Core 2 Cache
      │
Core 3 Cache
      │
   Coherence
      │
 Shared Memory

The exact coherence model varies by architecture.

27. System-Level Cache

Some SoCs include a cache or cache-like shared memory layer between processing engines and DRAM.

Conceptually:

CPU
GPU
NPU
ISP
 │
 ↓
Shared Cache
 │
 ↓
DRAM

This can reduce external memory traffic.

28. Security Processor

Modern SoCs frequently include dedicated security hardware.

It can support:

secure boot
encryption
key storage
authentication
trusted execution
hardware-isolated security functions

Security is therefore part of modern SoC architecture rather than an afterthought.

29. Display Engine

The display subsystem converts processed image data into signals suitable for the display.

It may handle:

display composition
refresh timing
HDR
multiple display outputs
display compression

Conceptually:

GPU / Media
     ↓
Display Engine
     ↓
Display

30. Camera Pipeline

A modern smartphone camera pipeline can involve multiple SoC components:

Camera Sensor
      ↓
     ISP
      ↓
   NPU / AI
      ↓
 Image Processing
      ↓
  GPU / Display

Different stages can use different processing engines.

31. AI Camera Processing

AI can assist with:

scene detection
segmentation
portrait effects
noise reduction
autofocus
HDR
image reconstruction

The NPU and ISP can therefore work together.

ISP
 ↓
Image Data
 ↓
NPU
 ↓
AI Analysis
 ↓
ISP / GPU
 ↓
Final Image

32. Audio Pipeline

Audio processing can involve:

Microphone
 ↓
DSP
 ↓
Noise Reduction
 ↓
NPU / AI
 ↓
Voice Recognition
 ↓
CPU / Application

Different processing engines cooperate according to workload requirements.

33. Video Pipeline

A video workflow can involve:

Camera
 ↓
ISP
 ↓
Media Engine
 ↓
GPU
 ↓
Display Engine
 ↓
Display

AI processing may be inserted into the pipeline where useful.

34. Connectivity Subsystem

An SoC platform can also integrate or connect to:

cellular modem
Wi-Fi
Bluetooth
GNSS
USB
PCIe
storage interfaces

This turns the SoC into a broader platform rather than merely a compute engine.

35. Storage Controllers

Modern SoCs can provide controllers for storage technologies.

Examples include:

UFS
NVMe
eMMC in some platforms

The controller manages communication between the SoC and storage device.

36. I/O

I/O = Input/Output.

SoCs may provide interfaces for:

USB
displays
cameras
storage
sensors
peripherals

These interfaces allow the SoC to communicate with the outside world.

37. SoC Architecture Table

Component	Primary Function
CPU	General-purpose computation
GPU	Graphics and parallel compute
NPU	AI / neural-network acceleration
ISP	Camera and image processing
DSP	Signal processing
Media Engine	Video/audio encoding and decoding
Modem	Cellular communication
Memory Controller	DRAM communication
Interconnect / NoC	Internal component communication
Security Engine	Hardware security
Display Engine	Display processing
Storage Controller	Storage communication
I/O Controllers	External interfaces

38. Why Specialized Engines Matter

Imagine processing a video entirely on the CPU.

CPU
 ↓
Decode
 ↓
Process
 ↓
Encode
 ↓
Display

This could consume substantial CPU resources.

Instead:

CPU → Control
Media Engine → Video
GPU → Graphics
NPU → AI
Display Engine → Output

Each component performs the task it is designed for.

This improves system efficiency.

39. Heterogeneous Workload Distribution

Modern SoCs dynamically distribute workloads.

Application
     ↓
Operating System / Runtime
     ↓
Workload Classification
     ↓
┌────┼────┬────┬────┐
CPU  GPU  NPU  DSP

This is a major reason modern devices can perform complex workloads within relatively small power budgets.

40. Power Management

Every SoC component does not need to run at full power simultaneously.

Modern SoCs can:

power-gate blocks
reduce frequency
reduce voltage
change operating states
turn unused engines off

Conceptually:

Unused Block
 ↓
Power Gating
 ↓
Minimal / Zero Dynamic Activity

This is essential for battery-powered devices.

41. DVFS

DVFS = Dynamic Voltage and Frequency Scaling.

The system dynamically adjusts:

voltage
frequency

according to workload requirements.

For example:

Light workload
→ Lower frequency
→ Lower voltage
→ Lower power

Heavy workload
→ Higher frequency
→ Higher power

42. Thermal Management

An SoC must remain within a safe thermal envelope.

Workload
 ↓
Power
 ↓
Heat
 ↓
Temperature
 ↓
Thermal Management

If temperatures become too high, the system can reduce performance.

43. SoC Performance Is System Performance

A smartphone SoC’s performance cannot be described by CPU benchmark results alone.

The final user experience depends on:

CPU
+
GPU
+
NPU
+
Memory
+
ISP
+
DSP
+
Media
+
Software
+
Power
+
Thermals

This is why SoC analysis should be broader than CPU analysis.

44. SoC vs Chipset

The terms SoC and chipset are often used interchangeably in consumer technology.

Technically, they are not always identical.

A chipset can refer to a broader collection of chips.

An SoC specifically integrates many functions into a single semiconductor die or tightly integrated silicon platform.

For consumer writing, terminology should be clearly defined.

45. Monolithic SoC

A monolithic SoC places many functions on one die.

Conceptually:

┌─────────────────────────────┐
│ CPU GPU NPU ISP DSP Modem   │
│ Memory Controller Security  │
│ Media Engine                │
└─────────────────────────────┘

Advantages can include:

compact design
short internal communication paths
strong integration

But very large monolithic dies can be expensive and difficult to manufacture.

46. Chiplet-Based Architecture

Modern high-performance processors increasingly use chiplets.

Instead of one huge die:

┌────────┐ ┌────────┐ ┌────────┐
│ CPU    │ │ I/O    │ │ Cache  │
│ Die    │ │ Die    │ │ Die    │
└────────┘ └────────┘ └────────┘

These are connected within a package.

This can improve:

scalability
manufacturing flexibility
yield
product segmentation

47. SoC vs Chiplet Architecture

Architecture	Main Characteristic
Monolithic SoC	Many functions integrated on one die
Multi-die SoC	Multiple dies integrated as one platform
Chiplet Architecture	Modular dies connected through high-speed interconnects
3D Integration	Dies stacked vertically

Modern computing increasingly combines these approaches.

48. Smartphone SoC Architecture

A typical flagship smartphone SoC may contain:

CPU
GPU
NPU
ISP
DSP
5G Modem
Media Engine
Memory Controller
Security
Display
I/O

This integration allows an enormous range of functions inside a small physical package.

49. Tablet SoC Architecture

Tablet SoCs use a similar architecture but may prioritize:

sustained CPU performance
GPU performance
display processing
AI
media
multitasking

Their larger physical designs can sometimes support greater thermal capacity than smartphones.

50. Laptop SoC Architecture

Modern laptop processors increasingly integrate:

CPU
GPU
NPU
media engines
memory controllers
security
I/O

This is especially important for thin and battery-efficient computers.

51. Desktop SoC Architecture

Desktop systems can use:

traditional CPU + chipset architectures
integrated CPU/GPU platforms
highly integrated desktop SoCs
chiplet-based processors

Desktop platforms generally have more flexibility in power, cooling and expansion.

52. Why SoC Integration Matters for AI

AI workloads can move through several engines.

For example:

Camera
 ↓
ISP
 ↓
NPU
 ↓
CPU
 ↓
GPU
 ↓
Display

Each engine contributes a specialized capability.

This is the foundation of modern heterogeneous AI computing.

53. SoC Memory Architecture

Modern SoCs increasingly depend on shared memory.

                  DRAM
                   │
              Memory Controller
                   │
             Shared Interconnect
          ┌────────┼────────┐
          ↓        ↓        ↓
         CPU      GPU      NPU

This architecture improves data sharing but makes bandwidth management extremely important.

54. SoC Bottlenecks

An SoC can become limited by:

CPU

Insufficient general-purpose compute.

GPU

Insufficient graphics throughput.

NPU

AI accelerator limitations.

Memory

Insufficient bandwidth or capacity.

Interconnect

Communication bottlenecks.

Thermals

Heat limits sustained performance.

Software

Poor workload distribution or optimization.

Therefore:

SoC performance is a system-level problem.

55. SoC Efficiency

A well-designed SoC tries to minimize unnecessary data movement.

For example:

Camera
 ↓
ISP
 ↓
NPU
 ↓
Memory

If data can remain close to the relevant engines, the system may reduce:

latency
memory traffic
energy consumption

56. SoC Specification Framework

Digital Plaza should analyze SoCs using:

Category	Questions
CPU	Core architecture, count, performance?
GPU	Architecture and graphics capability?
NPU	AI throughput and efficiency?
ISP	Camera processing capability?
DSP	Signal-processing capability?
Media	Video encode/decode support?
Modem	Cellular generation and capability?
Memory	Type, bandwidth and capacity?
Interconnect	How efficiently do blocks communicate?
Process	What semiconductor technology?
Power	What operating envelope?
Thermals	What sustained performance?
Software	How effectively is hardware utilized?

57. SoC Analysis: Specifications vs Reality

A manufacturer may advertise:

“AI engine — 100 TOPS”

But a serious analysis asks:

At what precision?
Which AI workloads?
What memory bandwidth?
What software support?
What sustained performance?
What power level?

Similarly:

“3.5 GHz CPU”

does not tell the complete CPU performance story.

And:

“X GPU cores”

does not directly determine graphics performance.

58. The Modern SoC Is a Computing Ecosystem

A modern SoC can be viewed as:

                 SYSTEM-ON-CHIP
                       │
     ┌─────────────────┼─────────────────┐
     ↓                 ↓                 ↓
  COMPUTE          SPECIALIZED        CONNECTIVITY
     │                 │                 │
 CPU GPU NPU       ISP DSP Media      Modem Wi-Fi
                                      Bluetooth
     │                 │                 │
     └─────────────────┼─────────────────┘
                       ↓
                 MEMORY + I/O
                       ↓
                    SYSTEM

This is a far more accurate mental model than:

“The SoC is the CPU.”

59. Common SoC Misconceptions

Myth 1: SoC means CPU.

False.

The CPU is only one component.

Myth 2: More CPU cores automatically make a better SoC.

False.

GPU, NPU, memory, ISP, modem and software also matter.

Myth 3: A high TOPS number means the entire SoC is powerful.

False.

TOPS describes a specific AI capability.

Myth 4: Integrated components are always slower.

Not necessarily.

Integration can improve efficiency and communication.

Myth 5: Every SoC has exactly the same components.

False.

SoC architectures vary dramatically by market and product.

60. The SoC as the Foundation of Modern Devices

The progression can be summarized:

Separate Chips
      ↓
Integrated Functions
      ↓
System-on-Chip
      ↓
Heterogeneous Computing
      ↓
AI-Centric SoC
      ↓
Highly Integrated Computing Platform

The modern SoC is becoming the central computing platform for:

smartphones
tablets
laptops
automotive systems
edge devices
embedded systems

61. Final Takeaway

A System-on-Chip is best understood as a complete heterogeneous computing platform integrated into silicon.

It can combine:

CPU

→ general-purpose computing

GPU

→ graphics and parallel computation

NPU

→ AI acceleration

ISP

→ image processing

DSP

→ signal processing

Media Engine

→ video/audio processing

Modem

→ cellular connectivity

Memory Controller

→ system memory access

Interconnect

→ communication between components

Security Engine

→ trusted and protected computation

Together they create a single coordinated computing system.

The most important lesson is:

Modern processor performance comes from cooperation between specialized engines, not from the CPU alone.

That is why the correct way to evaluate an SoC is to examine the entire architecture, memory system, interconnect, software stack, power envelope and thermal behavior.