Module 7 — CPU Pipelining

Learning Objectives

By the end of this module, you should understand:

• Why pipelining exists
• How a pipeline works
• The classic 5-stage pipeline
• Pipeline registers
• Instruction overlap
• Pipeline hazards
• Data forwarding
• Pipeline stalls
• Branch prediction
• Pipeline flushing
• Relation to ARM Cortex processors
• Relation to Operating Systems

1. Why Do We Need Pipelining?

Let's start with a simple question.

Suppose a CPU executes an instruction like:

ADD R1, R2, R3

Without pipelining, the CPU performs:

Only after completing all four stages does it begin the next instruction.

Imagine four instructions.

• Instruction 1
• Instruction 2
• Instruction 3
• Instruction 4

Without Pipeline

Time	Action
T1	I1 Fetch
T2	I1 Decode
T3	I1 Execute
T4	I1 WriteBack
T5	I2 Fetch
T6	I2 Decode
T7	I2 Execute
T8	I2 WriteBack
T9	I3 ...

Notice
Only one instruction is active at a time.
This wastes CPU resources.

Real-Life Analogy

Imagine a laundry process.

Without Pipeline

Very inefficient.

Instead

Wash A -> Dry A -> Fold A
Wash B -> Dry B -> Fold B

becomes

Time →	1	2	3	4	5
Washer	A	B	C
Dryer		A	B
Folder			A

Now all workers are busy simultaneously.
This is exactly how CPU pipelining works.

2. Basic Pipeline Idea

Instead of executing one instruction completely,
CPU overlaps multiple instructions.

Example

• Instruction1
• Instruction2
• Instruction3
• Instruction4

Pipeline

Cycle	Action 1	Action 2	Action 3	Action 4
1	Fetch I1
2	Decode I1	Fetch I2
3	Execute I1	Decode I2	Fetch I3
4	Write I1	Execute I2	Decode I3	Fetch I4

Notice
Four different instructions are active simultaneously.

3. Classical 5-Stage Pipeline

Most textbooks introduce this model.

Let's study every stage.

Stage 1 — Instruction Fetch (IF)

CPU reads instruction from memory.

Uses

• Program Counter
• Instruction Cache
• Instruction Register

Example

Stage 2 — Instruction Decode (ID)

Control Unit understands

ADD R1,R2,R3

Questions answered

• Opcode?
• Operands?
• Destination Register?
• Immediate value?
• Branch?

Stage 3 — Execute (EX)

ALU performs operation.

Example

Branch instructions also calculate target addresses here on many simple pipelines.

Stage 4 — Memory Access (MEM)

Only needed for load/store instructions.

Example

LOAD R1,[2000]

Memory
↓
Register

OR

STORE R1,[2000]

Register
↓
Memory

Arithmetic instructions often pass through this stage without accessing memory.

Stage 5 — Write Back (WB)

Result written into destination register.

Example

ADD R1,R2,R3

Result
↓
R1

Instruction finished.

4. Complete Pipeline Diagram

Cycle →	1	2	3	4	5	6	7	8	9
I1	IF	ID	EX	MEM	WB
I2		IF	ID	EX	MEM	WB
I3			IF	ID	EX	MEM	WB
I4				IF	ID	EX	MEM	WB
I5					IF	ID	EX	MEM	WB

Notice
Pipeline stays full.
After the pipeline is filled,
one instruction completes almost every clock cycle.

5. Pipeline Registers

Between stages,
CPU stores intermediate results.

These registers isolate each stage so they can work independently.

6. Pipeline Performance

Suppose
Each stage takes
1 ns

Without pipeline

Instruction
5 ns

10 instructions
50 ns

With pipeline

First instruction
5 ns
Every next instruction
1 ns

Total
5 + 9 = 14 ns

Huge improvement.

Important:
Pipelining increases throughput, not necessarily the latency of a single instruction.

7. Pipeline Hazards

Sometimes
Pipeline cannot continue smoothly.
These are called hazards.

Three types

• Structural
• Data
• Control

Let's examine each.

Structural Hazard

Occurs when two instructions need the same hardware resource.

Example

Only one memory port.

Solution

• Separate instruction and data caches (Harvard architecture)
• Multiple functional units
• Stall one instruction

Data Hazard

Most common interview question.

Example

ADD R1,R2,R3
SUB R4,R1,R5

Problem
Second instruction needs
R1

But
First instruction hasn't written it yet.

Timeline

Data Forwarding (Bypassing)

Instead of waiting,
CPU forwards the ALU result directly.

No need to wait for write-back.
This significantly reduces stalls.

Pipeline Stall

If forwarding isn't possible,
CPU pauses the pipeline.

Cycle	Action
1	Fetch
2	Decode
3	Execute
4	STALL
5	STALL
6	Continue

Performance decreases.

Control Hazard

Occurs with branch instructions.

Example

if(a>b)

CPU doesn't know
Which instruction comes next.

The pipeline may fetch the wrong instructions.

Pipeline Flush

Suppose CPU guessed incorrectly.

Already fetched

• Instruction100
• Instruction104
• Instruction108

Oops
Branch should have gone to
500

CPU discards all incorrect instructions.
This is called pipeline flushing.

Branch Prediction

Modern CPUs try to predict.

Example

if(loop)

Prediction
Branch Taken

If correct
No delay.
If wrong
Pipeline flushed.

Sophisticated CPUs use hardware predictors to improve accuracy.

ARM Pipeline

Classic ARM processors historically used simple pipelines (e.g., 3-stage or 5-stage depending on the core).

Modern ARM Cortex-A processors use much deeper pipelines, often combined with superscalar and out-of-order execution.

The exact pipeline depth depends on the specific Cortex core.

Embedded Example (ADAS)

Camera Driver

Every instruction flows through the CPU pipeline.

Good compiler optimization improves instruction scheduling and reduces stalls.

OS Perspective

Interrupt occurs.

Pipeline

• Instruction1
• Instruction2
• Instruction3

CPU

• Completes or safely abandons in-flight instructions depending on the architecture.
• Saves architectural state.
• Transfers control to the interrupt handler.

The operating system relies on the processor's defined exception model rather than manually managing the pipeline.

Pipeline vs Non-Pipeline

Non-Pipeline	Pipeline
One instruction at a time	Multiple instructions overlap
Low throughput	High throughput
CPU idle frequently	CPU stages stay busy
Simpler hardware	More complex hardware
Easier design	Hazard handling required

Interview Questions

Beginner

• What is pipelining?
• Why do CPUs use pipelining?
• What are the five stages?
• What is throughput?

Intermediate

• What are pipeline hazards?
• Difference between structural and data hazards?
• Explain forwarding.
• Explain pipeline stalls.
• What is branch prediction?

Senior

• How does forwarding reduce stalls?
• Explain control hazards in detail.
• What is pipeline flushing?
• Why does branch prediction improve performance?
• How does pipelining affect interrupt latency?
• Why doesn't pipelining reduce the execution time of an individual instruction?

Common Mistakes

• ❌ Thinking pipelining makes one instruction execute faster. It primarily increases overall throughput.
• ❌ Assuming every instruction accesses memory. Many instructions operate only on registers.
• ❌ Believing all hazards can be eliminated. They can often be reduced, not always eliminated.
• ❌ Thinking branch prediction is always correct. Incorrect predictions cause flushes and performance penalties.

Summary

Pipelining divides instruction execution into multiple stages that operate concurrently. Once the pipeline is full, the processor can complete approximately one instruction per clock cycle, dramatically increasing throughput. Modern CPUs use forwarding, hazard detection, branch prediction, and flushing mechanisms to maintain pipeline efficiency while ensuring correct program execution.

Mini Quiz

1. Why is pipelining needed?
2. Name the five classical pipeline stages.
3. Why does pipelining improve throughput but not the latency of a single instruction?
4. What is a data hazard?
5. What is forwarding?
6. Why do branch instructions create control hazards?
7. What happens during a pipeline flush?
8. How can a structural hazard occur?