# Failure Case Analysis: JPEG (Decomp) on x86-64

**Benchmark:** `cBench/consumer_jpeg_d`

This case study examines a failure of AutoPass on `cBench/consumer_jpeg_d`, which implements a JPEG decompression workload, where the generated pipeline degrades performance to 0.784× relative to the `-O3` baseline.  This page combines the original optimization log with a structured analysis of why AutoPass failed on this benchmark. 

- the optimization loop is **not random** — it diagnoses harmful edits and tries to correct them across rounds;
- but it is also **not omnipotent** — once an aggressive early policy pushes the program into a poor structural regime, later pass and parameter tuning may only partially recover performance.

---

## 1. Core Performance Metrics

| Metric | Baseline (O3) | Round 1 | Round 2 | Round 3 |
| --- | --- | --- | --- | --- |
| **Execution Time (s)** | **10.79** | 13.70 (**+26.97%**) | 13.71 (+27.06%) | 13.72 (+27.15%) |
| **IPC (Instr. Per Cycle)** | **2.61** | 2.21 (**-15.33%**) | 2.45 (+10.86% vs R1) | 2.49 (+12.67% vs R1) |
| **Total Instructions** | 58.75B | **56.82B (-3.28%)** | 61.28B (+4.31% vs BL) | 62.54B (+6.45% vs BL) |
| **Code Size (KB)** | **227.4** | 949.2 (**+317.41%**) | 670.5 (-29.36% vs R1) | 575.8 (-39.34% vs R1) |
| **L1-dcache-misses** | **4.38M** | 6.52M (**+48.86%**) | 5.12M (-21.47% vs R1) | 4.88M (-25.15% vs R1) |
| **Branch-misses** | 452M | 378M (-16.37%) | 412M | 425M |

---

## 2. High-Level Interpretation

This benchmark is a representative **failure case**. Although AutoPass reacts to bad decisions and improves several microarchitectural indicators from Round 1 to Round 3, **all optimized versions remain much slower than the `-O3` baseline**.

The core pattern is:

- **Round 1** makes an explainable but overly aggressive decision.
- **Round 2** and **Round 3** correctly diagnose and mitigate some of the damage.
- However, the later rounds cannot recover the baseline because the dominant bottleneck shifts to **memory behavior and alias-sensitive load redundancy**, which the current pass-edit space cannot effectively repair.

In short:

> AutoPass successfully **diagnoses** the failure, but it cannot fully **recover** from it on this benchmark.
> 

---

## 3. Baseline: Why `O3` Is Strong Here

### Baseline profile

- **Execution time:** 10.79 s
- **IPC:** 2.61
- **Code size:** 227.4 KB
- **L1-dcache misses:** 4.38M

### Observed bottleneck

The baseline logs identify roughly **1.21 billion cycles of memory stalls**, mainly associated with:

- pointer-heavy structures in **JPEG color conversion**
- **Huffman decoding**
- other memory-sensitive parts of the decode path

### Interpretation

The `-O3` pipeline keeps the binary compact and preserves high execution efficiency. The dominant performance limit is already **memory-side**, not simply lack of unrolling or vectorization.

That matters because it means aggressive structural expansion is risky: if the pipeline grows too much, any small improvement in instruction throughput may be outweighed by:

- front-end pressure
- I-cache stress
- worse locality
- unresolved memory dependencies

---

## 4. Round 1 — Over-Aggressive Structural Expansion

### 4.1 System Decision Logic

The Reasoning Agent adopts an aggressive strategy centered on increasing instruction-level parallelism (ILP):

- **Inlining decision**
    
    `inline-threshold = 750`
    
    This forces inlining of functions such as `jpeg_huff_decode` (Cost: 660).
    
- **Loop decision**
    
    `unroll-count = 12`
    
    `unroll-threshold = 800`
    
    These settings target the 8-iteration loops in `jpeg_idct_ifast`.
    
- **Vectorization decision**
    
    `slp-threshold = -5`
    
    This makes SLP vectorization more aggressive.
    
- **Pipeline adjustment**
    
    Removes Coroutine/OpenMP-related passes
    
    Repeats `LICM` (with speculation enabled) and `GVN`
    
- **Hardware target**
    
    `mtriple = x86_64-unknown-linux-gnu`
    
    `mcpu = skylake`
    

### 4.2 What went wrong

Round 1 is the most important step in the failure. The decisions are **explainable**, but the resulting pipeline is badly mismatched to the benchmark’s real execution behavior.

### Key observations

- **Execution time worsens by 26.97%**
- **Code size explodes by 317.41%**
- **IPC drops by 15.33%**
- **L1-dcache misses increase by 48.86%**
- **Total instructions decrease by 3.28%**

This is a crucial lesson:

> Fewer instructions do **not** imply faster execution.
> 

### 4.3 Why instruction reduction does not help

Round 1 reduces the total instruction count, but it does so by aggressively:

- inlining decode logic
- unrolling loops
- enabling more permissive vectorization

This creates a much larger binary and likely overwhelms the instruction delivery path. The performance counters suggest the following mechanism:

1. **Code footprint becomes too large**
    - binary grows from **227.4 KB** to **949.2 KB**
    - this is likely far beyond a comfortable front-end working set
2. **Instruction fetch / decode efficiency deteriorates**
    - IPC drops sharply from **2.61** to **2.21**
3. **Memory pressure increases**
    - L1 data-cache misses rise substantially
4. **Net result**
    - fewer dynamic instructions, but much lower throughput and worse runtime

### 4.4 Microarchitectural mismatch

This round shows a mismatch between static optimization logic and target-machine reality:

- `unroll-count = 12` is likely too large for the Skylake front end and uop-cache behavior
- `slp-threshold = -5` encourages vectorization even when the cost model is marginal
- for pointer-heavy JPEG decode paths, more aggressive SLP does not necessarily map to better execution

Even though branch misses decrease, the combination of:

- code bloat
- front-end pressure
- memory-side inefficiency

dominates the performance outcome.

### 4.5 Evaluation Agent’s corrective feedback

The Evaluation Agent recommends:

- **Size control**
    
    reduce `unroll-count` to **4–8**
    
    reduce `unroll-threshold` to **300–500**
    
- **Inlining adjustment**
    
    lower `inline-threshold` to **500–600**
    
- **Vectorization control**
    
    raise `slp-threshold` to **0 or higher**
    

This feedback is important because it shows that the system correctly identifies the failure as one of **over-aggressive expansion**, not a generic regression.

---

## 5. Round 2 — Valid Diagnosis, Limited Recovery

### 5.1 System Decision Logic

Round 2 follows the corrective signal from Round 1:

- `unroll-count = 6`
- `unroll-threshold = 400`
- `inline-threshold = 550`
- `slp-threshold = 2`

Additional changes:

- removes non-optimization auxiliary passes such as `transform-warning` and `cg-profile`
- keeps core `LICM` and `GVN` structure
- focuses on balancing size, ILP, and memory pressure

### 5.2 What improves

Round 2 successfully corrects several obvious pathologies:

- **Code size** drops from **949.2 KB** to **670.5 KB**
- **IPC** rises from **2.21** to **2.45**
- **L1-dcache misses** fall from **6.52M** to **5.12M**

These are meaningful improvements. The feedback loop is clearly doing useful work.

### 5.3 What does not improve

Despite these counter improvements:

- **Execution time remains bad**
13.71 s, slightly worse than Round 1

This is the most interesting part of the case.

Round 2 produces what looks like a “cleaner” code generation outcome:

- smaller binary
- better IPC
- fewer L1 misses

Yet runtime does not recover.

### 5.4 Why recovery stalls

The diagnostic log points to persistent **memory bottlenecks**:

- increased memory stalls
- unresolved `LoadClobber` behavior
- GVN and LICM blocked by alias-analysis limitations

So the optimization problem has shifted:

- in Round 1, the dominant issue is **over-expansion**
- in Round 2, the dominant issue becomes **memory dependence and alias-sensitive load redundancy**

At this point, pass tuning alone is not enough. The system has identified the problem correctly, but it does not yet have a sufficiently strong mechanism to resolve it.

### 5.5 Evaluation Agent’s next feedback

The Evaluation Agent recommends:

- **Suppress vectorization further**
    - increase `slp-threshold` more
- **Reduce unrolling further**
    - lower `unroll-count` to **4**
- **Strengthen alias handling**
    - enable more aggressive alias analysis
    - address persistent `LoadClobbered` points

This is a valid diagnosis. The challenge is that the current edit space has limited ability to change alias behavior deeply.

---

## 6. Round 3 — Conservative Tuning, Diminishing Returns

### 6.1 System Decision Logic

Round 3 adopts a more conservative strategy:

- `unroll-count = 4`
- `unroll-threshold = 500`
- `slp-threshold = 8`
- keep `inline-threshold = 550`
- add an extra `GVN` after `loop-unroll`

This is the most restrained and targeted pipeline among the three rounds.

### 6.2 What improves

Again, several metrics move in the right direction:

- **IPC** rises further to **2.49**
- **Code size** shrinks to **575.8 KB**
- **L1-dcache misses** fall again to **4.88M**

Relative to Round 1, the system clearly succeeds in reducing the damage from:

- excessive unrolling
- excessive SLP
- unnecessary code growth

### 6.3 Why runtime still does not improve

Even with those improvements:

- **Execution time remains 13.72 s**
- still about **27% slower** than baseline

The extra GVN pass helps only slightly. The log explicitly indicates:

- many remaining `LoadClobbered` remarks
- persistent alias-analysis barriers
- only diminishing returns from additional pass stacking

At this point, AutoPass is no longer obviously making “bad” local decisions. Instead, it is trapped in a regime where:

- the key bottleneck is not easy to repair using the available pass and parameter edits
- the structural damage from Round 1 has been reduced, but not reversed enough to restore the baseline execution regime

---

## 7. Why This Is a Useful Failure Case

This benchmark is informative for several reasons.

### 7.1 The system is explainable

The decisions are not arbitrary:

- Round 1 is driven by missed vectorization and hotspot analysis
- Round 2 reacts to code bloat and front-end/cache pressure
- Round 3 reacts to persistent memory stalls and alias-sensitive redundancy

Each stage is linked to real evidence.

### 7.2 The feedback loop is genuinely useful

From Round 1 to Round 3, AutoPass:

- reduces code size significantly
- improves IPC
- lowers cache misses

So the iterative loop is not just retrying randomly. It is performing meaningful correction.

### 7.3 Diagnosis does not guarantee recoverability

This is the key lesson.

AutoPass can correctly diagnose:

- too much unrolling
- too much inlining
- too much SLP
- persistent memory-side load redundancy

But diagnosis alone is not sufficient when the current action space cannot fully address the dominant cause of slowdown.

### 7.4 Current limitation of the edit space

The failure suggests that AutoPass is strongest when the optimization problem can be solved by:

- pass reordering
- pass repetition
- threshold tuning
- moderate structural edits

It is weaker when performance is dominated by:

- deep alias-sensitive memory behavior
- pointer-heavy load redundancy
- bottlenecks requiring stronger dataflow or memory disambiguation support

---

## 8. Main Takeaways

### Takeaway 1: aggressive first-round expansion can be dangerous

The most harmful step is Round 1. Once aggressive inlining, unrolling, and SLP push the benchmark into a poor microarchitectural regime, later rounds may only partially recover.

### Takeaway 2: instruction count is not enough

Round 1 reduces total instructions but still becomes much slower. This confirms that code-size growth, front-end throughput, and memory effects can dominate raw instruction count.

### Takeaway 3: iterative feedback still matters

Even though the final outcome remains poor, the system successfully diagnoses and mitigates part of the failure. That is still evidence that the agent loop is meaningful.

### Takeaway 4: memory-dominated cases need richer corrective operators

For benchmarks like JPEG decode, future improvements may require:

- stronger alias-analysis guidance
- tighter early-round size constraints
- explicit penalties for structural expansion
- memory-aware acceptance criteria
- richer passes or transformations beyond current threshold tuning

---