From layers to hardware
Where does the time go?
Model layers
Attention→FFN→…
Operators
Matmul→GELU→Matmul
Operator = one operation on tensors.
Batch size
8
Decoding
Draft→Target verify→Accept / correct
Operators & workflow
Model order gives dependencies. Chunks can overlap.
Before
After
Same workload · shared clock · same time scale
Memory-bound · compute waits for data.
0 / 17 ticks