| Acknowledgments |
|
xvii | |
| Preface |
|
xix | |
| About the Authors |
|
xxv | |
| Chapter 1 Introduction to Interconnection Networks |
|
1 | (24) |
|
1.1 Three Questions About Interconnection Networks |
|
|
2 | (2) |
|
1.2 Uses of Interconnection Networks |
|
|
4 | (9) |
|
1.2.1 Processor-Memory Interconnect |
|
|
5 | (3) |
|
|
|
8 | (3) |
|
1.2.3 Packet switching Fabric |
|
|
11 | (2) |
|
|
|
13 | (8) |
|
|
|
13 | (3) |
|
|
|
16 | (1) |
|
|
|
17 | (2) |
|
1.3.4 Router Architecture |
|
|
19 | (1) |
|
1.3.5 Performance of Interconnection Networks |
|
|
19 | (2) |
|
|
|
21 | (2) |
|
1.5 Organization of this Book |
|
|
23 | (2) |
| Chapter 2 A simple Interconnection Network |
|
25 | (20) |
|
2.1 Network Specifications and Constraints |
|
|
25 | (2) |
|
|
|
27 | (4) |
|
|
|
31 | (1) |
|
|
|
32 | (1) |
|
|
|
33 | (3) |
|
|
|
36 | (6) |
|
|
|
42 | (3) |
| Chapter 3 Topology Basics |
|
45 | (30) |
|
|
|
46 | (4) |
|
|
|
46 | (1) |
|
3.1.2 Direct and Indirect Networks |
|
|
47 | (1) |
|
3.1.3 Cuts and Bisections |
|
|
48 | (1) |
|
|
|
48 | (1) |
|
|
|
49 | (1) |
|
|
|
50 | (1) |
|
|
|
51 | (9) |
|
3.3.1 Throughput and Maximum Channel Load |
|
|
51 | (4) |
|
|
|
55 | (2) |
|
|
|
57 | (3) |
|
|
|
60 | (4) |
|
3.5 Case Study: The SGI Origin 2000 |
|
|
64 | (5) |
|
|
|
69 | (1) |
|
|
|
69 | (6) |
| Chapter 4 Butterfly Networks |
|
75 | (14) |
|
4.1 The Structure of Butterfly Networks |
|
|
75 | (2) |
|
4.2 Isomorphic Butterflies |
|
|
77 | (1) |
|
4.3 Performance and Packaging Cost |
|
|
78 | (3) |
|
4.4 Path Diversity and Extra stages |
|
|
81 | (3) |
|
4.5 Case Study: The BBN Butterfly |
|
|
84 | (2) |
|
|
|
86 | (1) |
|
|
|
86 | (3) |
| Chapter 5 Torus Networks |
|
89 | (22) |
|
5.1 The Structure of Torus Networks |
|
|
90 | (2) |
|
|
|
92 | (6) |
|
|
|
92 | (3) |
|
|
|
95 | (1) |
|
|
|
96 | (2) |
|
5.3 Building Mesh and Torus Networks |
|
|
98 | (2) |
|
|
|
100 | (2) |
|
5.5 Case Study: The MIT J-Machine |
|
|
102 | (4) |
|
|
|
106 | (1) |
|
|
|
107 | (4) |
| Chapter 6 Non-Blocking Networks |
|
111 | (34) |
|
6.1 Non-Blocking vs. Non-Interfering Networks |
|
|
112 | (1) |
|
|
|
112 | (4) |
|
|
|
116 | (18) |
|
6.3.1 structure and Properties of Clos Networks |
|
|
116 | (2) |
|
6.3.2 Unicast Routing on strictly Non-Blocking Clos Networks |
|
|
118 | (4) |
|
6.3.3 Unicast Routing on Rearrangeable Clos Networks |
|
|
122 | (4) |
|
6.3.4 Routing Clos Networks Using Matrix Decomposition |
|
|
126 | (2) |
|
6.3.5 Multicast Routing on Clos Networks |
|
|
128 | (5) |
|
6.3.6 Clos Networks with More Than Three stages |
|
|
133 | (1) |
|
|
|
134 | (1) |
|
|
|
135 | (2) |
|
6.6 Case Study: The Velio VC2002 (Zeus) Grooming switch |
|
|
137 | (5) |
|
|
|
142 | (1) |
|
|
|
142 | (3) |
| Chapter 7 Slicing and Dicing |
|
145 | (14) |
|
7.1 Concentrators and Distributors |
|
|
146 | (3) |
|
|
|
146 | (2) |
|
|
|
148 | (1) |
|
|
|
149 | (4) |
|
|
|
149 | (2) |
|
|
|
151 | (1) |
|
|
|
152 | (1) |
|
7.3 Slicing Multistage Networks |
|
|
153 | (2) |
|
7.4 Case Study: Bit Slicing in the Tiny Tera |
|
|
155 | (2) |
|
|
|
157 | (1) |
|
|
|
157 | (2) |
| Chapter 8 Routing Basics |
|
159 | (14) |
|
|
|
160 | (2) |
|
8.2 Taxonomy of Routing Algorithms |
|
|
162 | (1) |
|
|
|
163 | (1) |
|
8.4 Deterministic Routing |
|
|
164 | (4) |
|
8.4.1 Destination-Tag Routing in Butterfly Networks |
|
|
165 | (1) |
|
8.4.2 Dimension-Order Routing in Cube Networks |
|
|
166 | (2) |
|
8.5 Case Study: Dimension-Order Routing in the Cray T3D |
|
|
168 | (2) |
|
|
|
170 | (1) |
|
|
|
171 | (2) |
| Chapter 9 Oblivious Routing |
|
173 | (16) |
|
9.1 Valiant's Randomized Routing Algorithm |
|
|
174 | (2) |
|
9.1.1 Valiant's Algorithm on Torus Topologies |
|
|
174 | (1) |
|
9.1.2 Valiant's Algorithm on Indirect Networks |
|
|
175 | (1) |
|
9.2 Minimal Oblivious Routing |
|
|
176 | (4) |
|
9.2.1 Minimal Oblivious Routing on a Folded Clos (Fat Tree) |
|
|
176 | (2) |
|
9.2.2 Minimal Oblivious Routing on a Torus |
|
|
178 | (2) |
|
9.3 Load-Balanced Oblivious Routing |
|
|
180 | (1) |
|
9.4 Analysis of Oblivious Routing |
|
|
180 | (3) |
|
9.5 Case Study: Oblivious Routing in the Avici Terabit Switch Router(TSR) |
|
|
183 | (3) |
|
|
|
186 | (1) |
|
|
|
187 | (2) |
| Chapter 10 Adaptive Routing |
|
189 | (14) |
|
10.1 Adaptive Routing Basics |
|
|
189 | (3) |
|
10.2 Minimal Adaptive Routing |
|
|
192 | (1) |
|
10.3 Fully Adaptive Routing |
|
|
193 | (2) |
|
10.4 Load-Balanced Adaptive Routing |
|
|
195 | (1) |
|
10.5 Search-Based Routing |
|
|
196 | (1) |
|
10.6 Case Study: Adaptive Routing in the Thinking Machines CM-5 |
|
|
196 | (5) |
|
|
|
201 | (1) |
|
|
|
201 | (2) |
| Chapter 11 Routing Mechanics |
|
203 | (18) |
|
|
|
203 | (8) |
|
|
|
204 | (4) |
|
11.1.2 Node-Table Routing |
|
|
208 | (3) |
|
|
|
211 | (1) |
|
11.3 Case Study: Oblivious Source Routing in the IBM Vulcan Network |
|
|
212 | (5) |
|
|
|
217 | (1) |
|
|
|
217 | (4) |
| Chapter 12 Flow Control Basics |
|
221 | (12) |
|
12.1 Resources and Allocation Units |
|
|
222 | (3) |
|
12.2 Bufferless Flow Control |
|
|
225 | (3) |
|
|
|
228 | (2) |
|
|
|
230 | (1) |
|
|
|
230 | (3) |
| Chapter 1 3 Buffered Flow Control |
|
233 | (24) |
|
13.1 Packet-Buffer Flow Control |
|
|
234 | (3) |
|
13.2 Flit-Buffer Flow Control |
|
|
237 | (8) |
|
13.2.1 Wormhole Flow Control |
|
|
237 | (2) |
|
13.2.2 Virtual-Channel Flow Control |
|
|
239 | (6) |
|
13.3 Buffer Management and Backpressure |
|
|
245 | (6) |
|
13.3.1 Credit-Based Flow Control |
|
|
245 | (2) |
|
13.3.2 On/Off Flow Control |
|
|
247 | (2) |
|
13.3.3 Ack/Nack Flow Control |
|
|
249 | (2) |
|
13.4 Flit-Reservation Flow Control |
|
|
251 | (5) |
|
13.4.1 A Flit-Reservation Router |
|
|
252 | (1) |
|
|
|
253 | (2) |
|
|
|
255 | (1) |
|
|
|
256 | (1) |
|
|
|
256 | (1) |
| Chapter 14 Deadlock and Livelock |
|
257 | (28) |
|
|
|
258 | (5) |
|
14.1.1 Agents and Resources |
|
|
258 | (1) |
|
14.1.2 Wait-For and Holds Relations |
|
|
259 | (1) |
|
14.1.3 Resource Dependences |
|
|
260 | (1) |
|
|
|
260 | (2) |
|
14.1.5 High-Level (Protocol) Deadlock |
|
|
262 | (1) |
|
|
|
263 | (9) |
|
|
|
263 | (4) |
|
14.2.2 Restricted Physical Routes |
|
|
267 | (3) |
|
14.2.3 Hybrid Deadlock Avoidance |
|
|
270 | (2) |
|
|
|
272 | (5) |
|
14.3.1 Routing Subfunctions and Extended Dependences |
|
|
272 | (4) |
|
14.3.2 Duato's Protocol for Deadlock-Free Adaptive Algorithms |
|
|
276 | (1) |
|
|
|
277 | (2) |
|
14.4.1 Regressive Recovery |
|
|
278 | (1) |
|
14.4.2 Progressive Recovery |
|
|
278 | (1) |
|
|
|
279 | (1) |
|
14.6 Case Study: Deadlock Avoidance in the Cray T3E |
|
|
279 | (2) |
|
|
|
281 | (1) |
|
|
|
282 | (3) |
| Chapter 15 Quality of service |
|
285 | (20) |
|
15.1 Service Classes and Service Contracts |
|
|
285 | (2) |
|
15.2 Burstiness and Network Delays |
|
|
287 | (3) |
|
15.2.1 (sigma,rho) Regulated Flows |
|
|
287 | (1) |
|
15.2.2 Calculating Delays |
|
|
288 | (2) |
|
15.3 Implementation of Guaranteed services |
|
|
290 | (4) |
|
15.3.1 Aggregate Resource Allocation |
|
|
291 | (1) |
|
15.3.2 Resource Reservation |
|
|
292 | (2) |
|
15.4 Implementation of Best-Effort services |
|
|
294 | (3) |
|
|
|
294 | (2) |
|
15.4.2 Throughput Fairness |
|
|
296 | (1) |
|
15.5 Separation of Resources |
|
|
297 | (2) |
|
|
|
297 | (2) |
|
15.5.2 Non-interfering Networks |
|
|
299 | (1) |
|
15.6 Case Study: ATM Service Classes |
|
|
299 | (1) |
|
15.7 Case Study: Virtual Networks in the Avici TSR |
|
|
300 | (2) |
|
|
|
302 | (1) |
|
|
|
303 | (2) |
| Chapter 16 Router Architecture |
|
305 | (20) |
|
16.1 Basic Router Architecture |
|
|
305 | (5) |
|
|
|
305 | (3) |
|
16.1.2 The Router Pipeline |
|
|
308 | (2) |
|
|
|
310 | (2) |
|
16.3 Closing the Loop with Credits |
|
|
312 | (1) |
|
16.4 Reallocating a Channel |
|
|
313 | (3) |
|
16.5 Speculation and Lookahead |
|
|
316 | (3) |
|
16.6 Flit and Credit Encoding |
|
|
319 | (2) |
|
16.7 Case study: The Alpha 21364 Router |
|
|
321 | (3) |
|
|
|
324 | (1) |
|
|
|
324 | (1) |
| Chapter 17 Router Datapath Components |
|
325 | (24) |
|
17.1 Input Buffer Organization |
|
|
325 | (9) |
|
17.1.1 Buffer Partitioning |
|
|
326 | (2) |
|
17.1.2 Input Buffer Data structures |
|
|
328 | (5) |
|
17.1.3 Input Buffer Allocation |
|
|
333 | (1) |
|
|
|
334 | (9) |
|
|
|
335 | (3) |
|
|
|
338 | (4) |
|
|
|
342 | (1) |
|
|
|
343 | (1) |
|
17.4 Case Study: The Datapath of the IBM Colony Router |
|
|
344 | (3) |
|
|
|
347 | (1) |
|
|
|
348 | (1) |
| Chapter 18 Arbitration |
|
349 | (14) |
|
|
|
349 | (2) |
|
|
|
351 | (1) |
|
18.3 Fixed Priority Arbiter |
|
|
352 | (2) |
|
18.4 Variable Priority Iterative Arbiters |
|
|
354 | (4) |
|
18.4.1 Oblivious Arbiters |
|
|
354 | (1) |
|
18.4.2 Round-Robin Arbiter |
|
|
355 | (1) |
|
18.4.3 Grant-Hold Circuit |
|
|
355 | (2) |
|
18.4.4 Weighted Round-Robin Arbiter |
|
|
357 | (1) |
|
|
|
358 | (2) |
|
|
|
360 | (2) |
|
|
|
362 | (1) |
| Chapter 19 Allocation |
|
363 | (26) |
|
|
|
363 | (3) |
|
|
|
366 | (1) |
|
19.3 Separable Allocators |
|
|
367 | (6) |
|
19.3.1 Parallel Iterative Matching |
|
|
371 | (1) |
|
|
|
371 | (1) |
|
19.3.3 Lonely Output Allocator |
|
|
372 | (1) |
|
|
|
373 | (3) |
|
19.5 Incremental vs. Batch Allocation |
|
|
376 | (2) |
|
19.6 Multistage Allocation |
|
|
378 | (2) |
|
19.7 Performance of Allocators |
|
|
380 | (3) |
|
19.8 Case Study: The Tiny Tera Allocator |
|
|
383 | (2) |
|
|
|
385 | (1) |
|
|
|
386 | (3) |
| Chapter 20 Network Interfaces |
|
389 | (22) |
|
20.1 Processor-Network Interface |
|
|
390 | (4) |
|
20.1.1 Two-Register Interface |
|
|
391 | (1) |
|
20.1.2 Register-Mapped Interface |
|
|
392 | (1) |
|
20.1.3 Descriptor-Based Interface |
|
|
393 | (1) |
|
|
|
393 | (1) |
|
20.2 Shared-Memory Interface |
|
|
394 | (6) |
|
20.2.1 Processor-Network Interface |
|
|
395 | (2) |
|
|
|
397 | (1) |
|
20.2.3 Memory-Network Interface |
|
|
398 | (2) |
|
20.3 Line-Fabric Interface |
|
|
400 | (3) |
|
20.4 Case Study: The MIT M-Machine Network Interface |
|
|
403 | (4) |
|
|
|
407 | (1) |
|
|
|
408 | (3) |
| Chapter 21 Error Control |
|
411 | (16) |
|
21.1 Know Thy Enemy: Failure Modes and Fault Models |
|
|
411 | (3) |
|
21.2 The Error Control Process: Detection, Containment, and Recovery |
|
|
414 | (1) |
|
21.3 Link Level Error Control |
|
|
415 | (6) |
|
|
|
415 | (1) |
|
21.3.2 Link-Level Retransmission |
|
|
416 | (3) |
|
21.3.3 Channel Reconfiguration, Degradation, and shutdown |
|
|
419 | (2) |
|
21.4 Router Error Control |
|
|
421 | (1) |
|
21.5 Network-Level Error Control |
|
|
422 | (1) |
|
21.6 End-to-end Error Control |
|
|
423 | (1) |
|
|
|
423 | (1) |
|
|
|
424 | (3) |
| Chapter 22 Buses |
|
427 | (22) |
|
|
|
428 | (4) |
|
|
|
432 | (4) |
|
22.3 High Performance Bus Protocol |
|
|
436 | (5) |
|
|
|
436 | (2) |
|
22.3.2 Split-Transaction Buses |
|
|
438 | (1) |
|
|
|
439 | (2) |
|
22.4 From Buses to Networks |
|
|
441 | (2) |
|
22.5 Case Study: The PCI Bus |
|
|
443 | (3) |
|
|
|
446 | (1) |
|
|
|
446 | (3) |
| Chapter 23 Performance Analysis |
|
449 | (24) |
|
23.1 Measures of Interconnection Network Performance |
|
|
449 | (11) |
|
|
|
452 | (3) |
|
|
|
455 | (1) |
|
|
|
456 | (1) |
|
23.1.4 Common Measurement Pitfalls |
|
|
456 | (4) |
|
|
|
460 | (7) |
|
|
|
461 | (4) |
|
23.2.2 Probabilistic Analysis |
|
|
465 | (2) |
|
|
|
467 | (1) |
|
23.4 Case Study: Efficiency and Loss in the BBN Monarch Network |
|
|
468 | (2) |
|
|
|
470 | (1) |
|
|
|
471 | (2) |
| Chapter 24 Simulation |
|
473 | (22) |
|
|
|
473 | (2) |
|
|
|
475 | (3) |
|
24.2.1 Application-Driven Workloads |
|
|
475 | (1) |
|
24.2.2 Synthetic Workloads |
|
|
476 | (2) |
|
24.3 simulation Measurements |
|
|
478 | (6) |
|
|
|
479 | (2) |
|
24.3.2 Steady-State sampling |
|
|
481 | (1) |
|
24.3.3 Confidence Intervals |
|
|
482 | (2) |
|
|
|
484 | (7) |
|
24.4.1 Simulation Approaches |
|
|
485 | (3) |
|
24.4.2 Modeling Source Queues |
|
|
488 | (2) |
|
24.4.3 Random Number Generation |
|
|
490 | (1) |
|
|
|
491 | (1) |
|
|
|
491 | (1) |
|
|
|
492 | (3) |
| Chapter 25 Simulation Examples |
|
495 | (16) |
|
|
|
495 | (5) |
|
|
|
496 | (3) |
|
25.1.2 Throughput Distributions |
|
|
499 | (1) |
|
25.2 Flow Control Performance |
|
|
500 | (8) |
|
|
|
500 | (2) |
|
|
|
502 | (1) |
|
25.2.3 Injection Processes |
|
|
503 | (2) |
|
|
|
505 | (2) |
|
|
|
507 | (1) |
|
|
|
508 | (3) |
| Appendix A Nomenclature |
|
511 | (4) |
| Appendix B Glossary |
|
515 | (6) |
| Appendix C Network Simulator |
|
521 | (2) |
| Bibliography |
|
523 | (16) |
| Index |
|
539 | |