1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
|
<pre>Independent Submission D. Melman
Request for Comments: 6847 T. Mizrahi
Category: Informational Marvell
ISSN: 2070-1721 D. Eastlake 3rd
Huawei
January 2013
<span class="h1">Fibre Channel over Ethernet (FCoE) over</span>
<span class="h1">Transparent Interconnection of Lots of Links (TRILL)</span>
Abstract
Fibre Channel over Ethernet (FCoE) and Transparent Interconnection of
Lots of Links (TRILL) are two emerging standards in the data center
environment. While these two protocols are seemingly unrelated, they
have a very similar behavior in the forwarding plane, as both perform
hop-by-hop forwarding over Ethernet, modifying the packet's Media
Access Control (MAC) addresses at each hop. This document describes
an architecture for the integrated deployment of these two protocols.
Status of This Memo
This document is not an Internet Standards Track specification; it is
published for informational purposes.
This is a contribution to the RFC Series, independently of any other
RFC stream. The RFC Editor has chosen to publish this document at
its discretion and makes no statement about its value for
implementation or deployment. Documents approved for publication by
the RFC Editor are not a candidate for any level of Internet
Standard; see <a href="./rfc5741#section-2">Section 2 of RFC 5741</a>.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
<a href="http://www.rfc-editor.org/info/rfc6847">http://www.rfc-editor.org/info/rfc6847</a>.
Copyright Notice
Copyright (c) 2013 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and the IETF Trust's Legal
Provisions Relating to IETF Documents
(<a href="http://trustee.ietf.org/license-info">http://trustee.ietf.org/license-info</a>) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document.
<span class="grey">Melman, et al. Informational [Page 1]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
Table of Contents
<a href="#section-1">1</a>. Introduction ................................................. <a href="#page-2">2</a>
<a href="#section-2">2</a>. Abbreviations ................................................ <a href="#page-3">3</a>
<a href="#section-3">3</a>. FCoE over TRILL .............................................. <a href="#page-4">4</a>
<a href="#section-3.1">3.1</a>. FCoE over a TRILL Cloud ................................. <a href="#page-4">4</a>
<a href="#section-3.2">3.2</a>. FCoE over an RBridge .................................... <a href="#page-5">5</a>
<a href="#section-3.2.1">3.2.1</a>. FCRB ............................................... <a href="#page-5">5</a>
<a href="#section-3.2.2">3.2.2</a>. Topology ........................................... <a href="#page-7">7</a>
<a href="#section-3.2.3">3.2.3</a>. The FCRB Flow ..................................... <a href="#page-8">8</a>
<a href="#section-3.2.3.1">3.2.3.1</a>. Example - ENode to ENode ..................... <a href="#page-8">8</a>
<a href="#section-3.2.3.1.1">3.2.3.1.1</a>. Forwarding from A to C in Dense Mode .... <a href="#page-9">9</a>
<a href="#section-3.2.3.1.2">3.2.3.1.2</a>. Forwarding from A to C in Sparse Mode ... <a href="#page-9">9</a>
<a href="#section-3.2.3.2">3.2.3.2</a>. Example - ENode to Native FC Node ............ <a href="#page-10">10</a>
<a href="#section-3.2.3.3">3.2.3.3</a>. Example - ENode to ENode with Non-FCRB EoR ... <a href="#page-10">10</a>
3.2.3.4. Example - FCoE Control Traffic through an FCRB 11
<a href="#section-4">4</a>. Security Considerations ..................................... <a href="#page-12">12</a>
<a href="#section-5">5</a>. Acknowledgments ............................................. <a href="#page-12">12</a>
<a href="#section-6">6</a>. References .................................................. <a href="#page-12">12</a>
<a href="#section-6.1">6.1</a>. Normative References ................................... <a href="#page-12">12</a>
<a href="#section-6.2">6.2</a>. Informative References ................................. <a href="#page-12">12</a>
<span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span>
Data center networks are rapidly evolving towards a consolidated
approach, in which Ethernet is used as the common infrastructure for
all types of traffic. Storage traffic was traditionally dominated by
the Fibre Channel (FC) protocol suite. At the intersection between
these two technologies a new technology was born, Fibre Channel over
Ethernet (FCoE), in which native FC packets are encapsulated with an
FCoE encapsulation over an Ethernet header. FCoE is specified in
[<a href="#ref-FC-BB-5" title=""Information Technology - Fibre Channel - Backbone - 5 (FC-BB-5)"">FC-BB-5</a>]. (A future version of FCoE is under development and is
expected to be specified in a document to be referred to as FC-BB-6;
however, this is a work in progress and is beyond the scope of this
document.)
Traffic between two FCoE end nodes (ENodes) is forwarded through one
or more FCoE Forwarders (FCFs). An FCF takes a forwarding decision
based on the Fibre Channel destination ID (D_ID), and enforces
security policies between ENodes, also known as zoning. Once an FCF
takes a forwarding decision, it modifies the source and destination
MAC addresses of the packet, to reflect the path to the next-hop FCF
or ENode. An FCoE virtual link is an Ethernet link between an ENode
and an FCF, or between two FCFs. An FCoE virtual link may traverse
one or more Layer 2 bridges. FCFs use a routing protocol called
Fabric Shortest Path First (FSPF) to find the optimal path to each
<span class="grey">Melman, et al. Informational [Page 2]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
destination. An FCF typically has one or more native Fibre Channel
interfaces, allowing it to communicate with native Fibre Channel
devices, e.g., storage arrays.
TRILL [<a href="#ref-TRILL" title=""Routing Bridges (RBridges): Base Protocol Specification"">TRILL</a>] is a protocol for transparent least-cost routing, where
Routing Bridges (RBridges) forward traffic to their destination based
on a least-cost route, using a TRILL encapsulation header. RBridges
route TRILL-encapsulated packets based on the egress RBridge nickname
in the TRILL header. An RBridge routes a TRILL-encapsulated packet
after modifying its MAC addresses to reflect the path to the next-hop
RBridge and decrementing a Hop Count field.
TRILL and FCoE bear a strong resemblance in their forwarding planes.
Both protocols take a routing decision based on protocol addresses
above Layer 2, and both modify the Ethernet MAC addresses on a per-
hop basis. Each of the protocols uses its own routing protocol
rather than using any type of bridging protocol, such as the spanning
tree protocol [<a href="#ref-802.1Q" title=""IEEE Standard for Local and metropolitan area networks - Media Access Control (MAC) Bridges and Virtual Bridged Local Area Networks"">802.1Q</a>] or the Shortest Path Bridging protocol
[<a href="#ref-802.1Q" title=""IEEE Standard for Local and metropolitan area networks - Media Access Control (MAC) Bridges and Virtual Bridged Local Area Networks"">802.1Q</a>].
FCoE and TRILL are both targeted at the data center environment, and
their concurrent deployment is self-evident. This document describes
an architecture for the integrated deployment of these two protocols.
<span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Abbreviations</span>
DCB Data Center Bridging
ENode FCoE Node such as server or storage array
EoR End of Row
FC Fibre Channel
FCF FCoE Forwarder
FCoE Fibre Channel over Ethernet
FCRB FCF over RBridge
FIP FCoE Initialization Protocol
FSPF Fabric Shortest Path First
LAN Local Area Network
RBridge Routing Bridge
<span class="grey">Melman, et al. Informational [Page 3]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
SAN Storage Area Network
ToR Top of Rack
TRILL Transparent Interconnection of Lots of Links
WAN Wide Area Network
<span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. FCoE over TRILL</span>
<span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. FCoE over a TRILL Cloud</span>
The simplest approach for running FCoE traffic over a TRILL network
is presented in Figure 1. The figure illustrates a TRILL-enabled
network, in which FCoE traffic is transparently forwarded over the
TRILL cloud. The figure illustrates two ENodes, a Server and an FCoE
Storage Array, an FCF, and a native Fibre Channel SAN connected to
the FCF.
FCoE traffic between the two ENodes is sent from the first ENode over
the TRILL cloud to the FCF, and then back through the TRILL cloud to
the second ENode.
+---+
| |_________
| | \ ___ _
+---+ \/ \_/ \__ _ __
FCoE Storage _/ \ / \_/ \_
Array / TRILL / +---+ \_ \
(ENode A) \_ Cloud /________| |____/ SAN _/
/ \ | | \__ _/
\__/\_ ___/ +---+ \_/
+---+ / \_/ FCF
| |________/
| |
+---+
Server
(ENode B)
Figure 1. The "Separate Cloud" Approach
The configuration in Figure 1 separates the TRILL cloud(s) and the
FCoE cloud(s). The TRILL cloud routes FCoE traffic as standard
Ethernet traffic, and appears to the ENodes and FCF as an Ethernet
LAN. FCoE traffic routed over the TRILL cloud includes FCoE data
frames, as well as FCoE control traffic, including FCoE
<span class="grey">Melman, et al. Informational [Page 4]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
Initialization Protocol (FIP) frames. To eliminate frame loss due to
queue overflow, the switches in any TRILL Cloud used with FCoE would
likely implement and use the relevant DCB protocols [<a href="#ref-TRILLPFC" title=""TRILL: Support of IEEE 802.1 Priority- based Flow Control and Enhanced Transmission Selection"">TRILLPFC</a>]
[<a href="#ref-TRILLCN" title=""TRILL: Support of IEEE 802.1 Congestion Notification"">TRILLCN</a>].
The main drawbacks of the Separate Cloud approach are that RBridges
and FCFs are separate nodes in the network, resulting in more cabling
and boxes, and that communication between ENodes usually requires
traversing the TRILL cloud twice, so there are twice as many hops.
As mentioned above, data center networking is converging towards a
consolidated and cost-effective approach, where the same
infrastructure and equipment are used for both data and storage
traffic, and where high efficiency and minimal number of hops are
important factors when designing the network topology.
The Separate Cloud approach is presented as background to clarify the
motivation to develop an alternative approach with a higher level of
integration.
<span class="h3"><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. FCoE over an RBridge</span>
<span class="h4"><a class="selflink" id="section-3.2.1" href="#section-3.2.1">3.2.1</a>. FCRB</span>
Rather than using the Separate Cloud approach discussed in <a href="#section-3.1">Section</a>
<a href="#section-3.1">3.1</a>, an alternate approach is presented, where each switch
incorporates both an FCF entity and an RBridge entity. This
consolidated entity is referred to as FCoE-forwarder-over-RBridge
(FCRB).
Figure 2 illustrates an FCRB and its main building blocks. An FCRB
can be functionally viewed as two independent entities:
o An FCoE Forwarder (FCF) entity.
o An RBridge entity.
The FCF entity is connected to one of the ports of the RBridge, and
appears to the RBridge as a native Ethernet host. A detailed
description of the interaction between the layers is presented in
<a href="#section-3.2.3">Section 3.2.3</a>.
Note: In this document, the term "FCF" is used slightly differently
than defined in [<a href="#ref-FC-BB-5" title=""Information Technology - Fibre Channel - Backbone - 5 (FC-BB-5)"">FC-BB-5</a>] to emphasize the concept that an FCRB is
logically similar to an RBridge cascaded to an FCF. In the
terminology defined in [<a href="#ref-FC-BB-5" title=""Information Technology - Fibre Channel - Backbone - 5 (FC-BB-5)"">FC-BB-5</a>], an FCRB would be referred to as an
FCF, and the FCF building block in Figure 2 would be referred to as
an FC switching element.
<span class="grey">Melman, et al. Informational [Page 5]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
+-------------------+
|FCRB |
| +-----------+ | Native FC
| | FCF |------ Interface
| +-----+-----+ |
| | |
| +-----+-----+ |
| | RBridge | |
| +-+-+---+-+-+ |
| | | | | |
+-----|-|---|-|-----+
FCoE/ / | | |
+---+ Ethernet / / | | FCoE / Ethernet
| |___________________/ / | | over TRILL ___ _
| | / | | / \_/ \__
+---+ / | \_____________ _/ \
FCoE Storage / \_______________/ TRILL /
Array / \_ Cloud /
(ENode A) / / \
/ \__/\_ ___/
+---+ / \_/
| |______________/
| |
+---+
Server
(ENode B)
Figure 2. FCRB Entity in the Network
The FCRB entity maintains layer independence between the TRILL and
FCoE protocols, while enabling both protocols on the same network.
Note that FCoE traffic is always forwarded through an FCF and cannot
be forwarded directly between two ENodes. Thus, FCoE traffic between
ENodes A and B in the topology in Figure 1 is forwarded through the
path
ENode A-->TRILL cloud-->FCF-->TRILL cloud-->ENode B
As opposed to the topology in Figure 1, the FCF in Figure 2 is
adjacent to ENodes A and B. In Figure 2, the FCRB is connected to
ENodes A and B, and functions as the edge RBridge that connects these
two nodes to the TRILL cloud, as well as the FCF that forwards
traffic between these two nodes. Thus, traffic between ENodes A and
B in the topology in Figure 2 is forwarded through the path
ENode A-->FCRB-->ENode B
<span class="grey">Melman, et al. Informational [Page 6]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
Hence, the usage of FCRB entities allows TRILL and FCoE to use common
infrastructure and equipment, as opposed to requiring separate
infrastructure as shown in the Separate Cloud topology presented in
Figure 1.
<span class="h4"><a class="selflink" id="section-3.2.2" href="#section-3.2.2">3.2.2</a>. Topology</span>
The network configuration illustrated in Figure 3 shows a typical
topology of a data center network. Servers are hierarchically
connected through Top-of-Rack (ToR) switches, also known as access
switches, and each set of racks is aggregated through an End-of-Row
(EoR) switch. The EoR switches are aggregated to the core switches,
which may be connected to other clouds, such as an external WAN or a
native FC SAN.
_ __ _ __
/ \_/ \_ / \_/ \_
\_ \ \_ \ ....
/ SAN _/ / WAN _/
\__ _/ \__ _/
\_/ \_/
| |
| |
+------+ +------+
Core | | | |
FCoE over | | | |
RBridge | | | |
(FCRB) +------+ +------+
| \___ ___/ |
| \ / |
| \/ |
EoR +----+_______/\_______+----+
FCoE over | | | |
RBridge | | | |
(FCRB) +----+ +----+
/ \ / \
/ \ / \
ToR +---+ +---+ +---+ +---+
FCoE over | | | | | | | |
RBridge | | | | | | | |
(FCRB) +---+ +---+ +---+ +---+
/ \ / \ / \ / \
/ \ / \ / \ / \
+-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+
Servers/ | | | | | | | | | | | | | | | |
ENodes +-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+
A B C D E F G H
Figure 3. FCoE over RBridge Topology
<span class="grey">Melman, et al. Informational [Page 7]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
Note that in the example in Figure 3, all the ToR, EoR, and core
switches are FCRB entities, but it is also possible for some of the
network nodes to be pure RBridges, creating a topology in which FCRBs
are interconnected through TRILL clouds.
<span class="h4"><a class="selflink" id="section-3.2.3" href="#section-3.2.3">3.2.3</a>. The FCRB Flow</span>
<span class="h5"><a class="selflink" id="section-3.2.3.1" href="#section-3.2.3.1">3.2.3.1</a>. Example - ENode to ENode</span>
FCoE traffic sent between the two ENodes A and B in Figure 3 is
transmitted through the ToR FCRB, since A and B are connected to the
same ToR. Traffic between ENodes A and C must be forwarded through
the EoR FCRB.
The FCoE jargon distinguishes between two deployment modes:
o Sparse mode: an FCoE packet sent between two FCFs may be forwarded
over several hops of a Layer 2 network, allowing the underlying
Layer 2 network to determine the path between the two FCFs.
o Dense mode: each node along the path between two FCFs is also an
FCF, and the network is configured such that the forwarding
decision at each hop is taken at the FCF layer, allowing the path
between the two FCFs to be based on the FSPF routing protocol.
Figure 4 illustrates the traffic between ENodes A and C, which are
not connected to the same ToR. The following two subsections
describe the forwarding procedure in the Dense mode and in the Sparse
mode, respectively.
+--------+ +--------+ +--------+ +--------+ +--------+
| FCoE |.....| FCF |.....| FCF |.....| FCF |.....| FCoE |
| ENode | +--------+ +--------+ +--------+ | ENode |
| | |RBridge |.....|RBridge |.....|RBridge | | |
+--------+ +--------+ +--------+ +--------+ +--------+
|Ethernet|<===>|Ethernet|<===>|Ethernet|<===>|Ethernet|<===>|Ethernet|
+--------+ +--------+ +--------+ +--------+ +--------+
Server ToR 1 EoR ToR 2 FCoE Storage
ENode A FCRB FCRB FCRB Array
ENode C
Figure 4. Traffic between two ENodes - Example
<span class="grey">Melman, et al. Informational [Page 8]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
<span class="h6"><a class="selflink" id="section-3.2.3.1.1" href="#section-3.2.3.1.1">3.2.3.1.1</a>. Forwarding from A to C in Dense Mode</span>
o FCoE traffic from A is sent to ToR 1 over the Ethernet interface.
The destination MAC address is the address of the FCF entity at
ToR 1.
o ToR 1:
o The packet is forwarded to the FCF entity at the ToR. Thus,
forwarding between ENode A and the FCF at the ToR is
analogous to forwarding between two Ethernet hosts.
o The FCF entity at the ToR takes a forwarding decision based
on the FC addresses. This decision is based on the FSPF
routing protocol at the FCF layer. The FCF entity at the
ToR forwards the packet to the FCF entity in the EoR.
o The FCF then updates the destination MAC address of the
packet to the address of the EoR FCF.
o The packet is forwarded to the RBridge entity, where it is
encapsulated in a TRILL header, and sent to the RBridge at
the EoR over a single hop of the TRILL network.
o The RBridge entity in the EoR FCRB, acting as the egress RBridge,
decapsulates the TRILL header and forwards the FCoE packet to the
FCF entity. From this point, the forwarding process is similar to
the one described above for the ToR.
o A similar forwarding process takes place at the next-hop ToR FCRB,
where the FCRB finally forwards the FCoE packet to the target,
ENode C.
<span class="h6"><a class="selflink" id="section-3.2.3.1.2" href="#section-3.2.3.1.2">3.2.3.1.2</a>. Forwarding from A to C in Sparse Mode</span>
o Traffic is forwarded to ToR 1, as described in <a href="#section-3.2.3.1.1">Section 3.2.3.1.1</a>.
o The FCF in ToR 1, based on an FSPF forwarding decision, forwards
the packet to the FCF in ToR 2. The destination MAC address of
the FCoE packet is updated, reflecting the FCF in ToR 2. The
RBridge entity in ToR 2 adds a TRILL encapsulation, with an egress
RBridge nickname representing ToR 2.
o The packet reaches the EoR. The RBridge entity in the EoR routes
the packet to the RBridge entity in ToR 2.
o The packet reaches ToR 2. From this point on, the process is
identical to the one described in <a href="#section-3.2.3.1.1">Section 3.2.3.1.1</a>.
<span class="grey">Melman, et al. Informational [Page 9]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
<span class="h5"><a class="selflink" id="section-3.2.3.2" href="#section-3.2.3.2">3.2.3.2</a>. Example - ENode to Native FC Node</span>
+--------+ +--------+ +--------+ +---------+ +--------+
| FCoE |.....| FCF |.....| FCF |.....| FCF |.....| FC |
| ENode | +--------+ +--------+ +----+----+ |protocol|
| | |RBridge |.....|RBridge |.....| RB | | | stack |
+--------+ +--------+ +--------+ +----+ FC | | |
|Ethernet|<===>|Ethernet|<===>|Ethernet|<===>|Eth | |<===>| |
+--------+ +--------+ +--------+ +----+----+ +--------+
Server ToR EoR Core Native FC
ENode FCRB FCRB FCRB Storage Array
Figure 5. Example of Traffic between an
ENode and a Native FC Storage Array
Figure 5 illustrates a second example, where traffic is sent between
an ENode and an FC Storage Array, based on the network topology in
Figure 3.
o FCoE traffic from the ENode is sent to the ToR over the Ethernet
interface. The forwarding process through the ToR FCRB and
through the EoR is similar to the corresponding steps in <a href="#section-3.2.3.1">Section</a>
<a href="#section-3.2.3.1">3.2.3.1</a>.
o When the packet reaches the core FCRB, the egress RBridge entity
decapsulates the TRILL header and forwards the FCoE packet to the
FCF entity. The packet is then forwarded as a native FC packet
through the FC interface to the native FC node.
<span class="h5"><a class="selflink" id="section-3.2.3.3" href="#section-3.2.3.3">3.2.3.3</a>. Example - ENode to ENode with Non-FCRB EoR</span>
The example illustrated in Figure 6 is similar to the one shown in
Figure 4, except that the EoR is an RBridge rather than an FCRB.
+--------+ +--------+ +--------+ +--------+
| FCoE |.....| FCF |....................| FCF |.....| FCoE |
| ENode | +--------+ +--------+ +--------+ | ENode |
| | |RBridge |.....|RBridge |.....|RBridge | | |
+--------+ +--------+ +--------+ +--------+ +--------+
|Ethernet|<===>|Ethernet|<===>|Ethernet|<===>|Ethernet|<===>|Ethernet|
+--------+ +--------+ +--------+ +--------+ +--------+
Server ToR 1 EoR ToR 2 FCoE Storage
ENode A FCRB FCRB FCRB Array
ENode C
Figure 6. Example of Traffic between Two ENodes
<span class="grey">Melman, et al. Informational [Page 10]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
An FCoE packet sent from ENode A to C is forwarded as follows:
o The packet is sent to the FCF in ToR 1, as in the previous
example.
o The FCF in ToR 1 takes a forwarding decision based on the FC
addresses and forwards the packet to the next-hop FCF, which
resides in ToR 2. This forwarding decision is taken at the FCF
layer and is based on the FSPF routing protocol.
o The packet is then forwarded to the RBridge entity in ToR 1, where
it is encapsulated in a TRILL encapsulation, and forwarded to the
RBridge at ToR 2. The packet is routed over the TRILL cloud
through the RBridge at the EoR. The path through the TRILL cloud
is determined by TRILL's IS-IS routing protocol.
o Once the packet reaches ToR 2, it is forwarded in a similar manner
to the description in <a href="#section-3.2.3.1">Section 3.2.3.1</a>.
This example demonstrates that it is possible to have a hybrid
network, in which some of the nodes are FCRBs and some of the nodes
are RBridges. The forwarding procedure in this example is somewhat
similar to the sparse-mode forwarding described in <a href="#section-3.2.3.1.2">Section 3.2.3.1.2</a>.
<span class="h5"><a class="selflink" id="section-3.2.3.4" href="#section-3.2.3.4">3.2.3.4</a>. Example - FCoE Control Traffic through an FCRB</span>
The previous subsections focused on the data plane, i.e., storage
data exchanges transported over an FCoE encapsulation. FCoE also
requires control and management traffic that is used for initializing
sessions (i.e., FIP), distributing routing information (i.e., FSPF),
and administering and managing fabric.
The FCoE Initialization Protocol (FIP) uses Ethernet frames with a
dedicated Ethertype, allowing the FCF to distinguish these frames
from other traffic. FIP uses both unicast and multicast traffic.
The following example describes the forwarding scheme of a multicast
FIP packet sent through the network depicted in Figure 4:
o ENode A generates a multicast frame to a multicast MAC address
that represents all the FCFs (All-FCF-MAC).
o The packet is forwarded to the ToR FCRB node. The RBridge entity
forwards a copy of the packet to its FCF entity, and also sends
the packet through the TRILL cloud as a multicast TRILL
encapsulated packet.
<span class="grey">Melman, et al. Informational [Page 11]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
o Each of the FCRBs then receives the packet, forwards a copy to its
FCF entity, and forwards the packet through the TRILL network,
allowing all the FCFs to receive the packet.
While FIP packets have a dedicated Ethertype and frame format, other
types of FCoE control and management frames use the same FCoE
encapsulation as FCoE data traffic. Thus, the forwarding scheme for
such control traffic is similar to the examples described in the
previous subsections, with the exception that these frames can be
sent between ENodes, between FCFs, or between ENodes and FCFs.
<span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Security Considerations</span>
For general TRILL security considerations, see [<a href="#ref-TRILL" title=""Routing Bridges (RBridges): Base Protocol Specification"">TRILL</a>].
For general FCoE security considerations, see Annex D of [<a href="#ref-FC-BB-5" title=""Information Technology - Fibre Channel - Backbone - 5 (FC-BB-5)"">FC-BB-5</a>].
There are no additional security implications imposed by this
document.
<span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Acknowledgments</span>
The authors gratefully acknowledge Ayandeh Siamack and David Black
for their helpful comments. The authors also thank the T11 committee
for reviewing the document, and in particular Pat Thaler and Joe
White for their useful input.
<span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. References</span>
<span class="h3"><a class="selflink" id="section-6.1" href="#section-6.1">6.1</a>. Normative References</span>
[<a id="ref-TRILL">TRILL</a>] Perlman, R., Eastlake 3rd, D., Dutt, D., Gai, S., and A.
Ghanwani, "Routing Bridges (RBridges): Base Protocol
Specification", <a href="./rfc6325">RFC 6325</a>, July 2011.
[<a id="ref-FC-BB-5">FC-BB-5</a>] ANSI INCITS 462: "Information Technology - Fibre Channel -
Backbone - 5 (FC-BB-5)", May 2010.
<span class="h3"><a class="selflink" id="section-6.2" href="#section-6.2">6.2</a>. Informative References</span>
[<a id="ref-802.1Q">802.1Q</a>] "IEEE Standard for Local and metropolitan area networks -
Media Access Control (MAC) Bridges and Virtual Bridged
Local Area Networks", IEEE Std 802.1Q(tm), 2012 Edition,
October 2012.
<span class="grey">Melman, et al. Informational [Page 12]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span>
<span class="grey"><a href="./rfc6847">RFC 6847</a> FCoE over TRILL January 2013</span>
[<a id="ref-TRILLPFC">TRILLPFC</a>] Eastlake 3rd, D., Wadekar, M., Ghanwani, A., Agarwal, P.,
and T. Mizrahi, "TRILL: Support of IEEE 802.1 Priority-
based Flow Control and Enhanced Transmission Selection",
Work in Progress, January 2013.
[<a id="ref-TRILLCN">TRILLCN</a>] Eastlake 3rd, D., Wadekar, M., Ghanwani, A., Agarwal, P.,
and T. Mizrahi, "TRILL: Support of IEEE 802.1 Congestion
Notification", Work in Progress, January 2013.
Authors' Addresses
David Melman
Marvell
6 Hamada St.
Yokneam, 20692 Israel
EMail: davidme@marvell.com
Tal Mizrahi
Marvell
6 Hamada St.
Yokneam, 20692 Israel
EMail: talmi@marvell.com
Donald Eastlake 3rd
Huawei USA R&D
155 Beaver Street
Milford, MA 01757 USA
Phone: +1-508-333-2270
EMail: d3e3e3@gmail.com
Melman, et al. Informational [Page 13]
</pre>
|
