1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
|
<pre>Network Working Group D. McPherson
Request for Comments: 3345 TCB
Category: Informational V. Gill
AOL Time Warner, Inc.
D. Walton
A. Retana
Cisco Systems, Inc.
August 2002
<span class="h1">Border Gateway Protocol (BGP) Persistent Route Oscillation Condition</span>
Status of this Memo
This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2002). All Rights Reserved.
Abstract
In particular configurations, the BGP scaling mechanisms defined in
"BGP Route Reflection - An Alternative to Full Mesh IBGP" and
"Autonomous System Confederations for BGP" will introduce persistent
BGP route oscillation. This document discusses the two types of
persistent route oscillation that have been identified, describes
when these conditions will occur, and provides some network design
guidelines to avoid introducing such occurrences.
<span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span>
The Border Gateway Protocol (BGP) is an inter-Autonomous System
routing protocol. The primary function of a BGP speaking system is
to exchange network reachability information with other BGP systems.
In particular configurations, the BGP [<a href="#ref-1" title=""A Border Gateway Protocol 4 (BGP-4)"">1</a>] scaling mechanisms defined
in "BGP Route Reflection - An Alternative to Full Mesh IBGP" [<a href="#ref-2" title=""BGP Route Reflection - An Alternative to Full Mesh IBGP"">2</a>] and
"Autonomous System Confederations for BGP" [<a href="#ref-3" title=""Autonomous System Confederations for BGP"">3</a>] will introduce
persistent BGP route oscillation.
The problem is inherent in the way BGP works: locally defined routing
policies may conflict globally, and certain types of conflicts can
cause persistent oscillation of the protocol. Given current
practices, we happen to see the problem manifest itself in the
context of MED + route reflectors or confederations.
<span class="grey">McPherson, et al. Informational [Page 1]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
The current specification of BGP-4 [<a href="#ref-4" title=""A Border Gateway Protocol 4 (BGP-4)"">4</a>] states that the
MULTI_EXIT_DISC is only comparable between routes learned from the
same neighboring AS. This limitation is consistent with the
description of the attribute: "The MULTI_EXIT_DISC attribute may be
used on external (inter-AS) links to discriminate among multiple exit
or entry points to the same neighboring AS." [<a href="#ref-1" title=""A Border Gateway Protocol 4 (BGP-4)"">1</a>,<a href="#ref-4" title=""A Border Gateway Protocol 4 (BGP-4)"">4</a>]
In a full mesh iBGP network, all the internal routers have complete
visibility of the available exit points into a neighboring AS. The
comparison of the MULTI_EXIT_DISC for only some paths is not a
problem.
Because of the scalability implications of a full mesh iBGP network,
two alternatives have been standardized: route reflectors [<a href="#ref-2" title=""BGP Route Reflection - An Alternative to Full Mesh IBGP"">2</a>] and AS
confederations [<a href="#ref-3" title=""Autonomous System Confederations for BGP"">3</a>]. Both alternatives describe methods by which
route distribution may be achieved without a full iBGP mesh in an AS.
The route reflector alternative defines the ability to re-advertise
(reflect) iBGP-learned routes to other iBGP peers once the best path
is selected [<a href="#ref-2" title=""BGP Route Reflection - An Alternative to Full Mesh IBGP"">2</a>]. AS Confederations specify the operation of a
collection of autonomous systems under a common administration as a
single entity (i.e. from the outside, the internal topology and the
existence of separate autonomous systems are not visible). In both
cases, the reduction of the iBGP full mesh results in the fact that
not all the BGP speakers in the AS have complete visibility of the
available exit points into a neighboring AS. In fact, the visibility
may be partial and inconsistent depending on the location (and
function) of the router in the AS.
In certain topologies involving either route reflectors or
confederations (detailed description later in this document), the
partial visibility of the available exit points into a neighboring AS
may result in an inconsistent best path selection decision as the
routers don't have all the relevant information. If the
inconsistencies span more than one peering router, they may result in
a persistent route oscillation. The best path selection rules
applied in this document are consistent with the current
specification [<a href="#ref-4" title=""A Border Gateway Protocol 4 (BGP-4)"">4</a>].
The persistent route oscillation behavior is deterministic and can be
avoided by employing some rudimentary BGP network design principles
until protocol enhancements resolve the problem.
In the following sections a taxonomy of the types of oscillations is
presented and a description of the set of conditions that will
trigger route oscillations is given. We continue by providing
several network design alternatives that remove the potential of this
occurrence.
<span class="grey">McPherson, et al. Informational [Page 2]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
It is the intent of the authors that this document serve to increase
operator awareness of the problem, as well as to trigger discussion
and subsequent proposals for potential protocol enhancements that
remove the possibility of this to occur.
The oscillations are classified into Type I and Type II depending
upon the criteria documented below.
<span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Discussion of Type I Churn</span>
In the following two subsections we provide configurations under
which Type I Churn will occur. We begin with a discussion of the
problem when using Route Reflection, and then discuss the problem as
it relates to AS Confederations.
In general, Type I Churn occurs only when BOTH of the following
conditions are met:
1) a single-level Route Reflection or AS Confederations design is
used in the network AND
2) the network accepts the BGP MULTI_EXIT_DISC (MED) attribute
from two or more ASs for a single prefix and the MED values are
unique.
It is also possible for the non-deterministic ordering of paths to
cause the route oscillation problem. [<a href="#ref-1" title=""A Border Gateway Protocol 4 (BGP-4)"">1</a>] does not specify that paths
should be ordered based on MEDs but it has been proven that non-
deterministic ordering can lead to loops and inconsistent routing
decisions. Most vendors have either implemented deterministic
ordering as default behavior, or provide a knob that permits the
operator to configure the router to order paths in a deterministic
manner based on MEDs.
<span class="grey">McPherson, et al. Informational [Page 3]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
<span class="h3"><a class="selflink" id="section-2.1" href="#section-2.1">2.1</a>. Route Reflection and Type I Churn</span>
We now discuss Type I oscillation as it relates to Route Reflection.
To begin, consider the topology depicted in Figure 1:
---------------------------------------------------------------
/ -------------------- -------------------- \
| / \ / \ |
| | Cluster 1 | | Cluster 2 | |
| | | | | |
| | | *1 | | |
| | Ra(RR) . . . . . . . . . . . . . . Rd(RR) | |
| | . . | | . | |
| | .*5 .*4 | | .*12 | |
| | . . | | . | |
| | Rb(C) Rc(C) | | Re(C) | |
| | . . | | . | |
| \ . . / \ . / |
| ---.------------.--- ---------.---------- |
\ .(10) .(1) AS1 .(0) /
-------.------------.---------------------------.--------------
. . .
------ . ------------ .
/ \ . / \ .
| AS10 | | AS6 |
\ / \ /
------ ------------
. .
. .
. --------------
. / \
| AS100 |- 10.0.0.0/8
\ /
--------------
Figure 1: Example Route Reflection Topology
In Figure 1 AS1 contains two Route Reflector Clusters, Clusters 1 and
2. Each Cluster contains one Route Reflector (RR) (i.e., Ra and Rd,
respectively). An associated 'RR' in parentheses represents each RR.
Cluster 1 contains two RR Clients (Rb and Rc), and Cluster 2 contains
one RR Client (Re). An associated 'C' in parentheses indicates RR
Client status. The dotted lines are used to represent BGP peering
sessions.
The number contained in parentheses on the AS1 EBGP peering sessions
represents the MED value advertised by the peer to be associated with
the 10.0.0.0/8 network reachability advertisement.
<span class="grey">McPherson, et al. Informational [Page 4]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
The number following each '*' on the IBGP peering sessions represents
the additive IGP metrics that are to be associated with the BGP
NEXT_HOP attribute for the concerned route. For example, the Ra IGP
metric value associated with a NEXT_HOP learned via Rb would be 5;
while the metric value associated with the NEXT_HOP learned via Re
would be 13.
Table 1 depicts the 10.0.0.0/8 route attributes as seen by routers
Rb, Rc and Re, respectively. Note that the IGP metrics in Figure 1
are only of concern when advertising the route to an IBGP peer.
Router MED AS_PATH
--------------------
Rb 10 10 100
Rc 1 6 100
Re 0 6 100
Table 1: Route Attribute Table
For the following steps 1 through 5, the best path will be marked
with a '*'.
1) Ra has the following installed in its BGP table, with the path
learned via AS2 marked best:
NEXT_HOP
AS_PATH MED IGP Cost
-----------------------
6 100 1 4
* 10 100 10 5
The '10 100' route should not be marked as best, though this is
not the cause of the persistent route oscillation. Ra realizes
it has the wrong route marked as best since the '6 100' path
has a lower IGP metric. As such, Ra makes this change and
advertises an UPDATE message to its neighbors to let them know
that it now considers the '6 100, 1, 4' route as best.
2) Rd receives the UPDATE from Ra, which leaves Rd with the
following installed in its BGP table:
NEXT_HOP
AS_PATH MED IGP Cost
-----------------------
* 6 100 0 12
6 100 1 5
<span class="grey">McPherson, et al. Informational [Page 5]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
Rd then marks the '6 100, 0, 12' route as best because it has a
lower MED. Rd sends an UPDATE message to its neighbors to let
them know that this is the best route.
3) Ra receives the UPDATE message from Rd and now has the
following in its BGP table:
NEXT_HOP
AS_PATH MED IGP Cost
-----------------------
6 100 0 13
6 100 1 4
* 10 100 10 5
The first route (6 100, 0, 13) beats the second route (6 100,
1, 4) because of a lower MED. Then the third route (10 100,
10, 5) beats the first route because of lower IGP metric to
NEXT_HOP. Ra sends an UPDATE message to its peers informing
them of the new best route.
4) Rd receives the UPDATE message from Ra, which leaves Rd with
the following BGP table:
NEXT_HOP
AS_PATH MED IGP Cost
-----------------------
6 100 0 12
* 10 100 10 6
Rd selects the '10 100, 10, 6' path as best because of the IGP
metric. Rd sends an UPDATE/withdraw to its peers letting them
know this is the best route.
5) Ra receives the UPDATE message from Rd, which leaves Ra with
the following BGP table:
NEXT_HOP
AS_PATH MED IGP Cost
-----------------------
6 100 1 4
* 10 100 10 5
Ra received an UPDATE/withdraw for '6 100, 0, 13', which
changes what is considered the best route for Ra. This is why
Ra has the '10 100, 10, 5' route selected as best in Step 1,
even though '6 100, 1, 4' is actually better.
<span class="grey">McPherson, et al. Informational [Page 6]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
At this point, we've made a full loop and are back at Step 1. The
router realizes it is using the incorrect best path, and repeats
the cycle. This is an example of Type I Churn when using Route
Reflection.
<span class="h3"><a class="selflink" id="section-2.2" href="#section-2.2">2.2</a>. AS Confederations and Type I Churn</span>
Now we provide an example of Type I Churn occurring with AS
Confederations. To begin, consider the topology depicted in Figure
2:
---------------------------------------------------------------
/ -------------------- -------------------- \
| / \ / \ |
| | Sub-AS 65000 | | Sub-AS 65001 | |
| | | | | |
| | | *1 | | |
| | Ra . . . . . . . . . . . . . . . . . Rd | |
| | . . | | . | |
| | .*3 .*2 | | .*6 | |
| | . . | | . | |
| | Rb . . . . . Rc | | Re | |
| | . *5 . | | . | |
| \ . . / \ . / |
| ---.------------.--- ---------.---------- |
\ .(10) .(1) AS1 .(0) /
-------.------------.---------------------------.--------------
. . .
------ . ------------ .
/ \ . / \ .
| AS10 | | AS6 |
\ / \ /
------ ------------
. .
. .
. --------------
. / \
| AS100 |- 10.0.0.0/8
\ /
--------------
Figure 2: Example AS Confederations Topology
The number contained in parentheses on each AS1 EBGP peering session
represents the MED value advertised by the peer to be associated with
the 10.0.0.0/8 network reachability advertisement.
<span class="grey">McPherson, et al. Informational [Page 7]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
The number following each '*' on the IBGP peering sessions represents
the additive IGP metrics that are to be associated with the BGP
NEXT_HOP attribute for the concerned route.
For example, the Ra IGP metric value associated with a NEXT_HOP
learned via Rb would be 3; while the metric value associated with the
NEXT_HOP learned via Re would be 6.
Table 2 depicts the 10.0.0.0/8 route attributes as seen by routers
Rb, Rc and Re, respectively. Note that the IGP metrics in Figure 2
are only of concern when advertising the route to an IBGP peer.
Router MED AS_PATH
--------------------
Rb 10 10 100
Rc 1 6 100
Re 0 6 100
Table 2: Route Attribute Table
For the following steps 1 through 6 the best route will be marked
with an '*'.
1) Ra has the following BGP table:
NEXT_HOP
AS_PATH MED IGP Cost
-------------------------------
* 10 100 10 3
(65001) 6 100 0 7
6 100 1 2
The '10 100' route is selected as best and is advertised to Rd,
though this is not the cause of the persistent route
oscillation.
2) Rd has the following in its BGP table:
NEXT_HOP
AS_PATH MED IGP Cost
-------------------------------
6 100 0 6
* (65000) 10 100 10 4
The '(65000) 10 100' route is selected as best because it has
the lowest IGP metric. As a result, Rd sends an
UPDATE/withdraw to Ra for the '6 100' route that it had
previously advertised.
<span class="grey">McPherson, et al. Informational [Page 8]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
3) Ra receives the withdraw from Rd. Ra now has the following in
its BGP table:
NEXT_HOP
AS_PATH MED IGP Cost
-------------------------------
* 10 100 10 3
6 100 1 2
Ra received a withdraw for '(65001) 6 100', which changes what
is considered the best route for Ra. Ra does not compute the
best path for a prefix unless its best route was withdrawn.
This is why Ra has the '10 100, 10, 3' route selected as best,
even though the '6 100, 1, 2' route is better.
4) Ra's periodic BGP scanner runs and realizes that the '6 100'
route is better because of the lower IGP metric. Ra sends an
UPDATE/withdraw to Rd for the '10 100' route since Ra is now
using the '6 100' path as its best route.
Ra's BGP table looks like this:
NEXT_HOP
AS_PATH MED IGP Cost
-------------------------------
10 100 10 3
* 6 100 1 2
5) Rd receives the UPDATE from Ra and now has the following in its
BGP table:
NEXT_HOP
AS_PATH MED IGP Cost
-------------------------------
(65000) 6 100 1 3
* 6 100 0 6
Rd selects the '6 100, 0, 6' route as best because of the lower
MED value. Rd sends an UPDATE message to Ra, reporting that '6
100, 0, 6' is now the best route.
<span class="grey">McPherson, et al. Informational [Page 9]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
6) Ra receives the UPDATE from Rd. Ra now has the following in
its BGP table:
NEXT_HOP
AS_PATH MED IGP Cost
-------------------------------
* 10 100 10 3
(65001) 6 100 0 7
6 100 1 2
At this point we have made a full cycle and are back to step 1.
This is an example of Type I Churn with AS Confederations.
<span class="h3"><a class="selflink" id="section-2.3" href="#section-2.3">2.3</a>. Potential Workarounds for Type I Churn</span>
There are a number of alternatives that can be employed to avoid this
problem:
1) When using Route Reflection make sure that the inter-Cluster
links have a higher IGP metric than the intra-Cluster links.
This is the preferred choice when using Route Reflection. Had
the inter-Cluster IGP metrics been much larger than the intra-
Cluster IGP metrics, the above would not have occurred.
2) When using AS Confederations ensure that the inter-Sub-AS links
have a higher IGP metric than the intra-Sub-AS links. This is
the preferred option when using AS Confederations. Had the
inter-Sub-AS IGP metrics been much larger than the intra-Sub-AS
IGP metrics, the above would not have occurred.
3) Do not accept MEDs from peers (this may not be a feasible
alternative).
4) Utilize other BGP attributes higher in the decision process so
that the BGP decision algorithm never reaches the MED step. As
using this completely overrides MEDs, Option 3 may make more
sense.
5) Always compare BGP MEDs, regardless of whether or not they were
obtained from a single AS. This is probably a bad idea since
MEDs may be derived in a number of ways, and are typically done
so as a matter of operator-specific policy. As such, comparing
MED values for a single prefix learned from multiple ASs is
ill-advised. Of course, this mostly defeats the purpose of
MEDs, and as such, Option 3 may be a more viable alternative.
6) Use a full IBGP mesh. This is not a feasible solution for ASs
with a large number of BGP speakers.
<span class="grey">McPherson, et al. Informational [Page 10]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
<span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Discussion of Type II Churn</span>
In the following subsection we provide configurations under which
Type II Churn will occur when using AS Confederations. For the sake
of brevity, we avoid similar discussion of the occurrence when using
Route Reflection.
In general, Type II churn occurs only when BOTH of the following
conditions are met:
1) More than one tier of Route Reflection or Sub-ASs is used in
the network AND
2) the network accepts the BGP MULTI_EXIT_DISC (MED) attribute
from two or more ASs for a single prefix and the MED values are
unique.
<span class="grey">McPherson, et al. Informational [Page 11]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
<span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. AS Confederations and Type II Churn</span>
Let's now examine the occurrence of Type II Churn as it relates to AS
Confederations. Figure 3 provides our sample topology:
---------------------------------------------------------------
/ ------------------- \
| AS 1 / Sub-AS 65500 \ |
| | | |
| | Rc . . . . Rd | |
| | . *2 . | |
| \ . . / |
| .-----------------. |
| .*40 .*40 |
| --------------.----- --.----------------- |
| / . \ / . \ |
| | Sub-AS . | | . Sub-AS | |
| | 65501 . | | . 65502 | |
| | Rb | | Re | |
| | . | | . . | |
| | .*10 | | *2. .*3 | |
| | . | | . . | |
| | Ra | | . Rg . . . Rf | |
| \ . / . . / |
| ----------.---------- . -------------.------- |
\ .(0) .(1) .() /
----------------.---------------.-------------------.----------
. . .
--------- . ---------
|AS 200 | |AS 300 |
--------- ---------
. .
. .
-------------------
| AS 400 | - 10.0.0.0/8
-------------------
Figure 3: Example AS Confederations Topology
In Figure 3 AS 1 contains three Sub-ASs, 65500, 65501 and 65502. No
RR is used within the Sub-AS, and as such, all routers within each
Sub-AS are fully meshed. Ra and Rb are members of Sub-AS 65501. Rc
and Rd are members of Sub-AS 65500. Ra and Rg are EBGP peering with
AS 200, router Rf has an EBGP peering with AS 300. AS 200 and AS 300
provide transit for AS 400, and in particular, the 10/8 network. The
dotted lines are used to represent BGP peering sessions.
<span class="grey">McPherson, et al. Informational [Page 12]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
The number following each '*' on the BGP peering sessions represents
the additive IGP metrics that are to be associated with the BGP
NEXT_HOP. The number contained in parentheses on each AS 1 EBGP
peering session represents the MED value advertised by the peer to be
associated with the network reachability advertisement (10.0.0.0/8).
Rc, Rd and Re are the primary routers involved in the churn, and as
such, will be the only BGP tables that we will monitor step by step.
For the following steps 1 through 8 each router's best route will be
marked with a '*'.
1) Re receives the AS 400 10.0.0.0/8 route advertisement via AS
200 from Rg and AS 300 from Rf. Re selects the path via Rg and
AS 200 because of IGP metric (Re didn't consider MED because
the advertisements were received from different ASs).
NEXT_HOP
Router AS_PATH MED IGP Cost
------------------------------
Re * 200 400 1 2
300 400 3
Re sends an UPDATE message to Rd advertising its new best path
'200 400, 1'.
2) The '200 400, 0' path was advertised from Ra to Rb, and then
from Rb to Rc. Rd learns the '200 400, 1' path from Re.
NEXT_HOP
Router AS_PATH MED IGP Cost
-------------------------------
Rc * 200 400 0 50
Rd * 200 400 1 42
Re 300 400 3
* 200 400 1 2
<span class="grey">McPherson, et al. Informational [Page 13]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
3) Rc and Rd advertise their best paths to each other; Rd selects
'200 400, 0' because of the MED.
NEXT_HOP
Router AS_PATH MED IGP Cost
------------------------------
Rc * 200 400 0 50
200 400 1 44
Rd * 200 400 0 52
200 400 1 42
Re 300 400 3
* 200 400 1 2
Rd has a new best path so it sends an UPDATE to to Re,
announcing the new path and an UPDATE/withdraw for '200 400, 1'
to Rc.
4) Re now selects '300 400' (with no MED) because '200 400, 0'
beats '200 400, 1' based on MED and '300 400' beats '200 400,
0' because of IGP metric.
NEXT_HOP
Router AS_PATH MED IGP Cost
------------------------------
Rc * 200 400 0 50
Rd * 200 400 0 52
200 400 1 42
Re * 300 400 3
200 400 0 92
Re has a new best path and sends an UPDATE to Rd for '300 400'.
5) Rd selects the '300 400' path because of IGP metric.
NEXT_HOP
Router AS_PATH MED IGP Cost
------------------------------
Rc * 200 400 0 50
Rd 200 400 0 52
* 300 400 43
Re * 300 400 3
200 400 0 92
200 400 1 2
Rd has a new best path so it sends an UPDATE to Rc and a
UPDATE/withdraw to Re for '200 400, 0'.
<span class="grey">McPherson, et al. Informational [Page 14]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
6) Rc selects '300 400' because of the IGP metric. Re selects
'200 400, 1' because of the IGP metric.
NEXT_HOP
Router AS_PATH MED IGP Cost
------------------------------
Rc 200 400 0 50
* 300 400 45
Rd 200 400 0 52
* 300 400 43
Re 300 400 3
* 200 400 1 2
Rc sends an UPDATE/withdraw for '200 400, 0' to Rd. Re sends
an UPDATE for '200 400, 1' to Rd.
7) Rd selects '200 400, 1' as its new best path based on the IGP
metric.
NEXT_HOP
Router AS_PATH MED IGP Cost
------------------------------
Rc 200 400 0 50
* 300 400 45
Rd * 200 400 1 42
Re 300 400 3
* 200 400 1 2
Rd sends an UPDATE to Rc, announcing '200 400, 1' and
implicitly withdraws '300 400'.
8) Rc selects '200 400, 0'.
NEXT_HOP
Router AS_PATH MED IGP Cost
------------------------------
Rc * 200 400 0 50
200 400 1 44
Rd * 200 400 1 42
Re 300 400 3
* 200 400 1 2
At this point we are back to Step 2 and are in a loop.
<span class="grey">McPherson, et al. Informational [Page 15]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
<span class="h3"><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. Potential Workarounds for Type II Churn</span>
1) Do not accept MEDs from peers (this may not be a feasible
alternative).
2) Utilize other BGP attributes higher in the decision process so
that the BGP decision algorithm selects a single AS before it
reaches the MED step. For example, if local-pref were set based
on the advertising AS, then you first eliminate all routes except
those in a single AS. In the example, router Re would pick either
X or Y based on your local-pref and never change selections.
This leaves two simple workarounds for the two types of problems.
Type I: Make inter-cluster or inter-sub-AS link metrics higher
than intra-cluster or intra-sub-AS metrics.
Type II: Make route selections based on local-pref assigned to the
advertising AS first and then use IGP cost and MED to make
selection among routes from the same AS.
Note that this requires per-prefix policies, as well as near
intimate knowledge of other networks by the network operator. The
authors are not aware of ANY [large] provider today that performs
per-prefix policies on routes learned from peers. Implicitly
removing this dynamic portion of route selection does not appear
to be a viable option in today's networks. The main point is that
an available workaround using local-pref so that no two AS's
advertise a given prefix at the same local-pref solves type II
churn.
3) Always compare BGP MEDs, regardless of whether or not they were
obtained from a single AS. This is probably a bad idea since MEDs
may be derived in a number of ways, and are typically done so as a
matter of operator-specific policy and largely a function of
available metric space provided by the employed IGP. As such,
comparing MED values for a single prefix learned from multiple ASs
is ill-advised. This mostly defeats the purpose of MEDs; Option 1
may be a more viable alternative.
4) Do not use more than one tier of Route Reflection or Sub-ASs in
the network. The risk of route oscillation should be considered
when designing networks that might use a multi-tiered routing
isolation architecture.
5) In a RR topology, mesh the clients. For confederations, mesh the
border routers at each level in the hierarchy. In Figure 3, for
example, if Rb and Re are peers, then there's no churn.
<span class="grey">McPherson, et al. Informational [Page 16]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
<span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Future Work</span>
It should be stated that protocol enhancements regarding this problem
must be pursued. Imposing network design requirements, such as those
outlined above, are clearly an unreasonable long-term solution.
Problems such as this should not occur under 'default' protocol
configurations.
<span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Security Considerations</span>
This discussion introduces no new security concerns to BGP or other
specifications referenced in this document.
<span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. Acknowledgments</span>
The authors would like to thank Curtis Villamizar, Tim Griffin, John
Scudder, Ron Da Silva, Jeffrey Haas and Bill Fenner.
<span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. References</span>
[<a id="ref-1">1</a>] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP-4)", <a href="./rfc1771">RFC</a>
<a href="./rfc1771">1771</a>, March 1995.
[<a id="ref-2">2</a>] Bates, T., Chandra, R. and E. Chen, "BGP Route Reflection - An
Alternative to Full Mesh IBGP", <a href="./rfc2796">RFC 2796</a>, April 2000.
[<a id="ref-3">3</a>] Traina, P., McPherson, D. and J. Scudder, J., "Autonomous System
Confederations for BGP", <a href="./rfc3065">RFC 3065</a>, February 2001.
[<a id="ref-4">4</a>] Rekhter, Y. and T. Li, <a style="text-decoration: none" href='https://www.google.com/search?sitesearch=datatracker.ietf.org%2Fdoc%2Fhtml%2F&q=inurl:draft-+%22A+Border+Gateway+Protocol+4+%28BGP-4%29%22'>"A Border Gateway Protocol 4 (BGP-4)"</a>,
Work in Progress.
<span class="grey">McPherson, et al. Informational [Page 17]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-18" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
<span class="h2"><a class="selflink" id="section-8" href="#section-8">8</a>. Authors' Addresses</span>
Danny McPherson
TCB
EMail: danny@tcb.net
Vijay Gill
AOL Time Warner, Inc.
12100 Sunrise Valley Drive
Reston, VA 20191
EMail: vijay@umbc.edu
Daniel Walton
Cisco Systems, Inc.
7025 Kit Creek Rd.
Research Triangle Park, NC 27709
EMail: dwalton@cisco.com
Alvaro Retana
Cisco Systems, Inc.
7025 Kit Creek Rd.
Research Triangle Park, NC 27709
EMail: aretana@cisco.com
<span class="grey">McPherson, et al. Informational [Page 18]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-19" ></span>
<span class="grey"><a href="./rfc3345">RFC 3345</a> BGP Persistent Route Oscillation Condition August 2002</span>
<span class="h2"><a class="selflink" id="section-9" href="#section-9">9</a>. Full Copyright Statement</span>
Copyright (C) The Internet Society (2002). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.
McPherson, et al. Informational [Page 19]
</pre>
|
