1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
|
<pre>Internet Engineering Task Force (IETF) C. Perkins
Request for Comments: 6051 University of Glasgow
Updates: <a href="./rfc3550">3550</a> T. Schierl
Category: Standards Track Fraunhofer HHI
ISSN: 2070-1721 November 2010
<span class="h1">Rapid Synchronisation of RTP Flows</span>
Abstract
This memo outlines how RTP sessions are synchronised, and discusses
how rapidly such synchronisation can occur. We show that most RTP
sessions can be synchronised immediately, but that the use of video
switching multipoint conference units (MCUs) or large source-specific
multicast (SSM) groups can greatly increase the synchronisation
delay. This increase in delay can be unacceptable to some
applications that use layered and/or multi-description codecs.
This memo introduces three mechanisms to reduce the synchronisation
delay for such sessions. First, it updates the RTP Control Protocol
(RTCP) timing rules to reduce the initial synchronisation delay for
SSM sessions. Second, a new feedback packet is defined for use with
the extended RTP profile for RTCP-based feedback (RTP/AVPF), allowing
video switching MCUs to rapidly request resynchronisation. Finally,
new RTP header extensions are defined to allow rapid synchronisation
of late joiners, and guarantee correct timestamp-based decoding order
recovery for layered codecs in the presence of clock skew.
Status of This Memo
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Further information on
Internet Standards is available in <a href="./rfc5741#section-2">Section 2 of RFC 5741</a>.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
<a href="http://www.rfc-editor.org/info/rfc6051">http://www.rfc-editor.org/info/rfc6051</a>.
<span class="grey">Perkins & Schierl Standards Track [Page 1]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and the IETF Trust's Legal
Provisions Relating to IETF Documents
(<a href="http://trustee.ietf.org/license-info">http://trustee.ietf.org/license-info</a>) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
<a href="#section-1">1</a>. Introduction ....................................................<a href="#page-3">3</a>
<a href="#section-2">2</a>. Synchronisation of RTP Flows ....................................<a href="#page-4">4</a>
<a href="#section-2.1">2.1</a>. Initial Synchronisation Delay ..............................<a href="#page-5">5</a>
<a href="#section-2.1.1">2.1.1</a>. Unicast Sessions ....................................<a href="#page-5">5</a>
<a href="#section-2.1.2">2.1.2</a>. Source-Specific Multicast (SSM) Sessions ............<a href="#page-6">6</a>
<a href="#section-2.1.3">2.1.3</a>. Any-Source Multicast (ASM) Sessions .................<a href="#page-7">7</a>
<a href="#section-2.1.4">2.1.4</a>. Discussion ..........................................<a href="#page-8">8</a>
<a href="#section-2.2">2.2</a>. Synchronisation for Late Joiners ...........................<a href="#page-9">9</a>
<a href="#section-3">3</a>. Reducing RTP Synchronisation Delays ............................<a href="#page-10">10</a>
<a href="#section-3.1">3.1</a>. Reduced Initial RTCP Interval for SSM Senders .............<a href="#page-10">10</a>
<a href="#section-3.2">3.2</a>. Rapid Resynchronisation Request ...........................<a href="#page-10">10</a>
<a href="#section-3.3">3.3</a>. In-Band Delivery of Synchronisation Metadata ..............<a href="#page-11">11</a>
<a href="#section-4">4</a>. Application to Decoding Order Recovery in Layered Codecs .......<a href="#page-14">14</a>
<a href="#section-4.1">4.1</a>. In-Band Synchronisation for Decoding Order Recovery .......<a href="#page-14">14</a>
<a href="#section-4.2">4.2</a>. Timestamp-Based Decoding Order Recovery ...................<a href="#page-15">15</a>
<a href="#section-4.3">4.3</a>. Example ...................................................<a href="#page-16">16</a>
<a href="#section-5">5</a>. Security Considerations ........................................<a href="#page-18">18</a>
<a href="#section-6">6</a>. IANA Considerations ............................................<a href="#page-19">19</a>
<a href="#section-7">7</a>. Acknowledgements ...............................................<a href="#page-19">19</a>
<a href="#section-8">8</a>. References .....................................................<a href="#page-20">20</a>
<a href="#section-8.1">8.1</a>. Normative References ......................................<a href="#page-20">20</a>
<a href="#section-8.2">8.2</a>. Informative References ....................................<a href="#page-20">20</a>
<span class="grey">Perkins & Schierl Standards Track [Page 2]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
<span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span>
When using RTP to deliver multimedia content it's often necessary to
synchronise playout of audio and video components of a presentation.
This is achieved using information contained in RTP Control Protocol
(RTCP) sender report (SR) packets [<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>]. These are sent
periodically, and the components of a multimedia session cannot be
synchronised until sufficient RTCP SR packets have been received for
each RTP flow to allow the receiver to establish mappings between the
media clock used for each RTP flow, and the common (NTP-format)
reference clock used to establish synchronisation.
Recently, concern has been expressed that this synchronisation delay
is problematic for some applications, for example those using layered
or multi-description video coding. This memo reviews the operations
of RTP synchronisation, and describes the synchronisation delay that
can be expected. Three backwards compatible extensions to the basic
RTP synchronisation mechanism are proposed:
o The RTCP transmission timing rules are relaxed for source-specific
multicast (SSM) senders, to reduce the initial synchronisation
latency for large SSM groups. See <a href="#section-3.1">Section 3.1</a>.
o An enhancement to the extended RTP profile for RTCP-based feedback
(RTP/AVPF) [<a href="./rfc4585" title=""Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)"">RFC4585</a>] is defined to allow receivers to request
additional RTCP SR packets, providing the metadata needed to
synchronise RTP flows. This can reduce the synchronisation delay
when joining sessions with large RTCP reporting intervals, in the
presence of packet loss, or when video switching MCUs are
employed. See <a href="#section-3.2">Section 3.2</a>.
o Two RTP header extensions are defined, to deliver synchronisation
metadata in-band with RTP data packets. These extensions provide
synchronisation metadata that is aligned with RTP data packets,
and so eliminate the need to estimate clock skew between flows
before synchronisation. They can also reduce the need to receive
RTCP SR packets before flows can be synchronised, although it does
not eliminate the need for RTCP. See <a href="#section-3.3">Section 3.3</a>.
The immediate use-case for these extensions is to reduce the delay
due to synchronisation when joining a layered video session (e.g., an
H.264/SVC (Scalable Video Coding) session in Non-Interleaved
Timestamp-based (NI-T) mode [<a href="#ref-AVT-RTP-SVC">AVT-RTP-SVC</a>]). The extensions are not
specific to layered coding, however, and can be used in any
environment when synchronisation latency is an issue.
<span class="grey">Perkins & Schierl Standards Track [Page 3]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in <a href="./rfc2119">RFC 2119</a> [<a href="./rfc2119" title=""Key words for use in RFCs to Indicate Requirement Levels"">RFC2119</a>].
<span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Synchronisation of RTP Flows</span>
RTP flows are synchronised by receivers based on information that is
contained in RTCP SR packets generated by senders (specifically, the
NTP-format timestamp and the RTP timestamp). Synchronisation
requires that a common reference clock MUST be used to generate the
NTP-format timestamps in a set of flows that are to be synchronised
(i.e., when synchronising several RTP flows, the RTP timestamps for
each flow are derived from separate, and media specific, clocks, but
the NTP-format timestamps in the RTCP SR packets of all flows to be
synchronised MUST be sampled from the same clock). To achieve faster
and more accurate synchronisation, it is further RECOMMENDED that
senders and receivers use a synchronised common NTP-format reference
clock with common properties, especially timebase, where possible
(recognising that this is often not possible when RTP is used outside
of controlled environments); the means by which that common reference
clock and its properties are signalled and distributed is outside the
scope of this memo.
For multimedia sessions, each type of media (e.g., audio or video) is
sent in a separate RTP session, and the receiver associates RTP flows
to be synchronised by means of the canonical end-point identifier
(CNAME) item included in the RTCP Source Description (SDES) packets
generated by the sender or signalled out of band [<a href="./rfc5576" title=""Source-Specific Media Attributes in the Session Description Protocol (SDP)"">RFC5576</a>]. For
layered media, different layers can be sent in different RTP
sessions, or using different synchronisation source (SSRC) values
within a single RTP session; in both cases, the CNAME is used to
identify flows to be synchronised. To ensure synchronisation, an RTP
sender MUST therefore send periodic compound RTCP packets following
<a href="./rfc3550#section-6">Section 6 of RFC 3550</a> [<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>].
The timing of these periodic compound RTCP packets will depend on the
number of members in each RTP session, the fraction of those that are
sending data, the session bandwidth, the configured RTCP bandwidth
fraction, and whether the session is multicast or unicast (see
<a href="./rfc3550#section-6.2">RFC 3550, Section 6.2</a> for details). In summary, RTCP control traffic
is allocated a small fraction, generally 5%, of the session
bandwidth, and of that fraction, one quarter is allocated to active
RTP senders, while receivers use the remaining three quarters (these
fractions can be configured via the Session Description Protocol
(SDP) [<a href="./rfc3556" title=""Session Description Protocol (SDP) Bandwidth Modifiers for RTP Control Protocol (RTCP) Bandwidth"">RFC3556</a>]). Each member of an RTP session derives an RTCP
reporting interval based on these fractions, whether the session is
multicast or unicast, the number of members it has observed, and
whether it is actively sending data or not. It then sends a compound
<span class="grey">Perkins & Schierl Standards Track [Page 4]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
RTCP packet on average once per reporting interval (the actual packet
transmission time is randomised in the range [0.5 ... 1.5] times the
reporting interval to avoid synchronisation of reports).
A minimum reporting interval of 5 seconds is RECOMMENDED, except that
the delay before sending the initial report "MAY be set to half the
minimum interval to allow quicker notification that the new
participant is present" [<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>]. Also, for unicast sessions, "the
delay before sending the initial compound RTCP packet MAY be zero"
[<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>]. In addition, for unicast sessions, and for active senders
in a multicast session, the fixed minimum reporting interval MAY be
scaled to "360 divided by the session bandwidth in kilobits/second.
This minimum is smaller than 5 seconds for bandwidths greater than
72 kb/s" [<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>].
<span class="h3"><a class="selflink" id="section-2.1" href="#section-2.1">2.1</a>. Initial Synchronisation Delay</span>
A multimedia session comprises a set of concurrent RTP sessions among
a common group of participants, using one RTP session for each media
type. For example, a videoconference (which is a multimedia session)
might contain an audio RTP session and a video RTP session. To allow
a receiver to synchronise the components of a multimedia session, a
compound RTCP packet containing an RTCP SR packet and an RTCP SDES
packet with a CNAME item MUST be sent to each of the RTP sessions in
the multimedia session by each sender. A receiver cannot synchronise
playout across the multimedia session until such RTCP packets have
been received on all of the component RTP sessions. If there is no
packet loss, this gives an expected initial synchronisation delay
equal to the average time taken to receive the first RTCP packet in
the RTP session with the longest RTCP reporting interval. This will
vary between unicast and multicast RTP sessions.
The initial synchronisation delay for layered sessions is similar to
that for multimedia sessions. The layers cannot be synchronised
until the RTCP SR and CNAME information has been received for each
layer in the session.
<span class="h4"><a class="selflink" id="section-2.1.1" href="#section-2.1.1">2.1.1</a>. Unicast Sessions</span>
For unicast multimedia or layered sessions, senders SHOULD transmit
an initial compound RTCP packet (containing an RTCP SR packet and an
RTCP SDES packet with a CNAME item) immediately on joining each RTP
session in the multimedia session. The individual RTP sessions are
considered to be joined once any in-band signalling for NAT traversal
<span class="grey">Perkins & Schierl Standards Track [Page 5]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
(e.g., [<a href="./rfc5245" title=""Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal for Offer/Answer Protocols"">RFC5245</a>]) and/or security keying (e.g., [<a href="./rfc5764" title=""Datagram Transport Layer Security (DTLS) Extension to Establish Keys for the Secure Real-time Transport Protocol (SRTP)"">RFC5764</a>], [<a href="#ref-ZRTP" title=""ZRTP: Media Path Key Agreement for Unicast Secure RTP"">ZRTP</a>])
has concluded, and the media path is open. This implies that the
initial RTCP packet is sent in parallel with the first data packet
following the guidance in <a href="./rfc3550">RFC 3550</a> that "the delay before sending the
initial compound RTCP packet MAY be zero" and, in the absence of any
packet loss, flows can be synchronised immediately.
It is expected that NAT pinholes, firewall holes, quality-of-service,
and media security keys will have been negotiated as part of the
signalling, whether in-band or out-of-band, before the first RTCP
packet is sent. This should ensure that any middleboxes are ready to
accept traffic, and reduce the likelihood that the initial RTCP
packet will be lost.
<span class="h4"><a class="selflink" id="section-2.1.2" href="#section-2.1.2">2.1.2</a>. Source-Specific Multicast (SSM) Sessions</span>
For multicast sessions, the delay before sending the initial RTCP
packet, and hence the synchronisation delay, varies with the session
bandwidth and the number of members in the session. For a multicast
multimedia or layered session, the average synchronisation delay will
depend on the slowest of the component RTP sessions; this will
generally be the session with the lowest bandwidth (assuming all the
RTP sessions have the same number of members).
When sending to a multicast group, the reduced minimum RTCP reporting
interval of 360 seconds divided by the session bandwidth in kilobits
per second [<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>] should be used when synchronisation latency is
likely to be an issue. Also, as usual, the reporting interval is
halved for the first RTCP packet. Depending on the session bandwidth
and the number of members, this gives the average synchronisation
delays shown in Figure 1.
Session| Number of receivers:
Bandwidth| 2 3 4 5 10 100 1000 10000
--+------------------------------------------------
8 kbps| 2.73 4.10 5.47 5.47 5.47 5.47 5.47 5.47
16 kbps| 2.50 2.50 2.73 2.73 2.73 2.73 2.73 2.73
32 kbps| 2.50 2.50 2.50 2.50 2.50 2.50 2.50 2.50
64 kbps| 2.50 2.50 2.50 2.50 2.50 2.50 2.50 2.50
128 kbps| 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41
256 kbps| 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70
512 kbps| 0.35 0.35 0.35 0.35 0.35 0.35 0.35 0.35
1 Mbps| 0.18 0.18 0.18 0.18 0.18 0.18 0.18 0.18
2 Mbps| 0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.09
4 Mbps| 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04
Figure 1: Average Initial Synchronisation Delay in Seconds
for an RTP Session with 1 Sender
<span class="grey">Perkins & Schierl Standards Track [Page 6]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
These numbers assume a source-specific multicast channel with a
single active sender, assuming an average RTCP packet size of
70 octets. These intervals are sufficient for lip-synchronisation
without excessive delay, but might be viewed as having too much
latency for synchronising parts of a layered video stream.
The RTCP interval is randomised in the usual manner, so the minimum
synchronisation delay will be half these intervals, and the maximum
delay will be 1.5 times these intervals. Note also that these RTCP
intervals are calculated assuming perfect knowledge of the number of
members in the session.
<span class="h4"><a class="selflink" id="section-2.1.3" href="#section-2.1.3">2.1.3</a>. Any-Source Multicast (ASM) Sessions</span>
For ASM sessions, the fraction of members that are senders plays an
important role, and causes more variation in average RTCP reporting
interval. This is illustrated in Figure 2 and Figure 3, which show
the RTCP reporting interval for the same session bandwidths and
receiver populations as the SSM session described in Figure 1, but
for sessions with 2 and 10 senders, respectively. It can be seen
that the initial synchronisation delay scales with the number of
senders (this is to ensure that the total RTCP traffic from all group
members does not grow without bound) and can be significantly larger
than for source-specific groups. Despite this, the initial
synchronisation time remains acceptable for lip-synchronisation in
typical small-to-medium sized group video conferencing scenarios.
Note that multi-sender groups implemented using multi-unicast with a
central RTP translator (Topo-Translator in the terminology of
[<a href="./rfc5117" title=""RTP Topologies"">RFC5117</a>]) or mixer (Topo-Mixer), or some forms of video switching
MCU (Topo-Video-switch-MCU) distribute RTCP packets to all members of
the group, and so scale in the same way as an ASM group with regards
to initial synchronisation latency.
<span class="grey">Perkins & Schierl Standards Track [Page 7]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
Session| Number of receivers:
Bandwidth| 2 3 4 5 10 100 1000 10000
--+------------------------------------------------
8 kbps| 2.73 4.10 5.47 6.84 10.94 10.94 10.94 10.94
16 kbps| 2.50 2.50 2.73 3.42 5.47 5.47 5.47 5.47
32 kbps| 2.50 2.50 2.50 2.50 2.73 2.73 2.73 2.73
64 kbps| 2.50 2.50 2.50 2.50 2.50 2.50 2.50 2.50
128 kbps| 1.41 1.41 1.41 1.41 1.41 1.41 1.41 1.41
256 kbps| 0.70 0.70 0.70 0.70 0.70 0.70 0.70 0.70
512 kbps| 0.35 0.35 0.35 0.35 0.35 0.35 0.35 0.35
1 Mbps| 0.18 0.18 0.18 0.18 0.18 0.18 0.18 0.18
2 Mbps| 0.09 0.09 0.09 0.09 0.09 0.09 0.09 0.09
4 Mbps| 0.04 0.04 0.04 0.04 0.04 0.04 0.04 0.04
Figure 2: Average Initial Synchronisation Delay in Seconds
for an RTP Session with 2 Senders
Session| Number of receivers:
Bandwidth| 2 3 4 5 10 100 1000 10000
--+------------------------------------------------
8 kbps| 2.73 4.10 5.47 6.84 13.67 54.69 54.69 54.69
16 kbps| 2.50 2.50 2.73 3.42 6.84 27.34 27.34 27.34
32 kbps| 2.50 2.50 2.50 2.50 3.42 13.67 13.67 13.67
64 kbps| 2.50 2.50 2.50 2.50 2.50 6.84 6.84 6.84
128 kbps| 1.41 1.41 1.41 1.41 1.41 3.42 3.42 3.42
256 kbps| 0.70 0.70 0.70 0.70 0.70 1.71 1.71 1.71
512 kbps| 0.35 0.35 0.35 0.35 0.35 0.85 0.85 0.85
1 Mbps| 0.18 0.18 0.18 0.18 0.18 0.43 0.43 0.43
2 Mbps| 0.09 0.09 0.09 0.09 0.09 0.21 0.21 0.21
4 Mbps| 0.04 0.04 0.04 0.04 0.04 0.11 0.11 0.11
Figure 3: Average Initial Synchronisation Delay in Seconds
for an RTP Session with 10 Senders
<span class="h4"><a class="selflink" id="section-2.1.4" href="#section-2.1.4">2.1.4</a>. Discussion</span>
For unicast sessions, the existing RTCP SR-based mechanism allows for
immediate synchronisation, provided the initial RTCP packet is not
lost.
For SSM sessions, the initial synchronisation delay is sufficient for
lip-synchronisation, but may be larger than desired for some layered
codecs. The rationale for not sending immediate RTCP packets for
multicast groups is to avoid implosion of requests when large numbers
of members simultaneously join the group ("flash crowd"). This is
not an issue for SSM senders, since there can be at most one sender,
so it is desirable to allow SSM senders to send an immediate RTCP SR
<span class="grey">Perkins & Schierl Standards Track [Page 8]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
on joining a session (as is currently allowed for unicast sessions,
which also don't suffer from the implosion problem). SSM receivers
using unicast feedback would not be allowed to send immediate RTCP.
For ASM sessions, implosion of responses is a concern, so no change
is proposed to the RTCP timing rules.
In all cases, it is possible that the initial RTCP SR packet is lost.
In this case, the receiver will not be able to synchronise the media
until the reporting interval has passed, and the next RTCP SR packet
is sent. This is undesirable. <a href="#section-3.2">Section 3.2</a> defines a new RTP/AVPF
transport layer feedback message to request that an RTCP SR be
generated, allowing rapid resynchronisation in the case of packet
loss.
<span class="h3"><a class="selflink" id="section-2.2" href="#section-2.2">2.2</a>. Synchronisation for Late Joiners</span>
Synchronisation between RTP sessions is potentially slower for late
joiners than for participants present at the start of the session.
The reasons for this are three-fold:
1. Many of the optimisations that allow rapid transmission of RTCP SR
packets apply only at the start of a session. This implies that a
new participant may have to wait a complete RTCP reporting
interval for each session before receiving the necessary data to
synchronise media streams. This might potentially take several
seconds, depending on the configured session bandwidth and the
number of participants.
2. Additional synchronisation delay comes from the nature of the RTCP
timing rules. Packets are generated on average once per reporting
interval, but with the exact transmission times being randomised
+/- 50% to avoid synchronisation of reports. This is important to
avoid network congestion in multicast sessions, but does mean that
the timing of RTCP sender reports for different RTP sessions isn't
synchronised. Accordingly, a receiver must estimate the skew on
the NTP-format clock in order to align RTP timestamps across
sessions. This estimation is an essential part of an RTP
synchronisation implementation, and can be done with high accuracy
given sufficient reports. Collecting sufficient RTCP SR data to
perform this estimation, however, may require reception of several
RTCP reports, further increasing the synchronisation delay.
3. Many media codecs have the notion of periodic access points, such
that a newly joined receiver often cannot start decoding a media
stream until the packets corresponding to the access point have
been received. These access points may be sent less often than
RTCP SR packets, and so may be the limiting factor in starting
synchronised media playout for late joiners. The RTP extension
<span class="grey">Perkins & Schierl Standards Track [Page 9]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
for unicast-based rapid acquisition of multicast RTP sessions
[<a href="#ref-AVT-ACQUISITION-RTP">AVT-ACQUISITION-RTP</a>] may be used to reduce the time taken to
receive the access points in some scenarios.
These delays are likely an issue for tuning in to an ongoing
multicast RTP session, or for video switching MCUs.
<span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Reducing RTP Synchronisation Delays</span>
Three backwards compatible RTP extensions are defined to reduce the
possible synchronisation delay: a reduced initial RTCP interval for
SSM senders, a rapid resynchronisation request message, and RTP
header extensions that can convey synchronisation metadata in-band.
<span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>. Reduced Initial RTCP Interval for SSM Senders</span>
In SSM sessions where the initial synchronisation delay is important,
the RTP sender MAY set the delay before sending the initial compound
RTCP packet to zero, and send its first RTCP packet immediately upon
joining the SSM session. This is purely a local change to the sender
that can be implemented as a configurable option. RTP receivers in
an SSM session, sending unicast RTCP feedback, MUST NOT send RTCP
packets with zero initial delay; the timing rules defined in
[<a href="./rfc5760" title=""RTP Control Protocol (RTCP) Extensions for Single-Source Multicast Sessions with Unicast Feedback"">RFC5760</a>] apply unchanged to receivers.
<span class="h3"><a class="selflink" id="section-3.2" href="#section-3.2">3.2</a>. Rapid Resynchronisation Request</span>
The general format of an RTP/AVPF transport layer feedback message is
shown in Figure 4 (see [<a href="./rfc4585" title=""Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)"">RFC4585</a>] for details).
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P| FMT | PT=RTPFB=205 | length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of packet sender |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of media source |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: Feedback Control Information (FCI) :
: :
Figure 4: RTP/AVPF Transport Layer Feedback Message
<span class="grey">Perkins & Schierl Standards Track [Page 10]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
One new feedback message type, RTCP-SR-REQ, is defined with FMT = 5.
The Feedback Control Information (FCI) part of the feedback message
MUST be empty. The SSRC of the packet sender indicates the member
that is unable to synchronise media streams, while the SSRC of the
media source indicates the sender of the media it is unable to
synchronise. The length MUST equal 2.
If the RTP/AVPF profile [<a href="./rfc4585" title=""Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)"">RFC4585</a>] is in use, this feedback message
MAY be sent by a receiver to indicate that it's unable to synchronise
some media streams, and desires that the media source transmit an
RTCP SR packet as soon as possible (within the constraints of the
RTCP timing rules for early feedback). When it receives such an
indication, a media source that understands the RTCP-SR-REQ packet
SHOULD generate an RTCP SR packet as soon as possible while complying
with the RTCP early feedback rules. If the use of non-compound RTCP
[<a href="./rfc5506" title=""Support for Reduced-Size Real-Time Transport Control Protocol (RTCP): Opportunities and Consequences"">RFC5506</a>] was previously negotiated, both the feedback request and
the RTCP SR response may be sent as non-compound RTCP packets. The
RTCP-SR-REQ packet MAY be repeated once per RTCP reporting interval
if no RTCP SR packet is forthcoming. The media source may ignore
RTCP-SR-REQ packets if its regular schedule for transmission of
synchronisation metadata can be expected to allow the receiver to
synchronise the media streams within a reasonable time frame.
When using SSM sessions with unicast feedback, it is possible that
the feedback target and media source are not co-located. If a
feedback target receives an RTCP-SR-REQ feedback message in such a
case, the request should be forwarded to the media source. The
mechanism to be used for forwarding such requests is not defined
here.
If the feedback target provides a network management interface, it
might be useful to provide a log of which receivers send RTCP-SR-REQ
feedback packets and which do not, since those that do not will see
slower stream synchronisation.
<span class="h3"><a class="selflink" id="section-3.3" href="#section-3.3">3.3</a>. In-Band Delivery of Synchronisation Metadata</span>
The RTP header extension mechanism defined in [<a href="./rfc5285" title=""A General Mechanism for RTP Header Extensions"">RFC5285</a>] can be
adapted to carry an OPTIONAL NTP-format timestamp in RTP data
packets. If such a timestamp is included, it MUST correspond to the
same time instant as the RTP timestamp in the packet's header, and
MUST be derived from the same clock used to generate the NTP-format
timestamps included in RTCP SR packets. Provided it has knowledge of
the SSRC to CNAME mapping, either from prior receipt of an RTCP CNAME
packet or via out-of-band signalling [<a href="./rfc5576" title=""Source-Specific Media Attributes in the Session Description Protocol (SDP)"">RFC5576</a>], the receiver can use
the information provided as input to the synchronisation algorithm,
in exactly the same way as if an additional RTCP SR packet had been
received for the flow.
<span class="grey">Perkins & Schierl Standards Track [Page 11]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
Two variants are defined for this header extension. The first
variant extends the RTP header with a 64-bit NTP-format timestamp as
defined in [<a href="./rfc5905" title=""Network Time Protocol Version 4: Protocol and Algorithms Specification"">RFC5905</a>]. The second variant carries the lower 24-bit
part of the Seconds of a NTP-format timestamp and the 32 bits of the
Fraction of a NTP-format timestamp. The formats of the two variants
are shown in Figure 5 and Figure 6.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|1| CC |M| PT | sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+R
| timestamp |T
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+P
| synchronisation source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| 0xBE | 0xDE | length=3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+E
| ID-A | L=7 | NTP timestamp format - Seconds (bit 0-23) |x
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+t
|NTP Sec.(24-31)| NTP timestamp format - Fraction (bit 0-23) |n
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|NTP Frc.(24-31)| 0 (pad) | 0 (pad) | 0 (pad) |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| payload data |
| .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: Variant A/64-Bit NTP RTP Header Extension
<span class="grey">Perkins & Schierl Standards Track [Page 12]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|1| CC |M| PT | sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+R
| timestamp |T
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+P
| synchronisation source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| 0xBE | 0xDE | length=2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+E
| ID-B | L=6 | NTP timestamp format - Seconds (bit 8-31) |x
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+t
| NTP timestamp format - Fraction (bit 0-31) |n
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| payload data |
| .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: Variant B/56-Bit NTP RTP Header Extension
An NTP-format timestamp MAY be included in any RTP packets the sender
chooses, but it is RECOMMENDED when performing timestamp-based
decoding order recovery for layered codecs transported in multiple
RTP flows, as further specified in <a href="#section-4.1">Section 4.1</a>. This header
extension SHOULD be also sent in the RTP packets corresponding to a
video random access point, and in the associated audio packets, to
allow rapid synchronisation for late joiners in multimedia sessions,
and in video switching scenarios.
Note: The inclusion of an RTP header extension will reduce the
efficiency of RTP header compression, if it is used. Furthermore,
middleboxes that do not understand the header extensions may
remove them or may not update the content according to this memo.
In all cases, irrespective of whether in-band NTP-format timestamps
are included or not, regular RTCP SR packets MUST be sent to provide
backwards compatibility with receivers that synchronise RTP flows
according to [<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>], and robustness in the face of middleboxes
(RTP translators) that might strip RTP header extensions. If the
Variant B/56-bit NTP RTP header extension is used, RTCP sender
reports MUST be used to derive the upper 8 bits of the Seconds for
the NTP-format timestamp.
When SDP is used, the use of the RTP header extensions defined above
MUST be indicated as specified in [<a href="./rfc5285" title=""A General Mechanism for RTP Header Extensions"">RFC5285</a>]. Therefore, the
following URIs MUST be used:
<span class="grey">Perkins & Schierl Standards Track [Page 13]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
o The URI used for signalling the use of Variant A/64-bit NTP RTP
header extension in SDP is "urn:ietf:params:rtp-hdrext:ntp-64".
o The URI used for signalling the use of Variant B/56-bit NTP RTP
header extension in SDP is "urn:ietf:params:rtp-hdrext:ntp-56".
The use of these RTP header extensions can greatly improve the user
experience in IPTV channel surfing and in some interactive video
conferencing scenarios. Network management tools that attempt to
monitor the user experience may wish to log which sessions signal and
use these extensions.
<span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Application to Decoding Order Recovery in Layered Codecs</span>
Packets in RTP flows are often predictively coded, with a receiver
having to arrange the packets into a particular order before it can
decode the media data. Depending on the payload format, the decoding
order might be explicitly specified as a field in the RTP payload
header, or the receiver might decode the packets in order of their
RTP timestamps. If a layered encoding is used, where the media data
is split across several RTP flows, then it is often necessary to
exactly synchronise the RTP flows comprising the different layers
before layers other than the base layer can be decoded. Examples of
such layered encodings are H.264 SVC in NI-T mode [<a href="#ref-AVT-RTP-SVC">AVT-RTP-SVC</a>] and
MPEG surround multi-channel audio [<a href="./rfc5691" title=""RTP Payload Format for Elementary Streams with MPEG Surround Multi-Channel Audio"">RFC5691</a>]. As described in
<a href="#section-2">Section 2</a>, such synchronisation is possible in RTP, but can be
difficult to perform rapidly. Below, we describe how the extensions
defined in <a href="#section-3.3">Section 3.3</a> can be used to synchronise layered flows, and
provide a common timestamp-based decoding order.
<span class="h3"><a class="selflink" id="section-4.1" href="#section-4.1">4.1</a>. In-Band Synchronisation for Decoding Order Recovery</span>
When a layered, multi-description, or multi-view codec is used, with
the different components of the media being transferred on separate
RTP flows, the RTP sender SHOULD use periodic synchronous in-band
delivery of synchronisation metadata to allow receivers to rapidly
and accurately synchronise the separate components of the layered
media flow. There are three parts to this:
o The sender must negotiate the use of the RTP header extensions
described in <a href="#section-3.3">Section 3.3</a>, and must periodically and synchronously
insert such header extensions into all the RTP flows forming the
separate components of the layered, multi-description, or multi-
view flow.
o Synchronous insertion requires that the sender insert these RTP
header extensions into packets corresponding to exactly the same
sampling instant in all the flows. Since the header extensions
<span class="grey">Perkins & Schierl Standards Track [Page 14]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
for each flow are inserted at exactly the same sampling instant,
they will have identical NTP-format timestamps, hence allowing
receivers to exactly align the RTP timestamps for the component
flows. This may require the insertion of extra data packets into
some of the component RTP flows, if some component flows contain
packets for sampling instants that do not exist in other flows
(for example, a layered video codec, where the layers have
differing frame rates).
o The frequency with which the sender inserts the header extensions
will directly correspond to the synchronisation latency, with more
frequent insertion leading to higher per-flow overheads, but lower
synchronisation latency. It is RECOMMENDED that the sender insert
the header extensions synchronously into all component RTP flows
at least once per random access point of the media, but they MAY
be inserted more often.
The sender MUST continue to send periodic RTCP reports including SR
packets, and MUST ensure the RTP timestamp to NTP-format timestamp
mapping in the RTCP SR packets is consistent with that used in the
RTP header extensions. Receivers should use both the information
contained in RTCP SR packets and the in-band mapping of RTP and NTP-
format timestamps as input to the synchronisation process, but it is
RECOMMENDED that receivers sanity check the mappings received and
discard outliers, to provide robustness against invalid data (one
might think it more likely that the RTCP SR mappings are invalid,
since they are sent at irregular times and subject to skew, but the
presence of broken RTP translators could also corrupt the timestamps
in the RTP header extension; receivers need to cope with both types
of failure).
<span class="h3"><a class="selflink" id="section-4.2" href="#section-4.2">4.2</a>. Timestamp-Based Decoding Order Recovery</span>
Once a receiver has synchronised the components of a layered, multi-
description, or multi-view flow using the RTP header extensions as
described in <a href="#section-4.1">Section 4.1</a>, it may then derive a decoding order based
on the synchronised timestamps as follows (or it may use information
in the RTP payload header to derive the decoding order, if present
and desired).
There may be explicit dependencies between the component flows of a
layered, multi-description, or multi-view flow. For example, it is
common for layered flows to be arranged in a hierarchy, where flows
from "higher" layers cannot be decoded until the corresponding data
in "lower" layer flows has been received and decoded. If such a
decoding hierarchy exists, it MUST be signalled out of band, for
example using [<a href="./rfc5583" title=""Signaling Media Decoding Dependency in the Session Description Protocol (SDP)"">RFC5583</a>] when SDP signalling is used.
<span class="grey">Perkins & Schierl Standards Track [Page 15]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
Each component RTP flow MUST contain packets corresponding to all the
sampling instants of the RTP flows on which it depends. If such
packets are not naturally present in the RTP flow, the sender MUST
generate additional packets as necessary in order to satisfy this
rule. The format of these packets depends on the payload format
used. For H.264 SVC, the Empty Network Abstraction Layer (NAL) unit
packet [<a href="#ref-AVT-RTP-SVC">AVT-RTP-SVC</a>] should be used. Flows may also include packets
corresponding to additional sampling instants that are not present in
the flows on which they depend.
The receiver should decode the packets in all the component RTP flows
as follows:
o For each RTP packet in each flow, use the mapping contained in the
RTP header extensions and RTCP SR packets to derive the NTP-format
timestamp corresponding to its RTP timestamp.
o Group together RTP data packets from all component flows that have
identical calculated NTP-format timestamps.
o Processing groups in order of ascending NTP-format timestamps,
decode the RTP packets in each group according to the signalled
RTP flow decoding hierarchy. That is, pass the RTP packet data
from the flow on which all other flows depend to the decoder
first, then that from the next dependent flow, and so on. The
decoding order of the RTP flow hierarchy may be indicated by
mechanisms defined in [<a href="./rfc5583" title=""Signaling Media Decoding Dependency in the Session Description Protocol (SDP)"">RFC5583</a>] or by some other means.
Note that the decoding order will not necessarily match the packet
transmission order. The receiver will need to buffer packets for a
codec-dependent amount of time in order for all necessary packets to
arrive to allow decoding.
<span class="h3"><a class="selflink" id="section-4.3" href="#section-4.3">4.3</a>. Example</span>
The example shown in Figure 7 refers to three RTP flows A, B, and C,
containing a layered, a multi-view, or a multi-description media
stream. In the example, the dependency signalling as defined in
[<a href="./rfc5583" title=""Signaling Media Decoding Dependency in the Session Description Protocol (SDP)"">RFC5583</a>] indicates that flow A is the lowest RTP flow. Flow B is
the next higher RTP flow and depends on A. Flow C is the highest of
the three RTP flows and depends on both A and B. A media coding
structure is used that results in video access units (i.e., coded
video frames) present in higher flows but not present in all lower
flows. Flow A has the lowest frame rate. Flows B and C have the
same frame rate, which is higher than that of Flow A. The figure
shows the full video access units with their corresponding RTP
timestamps "(x)". The video access units are already re-ordered
according to their RTP sequence number order. The figure indicates
<span class="grey">Perkins & Schierl Standards Track [Page 16]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
the received video access unit part in decoding order within each RTP
flow, as well as the associated NTP media timestamps ("TS[..]"). As
shown in the figure, these timestamps may be derived using the
NTP-format timestamp provided in the RTCP sender reports as indicated
by the timestamp in "{x}", or derived directly from the NTP timestamp
contained in the RTP header extensions as indicated by the timestamp
in "<x>". Note that the timestamps are not in increasing order
since, in this example, the decoding order is different from the
output/presentation order.
The decoding order recovery process first advances to the video
access unit parts associated with the first available synchronous
insertion of the NTP timestamp into RTP header extensions at NTP
media timestamp TS=[8]. The receiver starts in the highest RTP
flow C and removes/ignores all preceding video access unit parts (in
decoding order) to video access unit parts with TS=[8] in each of the
de-jittering buffers of RTP flows A, B, and C. Then, starting from
flow C, the first media timestamp available in decoding order
(TS=[8]) is selected, and video access unit parts starting from RTP
flow A, and flows B and C are placed in order of the RTP flow
dependency as indicated by mechanisms defined in [<a href="./rfc5583" title=""Signaling Media Decoding Dependency in the Session Description Protocol (SDP)"">RFC5583</a>] (in the
example for TS[8]: first flow B and then flow C into the video access
unit AU(TS[8]) associated with NTP media timestamp TS=[8]). Then the
next media timestamp TS=[6] (RTP timestamp=(4)) in order of
appearance in the highest RTP flow C is processed, and the process
described above is repeated. Note that there may be video access
units with no video access unit parts present, e.g., in the lowest
RTP flow A (see, e.g., TS=[5]). The decoding order recovery process
could also be started after an RTP sender report containing the
mapping between the RTP timestamp and the NTP-format timestamp
(indicated as timestamps "(x){y}") has been received, assuming that
there is no clock skew in the source used for the NTP-format
timestamp generation.
<span class="grey">Perkins & Schierl Standards Track [Page 17]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-18" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
C:-(0)----(2)----(7)<8>--(5)----(4)----(6)-----(11)----(9){10}-
| | | | | | | |
B:-(3)----(5)---(10)<8>--(8)----(7)----(9){7}--(14)----(12)----
| | | |
A:---------------(3)<8>--(1)-------------------(7){12}-(5)-----
---------------------------------------decoding/transmission order->
TS:[1] [3] [8]=<8> [6] [5] [7] [12] [10]
Key:
A, B, C - RTP flows
Integer values in "()" - video access unit with its RTP timestamp as
indicated in its RTP packet.
"|" - indicates the corresponding parts of the
same video access unit AU(TS[..]) in the
RTP flows.
Integer values in "[]" - NTP media timestamp TS, sampling time
as derived from the NTP timestamp
associated with the video access unit
AU(TS[..]), consisting of video access unit
parts in the flows above.
Integer values in "<>" - NTP media timestamp TS as directly
taken from the NTP RTP header extensions.
Integer values in "{}" - NTP media timestamp TS as provided in the
RTCP sender reports.
Figure 7: Example of a Layered RTP Stream
<span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Security Considerations</span>
The security considerations of the RTP specification [<a href="./rfc3550" title=""RTP: A Transport Protocol for Real-Time Applications"">RFC3550</a>], the
extended RTP profile for RTCP-based feedback [<a href="./rfc4585" title=""Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)"">RFC4585</a>], and the
general mechanism for RTP header extensions [<a href="./rfc5285" title=""A General Mechanism for RTP Header Extensions"">RFC5285</a>] apply.
The RTP header extensions defined in <a href="#section-3.3">Section 3.3</a> include an NTP-
format timestamp. When an RTP session using this header extension is
protected by the Secure RTP (SRTP) framework [<a href="./rfc3711" title=""The Secure Real-time Transport Protocol (SRTP)"">RFC3711</a>], that header
extension is not part of the encrypted portion of the RTP data
packets or RTCP control packets; however, these NTP-format timestamps
are encrypted when using SRTP without this header extension. This is
a minor information leak, but one that is not believed to be
<span class="grey">Perkins & Schierl Standards Track [Page 18]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-19" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
significant. The inclusion of this header extension will also reduce
the efficiency of RTP header compression, if it is used.
Furthermore, middleboxes that do not understand the header extensions
may remove them or may not update the content according to this memo.
<span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. IANA Considerations</span>
The IANA has registered one new value in the table of FMT Values for
RTPFB Payload Types [<a href="./rfc4585" title=""Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)"">RFC4585</a>] as follows:
Name: RTCP-SR-REQ
Long name: RTCP Rapid Resynchronisation Request
Value: 5
Reference: <a href="./rfc6051">RFC 6051</a>
The IANA has also registered two new RTP Compact Header Extensions
[<a href="./rfc5285" title=""A General Mechanism for RTP Header Extensions"">RFC5285</a>], according to the following:
Extension URI: urn:ietf:params:rtp-hdrext:ntp-64
Description: Synchronisation metadata: 64-bit timestamp format
Contact: Thomas Schierl <ts@thomas-schierl.de>
IETF Audio/Video Transport Working Group
Reference: <a href="./rfc6051">RFC 6051</a>
Extension URI: urn:ietf:params:rtp-hdrext:ntp-56
Description: Synchronisation metadata: 56-bit timestamp format
Contact: Thomas Schierl <ts@thomas-schierl.de>
IETF Audio/Video Transport Working Group
Reference: <a href="./rfc6051">RFC 6051</a>
<span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. Acknowledgements</span>
This memo has benefited from discussions with numerous members of the
IETF AVT working group, including Jonathan Lennox, Magnus Westerlund,
Randell Jesup, Gerard Babonneau, Ingemar Johansson, Ali C. Begen,
Ye-Kui Wang, Roni Even, Michael Dolan, Art Allison, and Stefan
Doehla. The RTP header extension format of Variant A in <a href="#section-3.3">Section 3.3</a>
was suggested by Dave Singer, matching a similar mechanism specified
by the Internet Streaming Media Alliance (ISMA).
<span class="grey">Perkins & Schierl Standards Track [Page 19]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-20" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
<span class="h2"><a class="selflink" id="section-8" href="#section-8">8</a>. References</span>
<span class="h3"><a class="selflink" id="section-8.1" href="#section-8.1">8.1</a>. Normative References</span>
[<a id="ref-RFC2119">RFC2119</a>] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", <a href="https://www.rfc-editor.org/bcp/bcp14">BCP 14</a>, <a href="./rfc2119">RFC 2119</a>, March 1997.
[<a id="ref-RFC3550">RFC3550</a>] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, <a href="./rfc3550">RFC 3550</a>, July 2003.
[<a id="ref-RFC4585">RFC4585</a>] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J.
Rey, "Extended RTP Profile for Real-time Transport
Control Protocol (RTCP)-Based Feedback (RTP/AVPF)",
<a href="./rfc4585">RFC 4585</a>, July 2006.
[<a id="ref-RFC5285">RFC5285</a>] Singer, D. and H. Desineni, "A General Mechanism for RTP
Header Extensions", <a href="./rfc5285">RFC 5285</a>, July 2008.
[<a id="ref-RFC5506">RFC5506</a>] Johansson, I. and M. Westerlund, "Support for
Reduced-Size Real-Time Transport Control Protocol (RTCP):
Opportunities and Consequences", <a href="./rfc5506">RFC 5506</a>, April 2009.
[<a id="ref-RFC5583">RFC5583</a>] Schierl, T. and S. Wenger, "Signaling Media Decoding
Dependency in the Session Description Protocol (SDP)",
<a href="./rfc5583">RFC 5583</a>, July 2009.
[<a id="ref-RFC5760">RFC5760</a>] Ott, J., Chesterfield, J., and E. Schooler, "RTP Control
Protocol (RTCP) Extensions for Single-Source Multicast
Sessions with Unicast Feedback", <a href="./rfc5760">RFC 5760</a>, February 2010.
[<a id="ref-RFC5905">RFC5905</a>] Mills, D., Martin, J., Burbank, J., and W. Kasch,
"Network Time Protocol Version 4: Protocol and Algorithms
Specification", <a href="./rfc5905">RFC 5905</a>, June 2010.
<span class="h3"><a class="selflink" id="section-8.2" href="#section-8.2">8.2</a>. Informative References</span>
[<a id="ref-AVT-ACQUISITION-RTP">AVT-ACQUISITION-RTP</a>]
VerSteeg, B., Begen, A., VanCaenegem, T., and Z. Vax,
"Unicast-Based Rapid Acquisition of Multicast RTP
Sessions", Work in Progress, October 2010.
[<a id="ref-AVT-RTP-SVC">AVT-RTP-SVC</a>]
Wenger, S., Wang, Y., Schierl, T., and A. Eleftheriadis,
"RTP Payload Format for SVC Video Coding", Work
in Progress, October 2010.
<span class="grey">Perkins & Schierl Standards Track [Page 20]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-21" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
[<a id="ref-RFC3556">RFC3556</a>] Casner, S., "Session Description Protocol (SDP) Bandwidth
Modifiers for RTP Control Protocol (RTCP) Bandwidth",
<a href="./rfc3556">RFC 3556</a>, July 2003.
[<a id="ref-RFC3711">RFC3711</a>] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K.
Norrman, "The Secure Real-time Transport Protocol
(SRTP)", <a href="./rfc3711">RFC 3711</a>, March 2004.
[<a id="ref-RFC5117">RFC5117</a>] Westerlund, M. and S. Wenger, "RTP Topologies", <a href="./rfc5117">RFC 5117</a>,
January 2008.
[<a id="ref-RFC5245">RFC5245</a>] Rosenberg, J., "Interactive Connectivity Establishment
(ICE): A Protocol for Network Address Translator (NAT)
Traversal for Offer/Answer Protocols", <a href="./rfc5245">RFC 5245</a>,
April 2010.
[<a id="ref-RFC5576">RFC5576</a>] Lennox, J., Ott, J., and T. Schierl, "Source-Specific
Media Attributes in the Session Description Protocol
(SDP)", <a href="./rfc5576">RFC 5576</a>, June 2009.
[<a id="ref-RFC5691">RFC5691</a>] de Bont, F., Doehla, S., Schmidt, M., and R.
Sperschneider, "RTP Payload Format for Elementary Streams
with MPEG Surround Multi-Channel Audio", <a href="./rfc5691">RFC 5691</a>,
October 2009.
[<a id="ref-RFC5764">RFC5764</a>] McGrew, D. and E. Rescorla, "Datagram Transport Layer
Security (DTLS) Extension to Establish Keys for the
Secure Real-time Transport Protocol (SRTP)", <a href="./rfc5764">RFC 5764</a>,
May 2010.
[<a id="ref-ZRTP">ZRTP</a>] Zimmermann, P., Johnston, A., Ed., and J. Callas, "ZRTP:
Media Path Key Agreement for Unicast Secure RTP", Work
in Progress, June 2010.
<span class="grey">Perkins & Schierl Standards Track [Page 21]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-22" ></span>
<span class="grey"><a href="./rfc6051">RFC 6051</a> RTP Synchronisation November 2010</span>
Authors' Addresses
Colin Perkins
University of Glasgow
School of Computing Science
Glasgow G12 8QQ
UK
EMail: csp@csperkins.org
Thomas Schierl
Fraunhofer HHI
Einsteinufer 37
D-10587 Berlin
Germany
Phone: +49-30-31002-227
EMail: ts@thomas-schierl.de
Perkins & Schierl Standards Track [Page 22]
</pre>
|
