1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131
|
Network Working Group S. Armstrong
Request for Comments: 1301 Xerox
A. Freier
Apple
K. Marzullo
Cornell
February 1992
Multicast Transport Protocol
Status of this Memo
This memo provides information for the Internet community. It does
not specify an Internet standard. Distribution of this memo is
unlimited.
Summary
This memo describes a protocol for reliable transport that utilizes
the multicast capability of applicable lower layer networking
architectures. The transport definition permits an arbitrary number
of transport providers to perform realtime collaborations without
requiring networking clients (aka, applications) to possess detailed
knowledge of the population or geographical dispersion of the
participating members. It is not network architectural specific, but
does implicitly require some form of multicasting (or broadcasting)
at the data link level, as well as some means of communicating that
capability up through the layers to the transport.
Keywords: reliable transport, multicast, broadcast, collaboration,
networking.
Table of Contents
1. Introduction 2
2. Protocol description 3
2.1 Definition of terms 3
2.2 Packet format 6
2.2.1. Protocol version 7
2.2.2. Packet type and modifier 7
2.2.3. Subchannel 9
2.2.4. Source connection identifier 9
2.2.5. Destination connection identifier 10
2.2.6. Message acceptance 10
2.2.7. Heartbeat 12
2.2.8. Window 12
2.2.9. Retention 12
Armstrong, Freier & Marzullo [Page 1]
RFC 1301 Multicast Transport Protocol February 1992
2.3 Transport addresses 12
2.3.1. Unknown transport address 12
2.3.2. Web's multicast address 13
2.3.3. Member addresses 13
3. Protocol behavior 13
3.1. Establishing a transport 13
3.1.1. Join request 14
3.1.2. Join confirm/deny 16
3.2 Maintaining data consistency 17
3.2.1. Transmit tokens 17
3.2.2. Data transmission 20
3.2.3. Empty packets 23
3.2.4. Missed data 26
3.2.5. Retrying operations 26
3.2.6. Retransmission 27
3.2.7. Duplicate suppression 29
3.2.8. Banishment 29
3.3 Terminating the transport 29
3.3.1. Voluntary quits 30
3.3.2. Master quit 30
3.3.3. Banishment 30
3.4 Transport parameters 30
3.4.1. Quality of service 30
3.4.2. Selecting parameter values 31
3.4.3. Caching member information 33
A. Appendix: MTP as an Internet Protocol transport 34
A.1 Internet Protocol multicast addressing 34
A.2 Encapsulation 35
A.3 Fields of the bridge protocol 35
A.4 Relationship to other Internet Transports 36
References 36
Footnotes 37
Security Considerations 37
Authors' Addresses 38
1. Introduction
This document describes a flow controlled, atomic multicasting
transport protocol (MTP). The purpose of this document is to present
sufficient information to implement the protocol.
The MTP design has been influenced by the large body of the
networking and distributed systems literature and technology that has
been introduced during the last decade and a half. Representative
sources include [Xer81], [BSTM79] and [Pos81] for transport design,
and [Bog83] and [DIX82] for general concepts of broadcast and
multicast. [CLZ87] influenced MTP's retransmission mechanisms, and
[Fre84] influenced the transport timings. MTP over IP uses mechanisms
Armstrong, Freier & Marzullo [Page 2]
RFC 1301 Multicast Transport Protocol February 1992
described in [Dee89]. MTP's ordering and agreement protocols were
influenced by work done in [CM87], [JB89] and [Cri88]. Finally, a
description of MTP's philosophy and its motivation can be found in
[AFM91].
2. Protocol description
MTP is a transport in that it is a client of the network layer (as
defined by the OSI networking model) [1]. MTP provides reliable
delivery of client data between one or more communicating processes,
as well as a predefined principal process. The collection of
processes is called a web.
In addition to transporting data reliably and efficiently, MTP
provides the synchronization necessary for web members to agree on
the order of receipt of all messages and can agree on the delivery of
the message even in the face of partitions. This ordering and
agreement protocol uses serialized tokens granted by the master to
producers.
The processes may have any one of three levels of capability. One
member must be the master. The master instantiates and controls the
behavior of the web, including its membership and performance. Non
master members may be either producer/consumers or pure consumers.
The former class of member is permitted to transmit user data to the
entire membership (and expected to logically hear itself), while the
latter is prohibited from transmitting user data.
MTP is a negative acknowledgement protocol, exploiting the highly
reliable delivery of the local area and wide area network
technologies of today. Successful delivery of data is accepted by
consuming stations silently rather than having the successful
delivery noted to the producing process, thus reducing the amount of
reverse traffic required to maintain synchronization.
2.1 Definition of terms
The following terms are used throughout this document. They are
defined here to eliminate ambiguity.
consumer A consumer is a transport that is capable only of
receiving user data. It may transmit control packets,
such as negative acknowledgements, but may never transmit
any requests for the transmit token or any form of data
or empty messages.
heartbeat A heartbeat is an interval of time, nominally measured in
milliseconds. It is a key parameter in the transport's
Armstrong, Freier & Marzullo [Page 3]
RFC 1301 Multicast Transport Protocol February 1992
state and can be adapted to the requirements of the
transport's client to provide the desired quality of
service.
master The master is the principal member of the web. The master
capability is a superset of a producer member. The
master is mainly responsible for giving out transmit
tokens to members who wish to send data, and overseeing
the web's membership and operational parameters.
member A web member is any process that has been permitted to
join the web (by the master) as well as the master
itself.
membership Every member is classified as to its intentions for
class joining the web. Membership classes are defined to be
consumer, producer and master. Each successive class is a
formal superset of the previous.
message An MTP message is a concatenation of the user data
portions of a series of data packets with the last packet
in the series carrying an end of message indication. A
message may contain any number of bytes of user data,
including zero.
NSAP The network service access point. This is the network
address, or the node address of the machine, where a
service is available.
producer Producer is a class of membership that is a formal
superset of a consumer. A producer is permitted (and
expected) to transmit client data as well as consume data
transmitted by other producers.
retention Retention is one of the three fundamental parameters that
make up the transport's state (along with heartbeat and
window). Retention is a number of heartbeats, and though
applied in several different circumstances, is primarily
used as the number of heartbeats a producing client must
maintain buffered data should it need to be
retransmitted.
token In order to transmit, a producer must first be in
possesion of a token. Tokens are granted only by the
master and include the message sequence number.
Consequently, they are fundamental in the operation of
the ordering and agreement protocol used by MTP.
Armstrong, Freier & Marzullo [Page 4]
RFC 1301 Multicast Transport Protocol February 1992
TSAP The transport service access point. This is the address
that uniquely defines particular instantiation of a
service. TSAPs are formed by logically concatenating the
node's NSAP with a transport identifier (and perhaps a
packet/protocol type).
user data User data is the client information carried in MTP data
packets and treated as uninterpreted octets by the
transport. The end of message and subchannel indicators
are also be treated as user data.
web A collection of processes collaborating on the solution
of a single problem.
window The window is one of the fundamental elements of the
transport's state that can be controlled to affect the
quality of service being provided to the client. It
represents the number of user data carrying packets that
may be multicast into the web during a heartbeat by a
single member.
Armstrong, Freier & Marzullo [Page 5]
RFC 1301 Multicast Transport Protocol February 1992
2.2 Packet format
An MTP packet consists of a transport protocol header followed by a
variable amount of data. The protocol header, shown in Figure 1, is
part of every packet. The remainder of the packet is either user data
(packet type = data) or additional transport specific information.
The fields in the header are statically defined as n-bit wide
quantities. There are no undefined fields or fields that may at any
time have undefined values. Reserved fields, if they exist, must
always have a value of zero.
0 7 8 15 16 23 24 31
---------------------------------------------------------- -----
| protocol | packet | type | client | |
| version | type | modifier | channel | |
---------------------------------------------------------- |
| | |
| source connection identifier | |
---------------------------------------------------------- |
| | |
| destination connection identifier |
---------------------------------------------------------- transport
| | header
| message acceptance criteria |
---------------------------------------------------------- |
| | |
| heartbeat | |
---------------------------------------------------------- |
| | | |
| window | retention | |
---------------------------------------------------------- -----
| | |
| | |
| | |
| (data content and format |
| dependent on packet type | data
| and modifier) | fields
| |
| | |
| | |
| | |
---------------------------------------------------------- -----
Figure 1. MTP packet format
Armstrong, Freier & Marzullo [Page 6]
RFC 1301 Multicast Transport Protocol February 1992
2.2.1. Protocol version
The first 8 bits of the packet are the protocol version number. This
document describes version 1 of the Multicast Transport Protocol and
thus the version field has a value of 0x01.
2.2.2. Packet type and modifier
The second byte of the header is the packet type and the following
byte contains the packet type modifier. Typical control message
exchanges are in a request/response pair. The modifier field
simplifies the construction of responses by permitting reuse of the
incoming message with minimal modification. The following table gives
the packet type field values along with their modifiers. The
modifiers are valid only in the context of the type. In the prose of
the definitions and later in the document, the syntax for referring
to one of the entries described in the following table will be
type[modifier]. For example, a reference to data[eow] would be a
packet of type data with an end of window modifier.
type modifier description
data(0) data(0) The packet is one that contains user
information. Only the process possessing a
transmit token is permitted to send data
unless specifically requested to retransmit
previously transmitted data. All packets of
type data are multicast to the entire web.
eow(1) A data packet with the eow (end of window)
modifier set indicates that the transmitter
intends to send no more packets in this
heartbeat either because it has sent as many
as permitted given the window parameter or
simply has no more data to send during the
current heartbeat. This is not client
information but rather a hint to be used by
transport providers to synchronize the
computation and transmission of naks.
eom(2) Data[eom] marks the end of the message to the
consumers, and the surrendering of the
transmit token to the master. And like a
data[eow] a data[eom] packet implies the end
of window.
nak(1) request(0) A nak[request] packet is a consumer
requesting a retransmission of one or more
Armstrong, Freier & Marzullo [Page 7]
RFC 1301 Multicast Transport Protocol February 1992
data packets. The data field contains an
ordered list of packet sequence numbers that
are being requested. Naks of any form are
always unicast.
deny(1) A nak[deny] message indicates that the
producer source of the nak[deny]) cannot
retransmit one or more of the packets
requested. The process receiving the
nak[deny] must report the failure to its
client.
empty(2) dally(0) An empty[dally] packet is multicast to
maintain synchronization when no client data
is available.
cancel(1) If a producer finds itself in possession of a
transmit token and has no data to send, it
may cancel the token[request] by multicasting
an empty[cancel] message.
hibernate(2) If the master possesses all of the web's
transmit tokens and all outstanding messages
have been accepted or rejected, the master
may transmit empty[hibernate] packets at a
rate significantly slower than indicated by
the web's value of heartbeat.
join(3) request(0) A join[request] packet is sent by a process
wishing to join a web to the web's unknown
TSAP (see section 2.2.5).
confirm(1) The join[confirm] packet is the master's
confirmation of the destination's request to
join the web. It will be unicast by the
master (and only the master) to the station
that sent the join[request].
deny(2) A join[deny] packet indicates permission to
join the web was denied. It may only be
transmitted by the master and will be unicast
to the member that sent the join[request].
quit(4) request(0) A quit[request] may be unicast to the master
by any member of the web at any time to
indicate the sending process wishes to
withdraw from the web. Any member may unicast
a quit to another member requesting that the
Armstrong, Freier & Marzullo [Page 8]
RFC 1301 Multicast Transport Protocol February 1992
destination member quit the web due to
intolerable behavior. The master may
multicast a quit[request] requiring that the
entire web disband. The request will be
multicast at regular heartbeat intervals
until there are no responses to retention
requests.
confirm(1) The quit[confirm] packet is the indication
that a quit[request] has been observed and
appropriate local action has been taken.
Quit[confirm] are always unicast.
token(5) request(0) A token[request] is a producing member
requesting a transmit token from the master.
Such packets are unicast to the master.
confirm(1) The token[confirm] packet is sent by the
master to assign the transmit token to a
member that has requested it. token[confirm]
will be unicast to the member being granted
the token.
isMember(6) request(0) An isMember[request] is soliciting
verification that the target member is a
recognized member of the web. All forms of
the isMember packet are unicast to a specific
member.
confirm(1) IsMember[confirm] packets are positive
responses to isMember[requests].
deny(2) If the member receiving the isMember[request]
cannot confirm the target's membership in the
web, it responds with a isMember[deny].
2.2.3. Subchannel
The fourth byte of the transport header contains the client's
subchannel value. The default value of the subchannel field is zero.
Semantics of the subchannel value are defined by the transport client
and therefore are only applicable to packets of type data. All other
packet types must have a subchannel value of zero.
2.2.4. Source connection identifier
The source connection identifier field is a 32 bit field containing a
transmitting system unique value assigned at the time the transport
Armstrong, Freier & Marzullo [Page 9]
RFC 1301 Multicast Transport Protocol February 1992
is created. The field is used in identifying the particular transport
instantiation and is a component of the TSAP. Every packet
transmitted by the transport must have this field set.
2.2.5. Destination connection identifier
The destination connection identifier is the 32 bit identifier of the
target transport. From the point of view of a process sending a
packet, there are three types of destination connection identifiers.
First, there is the unknown connection identifier (0x00000000). The
unknown value is used only as the destination connection identifier
in the join[request] packet.
Second, there is the multicast connection identifier gleaned from the
join[confirm] message sent by the master. The multicast connection
identifier is used in conjunction with the multicast NSAP to form the
destination TSAP of all packets multicast to the entire web [2].
The last class of connection identifier is a unicast identifier and
is used to form the destination TSAP when unicasting packets to
individual members. Every member of the web has associated with it a
unicast connection identifier that is used to form its own unicast
TSAP.
2.2.6. Message acceptance
MTP ensures that all processes agree on which messages are accepted
and in what order they are accepted. The master controls this aspect
of the protocol by controlling allocation of transmit tokens and
setting the status of messages. Once a token for a message has been
assigned (see section 3.2.1) the master sets the status of that
message according to the following rules [AFM91]:
If the master has seen the entire message (i.e., has seen the
data[eom] and all intervening data packets), the status is accepted.
If the master has not seen the entire message but believes the
message sender is still operational and connected to the master (as
determined by the master), the status is pending.
If the master has not seen the entire message and believes the
sender to have failed or partitioned away, the status is rejected.
Message status is carried in the message acceptance record (see
Figure 2) of every packet, and processes learn the status of earlier
messages by processing this information.
The acceptance criteria is a multiple part record that carries the
Armstrong, Freier & Marzullo [Page 10]
RFC 1301 Multicast Transport Protocol February 1992
rules of agreement to determine the message acceptance. The most
significant 8 bits is a flag that, if not zero, indicates
synchronization is required. The field may vary on a per message
basis as directed by producing transport's client. The default is
that no synchronization is required.
The second part of the record is a 12 element vector that represents
the status of the last 12 messages transmitted into the web.
0 7 8 15 16 23 24 31
---------------------------------------------------------
| | |
| synchro | tri-state bitmask[12] |
---------------------------------------------------------
| message | packet sequence |
| sequence number | number |
---------------------------------------------------------
Figure 2. Message acceptance record
Each element of the array is two bits in length and may have one of
three values: accepted(0), pending(1) or rejected(2). Initially, the
bit mask is set to all zeros. When the token for message m is
transmitted, the first (left-most) element of the vector represents
the the state of message m - 1, the second element of the vector is
the status of message m - 2, and so forth. Therefore the status of
the last 12 messages are visible, the status of older messages are
lost, logically by shifting the elements out of the vector. Only the
master is permitted to set the status of messages. The master is not
permitted to shift a status of pending beyond the end of the vector.
If that situation arises, the master must instead not confirm any
token[request] until the oldest message can be marked as either
rejected or accepted.
Message sequence numbers are 16 bit unsigned values. The field is
initialized to zero by the master when the transport is initialized,
and incremented by one after each token is granted. Only the master
is permitted to change the value of the message sequence number. Once
granted, that message sequence number is consumed and the state of
the message must eventually become either accepted or rejected. No
transmit tokens may be granted if the assignment of a message
sequence number that would cause a value of pending to be shifted
beyond the end of the status vector.
Packet sequence numbers are unsigned 16 bit numbers assigned by the
producing process on a per message basis. Packet sequence numbers
start at a value of zero for each new message and are incremented by
one (consumed) for each data packet making up the message. Consumers
Armstrong, Freier & Marzullo [Page 11]
RFC 1301 Multicast Transport Protocol February 1992
detecting missing packet sequence numbers must send a nak[request] to
the appropriate producer to recover the missed data.
Control packets always contain the message acceptance criteria with a
synchronization flag set to zero (0x00), the highest message sequence
number observed and a packet sequence number one greater than
previously observed. Control packets do not consume any sequence
numbers. Since control messages are not reliably delivered, the
acceptance criteria should only be checked to see if they fall within
the proper range of message numbers, relative to the current message
number of the receiving station. The range of acceptable sequence
numbers should be m-11 to m-13, inclusive, where m is the current
message number.
2.2.7. Heartbeat
Heartbeat is an unsigned 32 bit field that has the units of
milliseconds. The value of heartbeat is shared by all members of the
web. By definition at least one packet (either data, empty or quit
from the master) will be multicast into the web within every
heartbeat period.
2.2.8. Window
The allocation window (or simply window) is a 16 bit unsigned field
that indicates the maximum number of data packets that can be
multicasted by a member in a single heartbeat. It is the sum of the
retransmitted and new data packets.
2.2.9. Retention
The retention field is a 16 bit unsigned value that is the number of
heartbeats for which a producer must retain transmitted client data
and state for the purpose of retransmission.
2.3 Transport addresses
Associated with each transport are logically three transport service
access points (TSAP), logically formed by the concatenation of a
network service access point (NSAP) and a transport connection
identifier. These TSAPs are the unknown TSAP, the web's multicast
TSAP and each individual member's TSAP.
2.3.1. Unknown transport address
Stations that are just joining must use the multicast NSAP associated
with the transport, but are not yet aware of either the web's
multicast TSAP the master process' TSAP. Therefore, joining stations
Armstrong, Freier & Marzullo [Page 12]
RFC 1301 Multicast Transport Protocol February 1992
fabricate a temporary TSAP (referred to as a unknown TSAP) by using a
connection identifier reserved to mean unknown (0x00000000). The
join[confirm] message will be sourced from the master's TSAP and will
include the multicast transport connection identifier in the data
field. Those values must be extracted from the join[confirm] and
remembered by the joining process.
2.3.2. Web's multicast address
The multicast TSAP is formed by logically concatenating the multicast
NSAP associated with the transport creation and the transport
connection identifier returned in the data field of the join[confirm]
packet. If more than one network is involved in the web, then the
multicast transport address becomes a list, one for each network
represented. This list is supplied in the data field of
token[confirm] packets.
The multicast TSAP is used as the target for all messages that are
destined to the entire web, such as data and empty. The master's
decision to abandon the transport (quit) is also sent to the
multicast transport address.
2.3.3. Member addresses
The member TSAP is formed by using the process' unicast NSAP
concatenated with a locally generated unique connection identifier.
That TSAP must be the source of every packet transmitted by the
process, regardless of its destination, for the lifetime of the
transport.
Packets unicast to specific members must contain the appropriate
TSAP. For producers and consumers this is not difficult. The only
TSAPs of interest are the master and the station(s) currently
transmitting data.
3. Protocol behavior
This section defines the expectations of the protocol implementation.
These expectations should not be considered guidelines or hints, but
rather part the protocol.
3.1 Establishing a transport
Before any rendezvous can be affected, a process must first acquire
an NSAP that will be the service access point for the instantiation
[3]. The process that first establishes at that NSAP is referred to
as the master of the web. The decision as to what process acts as the
master must be made a priori in order to guarantee unambiguous
Armstrong, Freier & Marzullo [Page 13]
RFC 1301 Multicast Transport Protocol February 1992
creation in the face of network partitions. The process should make a
robust effort to verify that the NSAP being used is not already in
service. It may do so by repeatedly sending join[requests] to the
web's unknown TSAP. If there is no response to repeated transmissions
the process may be relatively confident that the NSAP is not in use
and proceed with the creation of the web. If not, the creation must
be aborted and the situation reported to its client.
3.1.1. Join request
Additional members may join the web at any time after the
establishment of the master by the joining process sending a
join[request] to the unknown TSAP. The joining process should have
already assigned a unique connection identifier to its transport
instantiation that will be used in the source TSAP of the
join[request]. The join[request] must contain zeros in all of the
acceptance fields. The heartbeat, window and retention parameters are
filled in as requested by the transport provider's client. The data
of the message must contain the type, class and quality of service
parameters that the client has requested.
field class definition
membership class master(0) There can be only a single web
master, and that member has all
privileges of a producer class member
plus those acquitted only to the
master.
producer(1) A process that has producer class
membership wishes to transmit data
into the web as well as consume.
consumer(2) A consumer process is a read only
process. It will send naks in order
to reliably receive data but will
never ask for or be permitted to take
possession of a transmit token.
transport class reliable(0) Specifies a reliable transport, i.e.,
one that will generate and process
naks. The implication is that the
data will be reliably delivered or
the failure will be detected and
reported to the client.
unreliable(1) The transport supports best
Armstrong, Freier & Marzullo [Page 14]
RFC 1301 Multicast Transport Protocol February 1992
effort delivery. Such a transport may
still fail if the error rates are too
high, but tolerable loss or
corruption of data will be permitted
[4].
transport type NxN(0) The transport will accept multiple
processes with producing capability.
1xN(1) A 1xN transport permits only a single
producer whose identity was
established a priori.
The client's desire for minimum throughput (expressed in kilobytes
per second) is the lowest value that will be accepted. That
throughput is calculated using the heartbeat and window parameters of
the transport, and the maximum data unit size, not by measuring
actual traffic. Any member that suggests a combination of those
parameters that result in an unacceptable throughput will be ignored
or asked to withdraw from the web.
A joining client may also suggest a maximum data unit size. This
field is expressed as a number of bytes that can be included in a
data packet as client data.
If no response is received in a single heartbeat, the join[request]
should be retransmitted using the same source TSAP so the master can
detect the difference between a new process and a retransmission of a
join[request].
Armstrong, Freier & Marzullo [Page 15]
RFC 1301 Multicast Transport Protocol February 1992
3.1.2. Join confirm/deny
Only the master of the web will respond to join[request]. The
response may either permit the entry of the new process or deny it.
The request to join may be denied because the new member is
specifying service parameters that are in conflict with those
established by the master. If the join is confirmed the
join[confirm] will be unicast by the master with a data field that
contains the web's current operating parameters. If those parameters
are unacceptable to the joining process it may decide to withdraw
from the web. Otherwise the parameters must be accepted as the
current operating values.
0 7 8 15 16 23 24 31
---------------------------------------------------------- -----
| protocol | packet | type | client | |
| version | type | modifier | channel | |
---------------------------------------------------------- |
| | |
| source connection identifier | |
---------------------------------------------------------- |
| | |
| destination connection identifier |
---------------------------------------------------------- transport
| | header
| message acceptance criteria |
---------------------------------------------------------- |
| | |
| heartbeat | |
---------------------------------------------------------- |
| | | |
| window | retention | |
---------------------------------------------------------- -----
| member | transport | transport | | |
| class | class | type | reserved | |
----------------------------------------------------------
| minimum | maximum data | data
| throughput | unit size |
---------------------------------------------------------- |
| multicast connection | |
| identifier | |
---------------------------------------------------------- -----
Figure 3. join packet
The join[confirm] will also contain the multicast connection
identifier. This must be used to form the TSAP that will be the
destination for all multicast messages for the transport. The source
Armstrong, Freier & Marzullo [Page 16]
RFC 1301 Multicast Transport Protocol February 1992
of the join[confirm] message will be the master's TSAP and must be
recorded by the member for later use.
The master must be in possession of all the transmit tokens when it
sends a join[confirm]. Requiring the master to have the transmit
tokens insures that the joining member will enter the web and observe
only complete messages. It also permits a notification of the
master's client of the join so that application state may be
automatically sent to the newly joining member. The newly joined
member may be on a network not previously represented in the web's
membership, thus requiring a new multicast TSAP be added to the
existing list. The entire list will be conveyed in the data field of
all subsequent token[confirm] messages (described later).
3.2 Maintaining data consistency
The transport is responsible for maintaining the consistency of the
data submitted for delivery by producing clients. The actual client
data, while representing the bulk of the information that flows
through the web, is accompanied by significant amounts of protocol
state information. In addition to the state information piggybacked
with the client data, there is a minimum amount of protocol packets
that are purely for use by the transport, invisible to the transport
client.
3.2.1. Transmit tokens
Before any process may transmit client data or state it must first
possess a transmit token. It may acquire the token by transmitting a
token[request] to the master. Requests should be unicast to the
master's TSAP and should be retransmitted at intervals approximately
equal to the heartbeat. Since it is the central source for a transmit
token, the master may apply some fairness algorithms to the passing
of permission to transmit. At a minimum the requests should be queued
in a first in, first out order. Duplicate requests from a single
member should be ignored, keeping instead the first unhonored
request. When appropriate, the master will send a member with a
request pending a token[confirm]. The data field of the response
contains all the multicast TSAPs that are represented in the current
web at that point in time.
If the master detects no data or heartbeat messages being transmitted
into the web it will assume the token is lost, presumably because the
member holding the token has failed or has become partitioned away
from the master. In such cases, the master may attempt to confirm the
state of the process (perhaps by sending isMember[request]). If the
member does not respond it is removed from the active members of the
web, the message is marked as rejected, the token is assumed by the
Armstrong, Freier & Marzullo [Page 17]
RFC 1301 Multicast Transport Protocol February 1992
master.
Figure 4 shows a timing diagram of a token pass. Increasing time is
towards the bottom of the figure. In this figure, process A has a
token, and process B requests a token when there are no free tokens.
A master B
"A" multicasts data | | "B" requests
|\ | | transmit token
| \ | /|
| \ | / |
| \ | / |
"A" multicasts data | \ | / | "B" retransmits
w/eom set |\ \| / | token request
| \ \V /|
| \ |\ / |
| \ | V / |
| \ | / |
| \| / |
| \V |
| |\ |
| | V |
| |\ | Master assigns
| | \ | token to "B"
| | \ |
| | \ |
| | \ |
| | V|
| | |
| | /| "B" multicasts
| | / | data
| | / |
| | / |
| | / |
| |/ |
| / |
| /| |
| V | |
| | |
Figure 4. Acquiring the token
Token packets, like other control packets, do not consume sequence
numbers. Hence, the master must be able to use another mechanism to
determine whether multiple token[request] from a single member are
actually requests for a separate token, or are a retransmission of a
token[request]. To carry out this obligation, the master and the
members must have an implicit understanding of each other's state.
Armstrong, Freier & Marzullo [Page 18]
RFC 1301 Multicast Transport Protocol February 1992
0 7 8 15 16 23 24 31
---------------------------------------------------------- -----
| protocol | packet | type | client | |
| version | type | modifier | channel | |
---------------------------------------------------------- |
| | |
| source connection identifier | |
---------------------------------------------------------- |
| | |
| destination connection identifier |
---------------------------------------------------------- transport
| | header
| message acceptance criteria |
---------------------------------------------------------- |
| | |
| heartbeat | |
---------------------------------------------------------- |
| | | |
| window | retention | |
---------------------------------------------------------- -----
| | |
| | |
| TSAPs of all networks |
| represented in the web | data
| membership |
| | |
| | |
---------------------------------------------------------- -----
Figure 5. token packet
Assume that the token, as viewed by the master, has three states:
idle The token is not currently assigned. Specifically the
message number that it defines is not represented in the
current message acceptance vector.
pending The token has been assigned by the master via a
token[confirm] packet, but the master has not yet seen
any data packets to indicate that the from the producing
member received the notification.
busy The token has been assigned and the master has seen data
packets carrying the assigned message number. The message
comprised by those packets is still represented in the
message acceptance vector.
Furthermore, a token that is not idle also has associated with its
Armstrong, Freier & Marzullo [Page 19]
RFC 1301 Multicast Transport Protocol February 1992
state the TSAP of the process that owns (or owned) the token.
Based on this state, the master will respond to any process that has
a token in pending state with a reassignment of that token. This is
based on the assumption that the original token[confirm] was not
received by the requesting process. The only other possibility is
that the process did receive the token and transmitted data packets
using that token, but the master did not see them. But data messages
are by design multi-packet messages, padded with empty packets if
necessary. The possibility of the master missing all of the packets
of a message is considered less than the possibility of the
requesting process missing a single token[confirm] packet.
The process requesting tokens must consider the actions of the master
and what prompted them. In most cases the assumptions made by the
master will be correct. However, there are two ambiguous situations.
There is the situation that the master is most directly addressing,
not knowing whether the requesting process has failed to observe the
token[confirm] or the master has failed to see data packets
transmitted by the producing process. There is also the possibility
that the requesting process timed out too quickly and the
retransmission of the token[request] passed the token[confirm] in the
night. In any case the producing process may find itself in
possession of a token for which it has no need. These can be
dismissed by sending an empty[cancel] packet.
Another possibility is that the requesting process has actually made
use of the assigned token and is requesting another token. Unless the
master has observed data using the token, the master will still
consider the token pending. Therefore, a process that receives a
duplicate token[confirm] should interpret it as a nak and retransmit
any data packets previously sent using the token's message sequence
number.
3.2.2. Data transmission
Data is provided by the transport client in the form of uninterpreted
bytes. The bytes are encapsulated in packets immediately following
the protocol's fixed overhead fields. The packet may have any number
of data bytes between zero and the maximum number of bytes of a
network protocol packet minus the network overhead and the fixed
transport overhead. Every packet that consumes a sequence number
must contain either client data or client state transitions such as
the end of message indicator or a subchannel transition.
Packets are transmitted in bursts of packets called windows. The
protocol guarantees that no more than the current value of window
data packets will be transmitted by a single process during a
Armstrong, Freier & Marzullo [Page 20]
RFC 1301 Multicast Transport Protocol February 1992
heartbeat. Every packet transmitted always contains the latest
heartbeat, window and retention information. If full packets are
unavailable [5], empty[dally] messages should be transmitted instead.
The only packets that will be transmitted containing less than
maximum capacity will be data[eom] or those containing client
subchannel transitions.
Armstrong, Freier & Marzullo [Page 21]
RFC 1301 Multicast Transport Protocol February 1992
----- | |
| |\ |
| | \ |
|\ \ |
heartbeat | \ \ |
|\ \ \ |
| | \ \ V| data(n)
| | \ \ |
----- | \ V| data(n+1)
|\ \ |
| \ V| data(n+w-1) w/eow
|\ \ |
| \ \ |
|\ \ \ |
| \ \ V| data(n+w)
| \ \ |
----- | \ V| data(n+w+1)
|\ \ |
| \ V| data(n+2w-1) w/eow
w = window = 3 | \ |
r = retention = 2 | \ |
| \ |
| V| empty(n+2w)
| |
----- | |
|\ |
| \ |
| \ |
| \ |
| \ |
| V| data(n+2w) w/eom
| | Packets n..n+w-1 are released,
----- | | token is surrendered.
| |
| |
| |
| |
| |
| |
| |
----- | | Packets n+w..n+2w-1 are released.
Figure 6. Normal data transmission
Figure 6 shows a timing diagram of a process transmitting into a web
(without any complicating naks). Increasing time is towards the
bottom of the figure. The transmitting process is obligated to
Armstrong, Freier & Marzullo [Page 22]
RFC 1301 Multicast Transport Protocol February 1992
retransmit requested packets for at least retention heartbeat
intervals after their first transmission.
0 7 8 15 16 23 24 31
---------------------------------------------------------- -----
| protocol | packet | type | client | |
| version | type | modifier | channel | |
---------------------------------------------------------- |
| | |
| source connection identifier | |
---------------------------------------------------------- |
| | |
| destination connection identifier |
---------------------------------------------------------- transport
| | header
| message acceptance criteria |
---------------------------------------------------------- |
| | |
| heartbeat | |
---------------------------------------------------------- |
| | | |
| window | retention | |
---------------------------------------------------------- -----
| | |
| uninterpreted data |
| | data
| |
| | |
---------------------------------------------------------- -----
Figure 7. data packet
3.2.3. Empty packets
An empty packet is a control packet multicast into the web at regular
intervals by a producer possessing a transmit token when no client
data is available. Empty packets are sent to maintain synchronization
and to advertise the maximum sequence number of the producer. It
provides the opportunity for consuming processes to detect and
request retransmission of missed data as well as identifying the
owner of a transmit token.
Armstrong, Freier & Marzullo [Page 23]
RFC 1301 Multicast Transport Protocol February 1992
0 7 8 15 16 23 24 31
---------------------------------------------------------- -----
| protocol | packet | type | client | |
| version | type | modifier | channel | |
---------------------------------------------------------- |
| | |
| source connection identifier | |
---------------------------------------------------------- |
| | |
| destination connection identifier |
---------------------------------------------------------- transport
| | header
| message acceptance criteria |
---------------------------------------------------------- |
| | |
| heartbeat | |
---------------------------------------------------------- |
| | | |
| window | retention | |
---------------------------------------------------------- -----
Figure 8. empty packet
There are two situations where the empty[dally] packet is used. The
first is when there is insufficient data for a full packet presented
by the client during a heartbeat. Partial packets should not be
transmitted unless there is a client transition to be conveyed, yet
something must be transmitted during a heartbeat or the master may
think the process owning a transmit token has failed. Empty[dally] is
used instead of a data packet until the client provides additional
data to fill a packet or indicates a state transition such as an end
of message or subchannel transition.
The second situation where empty[dally] is used is after the
transmission of short messages. Each message should consist of
multiple packets in order to enhance the possibility that consumers
will observe at least one packet of a message and therefore be able
to identify the producer. The transport parameter retention has
approximately the correct properties for that insurance. Therefore, a
message must consist of at least retention packets. If the client
data does not require that many packets, empty[dally] packets must be
appended. A process that has no transmittable data and is in
possession of a transmit token must send an empty[cancel].
Transmissions of empty[cancel] packets pass the ownership of the
transmit token back to the master. When the master observes the
control packet, it will mark the referenced to message as rejected so
that other consumers do not believe the message lost and attempt to
recover.
Armstrong, Freier & Marzullo [Page 24]
RFC 1301 Multicast Transport Protocol February 1992
During periods of no activity (i.e., after all messages have been
either accepted or rejected and there are no outstanding transmit
tokens) the master may enter hibernation mode by transmitting
empty[hibernate] packets. In that mode the master will increase the
value of the transport parameter heartbeat in order to reduce network
traffic. Such packets are used to indicate that the packet's
heartbeat field should not be used for resource computation by those
processes that observe it.
Armstrong, Freier & Marzullo [Page 25]
RFC 1301 Multicast Transport Protocol February 1992
3.2.4. Missed data
The most common method of detecting data loss will be the reception
of a data or a heartbeat message that has a sequence number greater
than expected from that producer. The second most common method will
be a message fragment (missing the end of message) and seeing no more
data or empty packets from the producer of the fragment for more than
a single heartbeat. In any case the consumer process directs a
negative acknowledgment (nak) to the producer of the incomplete
message. The data field of the nak message contains a list of
ascending sequence number pairs the consumer needs to recover the
missed data.
0 7 8 15 16 23 24 31
---------------------------------------------------------- -----
| protocol | packet | type | client | |
| version | type | modifier | channel | |
---------------------------------------------------------- |
| | |
| source connection identifier | |
---------------------------------------------------------- |
| | |
| destination connection identifier |
---------------------------------------------------------- transport
| | header
| message acceptance criteria |
---------------------------------------------------------- |
| | |
| heartbeat | |
---------------------------------------------------------- |
| | | |
| window | retention | |
---------------------------------------------------------- -----
| | | |
| message sequence (low) | packet sequence (low) |
---------------------------------------------------------- data
| | |
| message sequence (high) | packet sequence (high) | |
---------------------------------------------------------- -----
Figure 9. nak packet
3.2.5. Retrying operations
Operations must be retried in order to assure that a single packet
loss does not cause transport failure. In general the right numbers
to do that with exist in the transport. The proper interval between
retries is the transport's time constant or heartbeat. The proper
Armstrong, Freier & Marzullo [Page 26]
RFC 1301 Multicast Transport Protocol February 1992
number of retries is retention.
Operations that are retriable (and represented by their respective
message types) are join, nak, token, isMember and quit. Another
application for the heartbeat and retention is when transmitting
empty messages. Empty[dally] messages are transmitted any time data
is not available but the data[eom] has not yet been sent. Any process
not observing data or empty for more than retention heartbeat
intervals will assume to have failed or partitioned away and the
transport will be abandoned.
3.2.6. Retransmission
If the producer receives a nak[request] from a consumer process
requesting the retransmission of a packet that is no longer
available, the producer must send a nak[deny] to the source of the
request. If that puts the consumer in a failed state, the consumer
will initiate the withdrawal from the web. If a producer receives a
nak[request] from a consumer requesting the retransmission of one or
more packets, those packets will be multicast to the entire web [6].
All will contain the original client information (such as subchannel
and end of message state) and message and packet sequence number.
However, the retransmitted packets must contain updated protocol
parameter information (heartbeat, window and retention).
Retransmitted packets are subject to the same constraints regarding
heartbeat and window as original transmissions. Therefore the
producer's retransmissions consume a portion of the allocation window
allowing less new data to be transmitted in a single heartbeat.
Retransmitted packets have priority over (i.e., should be transmitted
before) new data packets.
Armstrong, Freier & Marzullo [Page 27]
RFC 1301 Multicast Transport Protocol February 1992
----- | | retransmission count = rx=0
| |\ |
| | \ |
| |\ \ |
| | \ \ |
| |\ \ \ |
| | \ \ V| data(n)
| | \ \ |
| \ *| data(n+1)
heartbeat | \ |
| V| data(n+w-1-rx) w/eow rx=0
| | |
| | /| nak(n') of n+1
| | / |
| | / |
| | / |
| | / |
| |V |
----- | |
|\ |
| \ |
|\ \ |
| \ \ |
|\ \ \ |
w = window = 3 | \ \ *| retransmission(n+1) rx=1
r = retention = 1 | \ \ |
| \ V| data(n+w)
| \ |
| V| data(n+2w-1-rx) w/eow rx=1
| |
| /| nak(n') of n+1
| / |
----- | / |
|\ / |
| / |
|V \ |
|\ \ |
| \ \ |
|\ \ V| data(n+2w-rx) rx=1
| \ \ | Packets n..n+w-1-0 can be released.
| \ \ |
| \ V| nak deny(n+1) rx=2
| \ |
| V| data(n+3w-1-rx) w/eom rx=2
| |
----- | | Packets n+w..n+2w-1-1 are released.
Figure 10. naks and retransmission
Armstrong, Freier & Marzullo [Page 28]
RFC 1301 Multicast Transport Protocol February 1992
3.2.7. Duplicate suppression
The consumer must be prepared to ignore duplicate packets received.
They will invariably be the result of the producer's retransmission
in response to another consumer's nak.
3.2.8. Banishment
If at any time a process detects another in violation of the protocol
it may ask the offending process to withdraw from the web by
unicasting to it a quit[request] that has the target field set to the
value of the offender's TSAP. Any member that exhibits a detectable
and recoverable protocol violation and still responds willingly to
the quit[request] will be noted as having truly correct social
behavior.
0 7 8 15 16 23 24 31
---------------------------------------------------------- -----
| protocol | packet | type | client | |
| version | type | modifier | channel | |
---------------------------------------------------------- |
| | |
| source connection identifier | |
---------------------------------------------------------- |
| | |
| destination connection identifier |
---------------------------------------------------------- transport
| | header
| message acceptance criteria |
---------------------------------------------------------- |
| | |
| heartbeat | |
---------------------------------------------------------- |
| | | |
| window | retention | |
---------------------------------------------------------- -----
| |
| target TSAP |
| |
----------------------------------------------------------
Figure 11. quit packet
3.3 Terminating the transport
Transport termination is an advisory process that may be initiated by
any member of the web. No process should intentionally quit the web
while it has retransmittable data buffered. Stations should make
Armstrong, Freier & Marzullo [Page 29]
RFC 1301 Multicast Transport Protocol February 1992
every reasonable attempt advise the master of their intentions to
withdraw, as their departure may collapse the topology of the web and
eliminate the need to carry multicast messages across network
boundaries.
3.3.1. Voluntary quits
Voluntary quit[requests] are unicast to the master's TSAP. When the
master receives a quit from a member of the web, it responds with a
quit[confirm] packet. At that time the member will be formally
removed from the web. The request should be retransmitted at
heartbeat intervals until the confirmation is received from the
master or as many times as the web's value of retention.
3.3.2. Master quit
If the master initiates the transport termination it effects all
members of the web. The master will retain all transmit tokens and
refuse to assign them. Once the tokens are acquired, the master will
multicast a quit[request] to the entire web. That request should be
acknowledged by every active member. When the master receives no
confirmations for retention transmissions, it may assume every member
has terminated its transport and then may follow suit.
3.3.3. Banishment
If the master receives any message other than a join[request] from a
member that it does not recognize, it should transmit a quit[request]
with that process as a target. This covers cases where the consumer
did not see the termination reply and retransmitted its original quit
request, as well as unannounced and rejected consumers.
3.4 Transport parameters
The following section provides guidelines and rationale for selecting
reasonable transport quality of service parameters. It also describes
some of the reasoning behind the ranges of values presented.
3.4.1. Quality of service
Active members of the web may suggest changes in the transport's
quality of service parameters during the lifetime of the transport.
Producers in general adjust the transport's parameters to encourage a
higher level of throughput. Since consumers are responsible for
certifying reliable delivery, it is expected that they will provide
the force encouraging more reliability and stability. Both are trying
to optimize the quality of service. The negotiation that took place
when members joined the web included the clients' desires with
Armstrong, Freier & Marzullo [Page 30]
RFC 1301 Multicast Transport Protocol February 1992
regards to the worst case behavior that will be tolerated. If a
member cannot maintain the negotiated lower bound, it may asked to
withdraw from the web. That process will be sent a unicast message
(quit[request]) indicating that it should retire. There are
essentially three parameters maintained by the transport that reflect
the client's quality of service requirements: heartbeat, window and
retention. These three parameters can be adapted by the transport to
reflect the capability of the members, the type of application being
supported and the network topology. When members join the web, they
suggest values for the quality of service parameters to the master.
If the parameters are acceptable, the master will respond with the
web's current operating values. During the lifetime of the web, it is
expected that the parameters be modified by its members, though they
may never result in a quality of service less than the lower bounds
established by the joining procedure. Producers may try to improve
performance by reducing the heartbeat interval and increasing the
window size. This will have the effect of increasing the resources
committed to the transport at any time. In order to keep the
resources under control, the producer may also reduce the retention.
Consumers must rely on their clients to consume the data occupying
the resources of the transport. To do so the consumer transport
implementation must monitor the level of committed resources to
insure that it does not exceed its capabilities. Since MTP is a NAK
based protocol, the consumer is required to tell the producer if a
change in parameters is required. The new information must be
delivered to the producer(s) before the consumer's resource situation
becomes critical in order to avoid missing data.
For more stable operation, consumers would try to extend the
heartbeat interval and reduce the window. To a certain degree, they
could also attempt to reduce the value of retention in order to
reduce the amount of resources required to support the transport.
However, that requires a more stringent real-time capability.
3.4.2. Selecting parameter values
The value of heartbeat is approximately the transport time constant.
Assuming that the transport can be modelled as a closed loop system
function, reaction to feedback into the transport should settle out
in three time constants. In a transport that is constrained to a
single network, the dominant cause of processing delay of the
transport will most likely be page fault resolution time.
For example, using a one MIP processor on a ethernet and an industry
standard disk, the worst case page fault resolution requiring two
seeks (one to write out a dirty page, another to swap in the new
page) and an average seek time of 40 milliseconds, page fault
Armstrong, Freier & Marzullo [Page 31]
RFC 1301 Multicast Transport Protocol February 1992
resolution should be less than 80 milliseconds. Allowing for some
additional overhead and scheduling delays, two times the worst case
page fault resolution time would appear to be the minimum suitable
transport time constant one could expect. So,
Heartbeat (minimum) = 160 - 200 milliseconds.
The transmit time for a full (ethernet) packet is approximately 1.2
milliseconds. Processing time should be less than 3 milliseconds
(ignoring possible overlapped processing). Assuming disk access (with
no faulting) is equivalent, and the total time per packet is the sum
of the parts, or 8.4 milliseconds. Therefore, the theoretical maximum
value would be approximately 17 packets per heartbeat. The transport
should be capable of approximately 120 packets per second, or 19.2
packets per heartbeat.
Window (maximum) = 17 - 20 packets per heartbeat.
The (theoretical) throughput with these parameters in effect is 180
kilobytes per second.
Reducing retention may introduce instability because the consumers
will have less opportunity to react to missing data. Data can be
missed for a variety of reasons. If constrained to the local net the
data lost due to data link corruption should be in the neighborhood
of one packet in every 50,000 (bit error rate of approximately 10-9).
Telephony links (between routers, for instance) exhibit similar
characteristics. Several orders of magnitude more packets are lost at
receiving processes, including packet switch routers, than over the
physical links. The losses are usually a result of congestion and
resource starvation at lower layers due to the processing of (nearly)
back to back packets. The incidental packet loss of this type is
virtually unavoidable. One can only require that a receiving process
be capable of receiving some number of back to back packets
successfully, and that number must be at least greater then the value
of window. And beyond that the probability of success can be made as
close to unity as required by providing the receiver the opportunity
to observe the data multiple times.
The receiving process must detect packet loss. The simplest method is
to notice gaps in the received message/packet sequence numbers. Such
detection should be done after receiving an end of window or other
state transition indication. As such, the naks cannot be transmitted,
let alone received, until the following heartbeat. In order to not
have any single packet loss cause transport failure, the naks should
have the opportunity to be transmitted at least twice.
When the loss is detected, the nak must be transmitted and should be
Armstrong, Freier & Marzullo [Page 32]
RFC 1301 Multicast Transport Protocol February 1992
received at the producing process in less than two heartbeats after
the data it references was transmitted. Again, it is the detection
time that dominates, not the transmission of the nak.
Retention (minimum) = 3.
The resources committed to a producing transport using the above
assumptions are buffers sufficient for 80 packets of 1500 bytes each.
Each buffer will be committed for 600 - 800 milliseconds.
Transports that span multiple networks have unique problems. One such
problem is that if a router drops a packet, all the processes on the
remote network may attempt to send a nak[request] at the same time.
That is not likely to enhance the router's quality of service.
Furthermore, it is obvious that any one nak[request] will suffice to
prompt the producer to retransmit the desired packet. To reduce the
number of nak[requests] in this situation, the following scheme might
be employed.
First, extend the value of retention to a minimum value of N. Then
use a randomizing function that returns a value between zero and N -
2, choose how many heartbeat intervals to dally before sending the
nak[request], thus spreading out the transmissions over time. In
order for the method to be meaningful, the minimum value of retention
must be adjusted.
Retention (minimum) = 5 (for internet cases)
3.4.3. Caching member information
In order to reduce transport member interaction and to enhance
performance, a certain amount of caching should be employed by
producing members. These caches may be filled by gleaning information
from reliable sources such as multicast data or, when all else fails,
from responses solicited from the web's master by use of the
isMember[request]. IsMember[request] requests are unicast to a member
that is believed to have an accurate state of the web, at least to
the degree that it can answer the question posed. The destination of
such a message is usually the master. But in cases where a process
(such as the master) wants to verify that a process believes itself
to be valid, it can assign the target TSAP and the destination to be
the same. It is assumed that every process can verify itself.
If the member receiving the isMember[request] can confirm the
target's active membership status in the web, it responds with a
unicast isMember[confirm]. The data field contains the credibility
value of the confirmation, that is the time (in milliseconds) since
the information was confirmed from a reliable source.
Armstrong, Freier & Marzullo [Page 33]
RFC 1301 Multicast Transport Protocol February 1992
Caches are risky as the information stored in them can become stale.
Consequently, with only a few exceptions, the entries should be aged,
and when sufficiently old, discarded. Ideally they may be renewed by
the same gleanable sources alluded to in the previous paragraph. If
not, they are simply discarded and refilled when needed.
Web membership may be gleaned from any packet that does not have a
value of unknown as the destination connection identifier. A
producing transport may extract the TSAP from such packets and either
create or refresh local caches. Then, if in the process of
transmitting and NAK is received from one of the members whose
identity is cached, no explicit request will be needed to verify the
source's membership.
The explicit source of membership information is the master.
Information can be requested by using the isMember message.
Information gathered in that manner should be treated the same as
gleaned information with respect to aging.
The aging is a function of the transport's time constant, or
heartbeat, and the retention. Information about a producing member
must be cached at least as long as that producer has incomplete
messages. It may be cached longer. The namespace for both sequence
numbers and connection identifiers is intentionally long to insure
that reuse of those namespaces will not likely collide.
A. Appendix: MTP as an Internet Protocol transport
MTP is a transport layer protocol, designed to be layered on top of a
number of different network layer protocols. Such a protocol must
provide certain facilities that MTP expects. In particular, the
underlying network level protocol must provide "ports" or "sockets"
to facilitate addressing of processes within a machine, and a
mechanism for multicast addressing of datagrams. These two
addressing facilities are also used to formulate the NSAP for MTP on
IP.
A.1 Internet Protocol multicast addressing
MTP on Internet Protocol uses the Internet Protocol multicast
mechanisms defined in RFC 1112, "Host Extensions for IP
Multicasting". MTP requires "Level 2" conformance described in that
paper, for hosts which need to both send and receive multicast
packets, both on the local net and on an internet. MTP on Internet
Protocol uses the permanent host group address 224.0.1.9.
Armstrong, Freier & Marzullo [Page 34]
RFC 1301 Multicast Transport Protocol February 1992
A.2 Encapsulation
The Internet Protocol does not provide a port mechanism - ports are
defined at the transport level instead. In order to encapsulate MTP
packet within Internet Protocol packets, a simple convergence or
"bridge" protocol must be defined to run on top of Internet Protocol,
which will provide MTP with the mechanism needed to deliver packets
to the proper processes. We will call this protocol the
"MTP/Internet Protocol Bridge Protocol", or just "Bridge". The
protocol header is encapsulated the Internet Protocol data - the
protocol field of the Internet Protocol packet carries the value
indicating this packet is an MTP packet (92 decimal). The MTP packet
itself is encapsulated in the Bridge data. Figure A.1 shows the
positions of the fields within the MTP packet while table A.1 defines
the contents of those fields.
A.3 Fields of the bridge protocol
0 7 8 15 16 23 24 31
----------------------------------------------------------
| | |
| destination port | source port |
----------------------------------------------------------
| | |
| length | checksum |
----------------------------------------------------------
| |
| client data |
----------------------------------------------------------
Figure A.1 MTP bridge protocol header fields
destination port The port to which the packet is destined or sinked.
source port The port from which the packet originates or is sourced.
length The length in octets of the bridged packet, including
header and all data (the MTP packet). The minimum value
in this field is 8, the maximum is 65535. The length
does not include any padding bytes that were used to
compute the checksum. Note that though this field allows
for very long packets, most networks have significantly
shorter maximum frame sizes - the allowable and optimal
packet size must be determined by means beyond the scope
of this specification.
checksum The 16 bit one's compliment of the one's compliment sum
of the entire bridge protocol header and data, padded
Armstrong, Freier & Marzullo [Page 35]
RFC 1301 Multicast Transport Protocol February 1992
with a zero octet (if necessary) to make multiple 16 bit
quanities. A computed checksum of all zeros should be
changed to all ones. The checksum field is optional -
all zeros in the field indicate that checksums are not in
use.
data The data field is the field that carries the actual
transport data. A single MTP packet will be carried the
data field of each bridge packet.
A.4 Relationship to other Internet Protocol Transports
The astute reader might note that the MTP/Bridge Protocol looks much
like the User Datagram Protocol (UDP). UDP itself was not used
because the protocol field in the Internet Protocol packet should
reflect the fact that the higher level protocol of interest is MTP.
References
AFM91 Armstrong, S., A. Freier and K. Marzullo, "MTP: An Atomic
Multicast Transport Protocol", Xerox Webster Research Center
technical report X9100359, March 1991.
Bog83 Boggs, D., "Internet Broadcasting", Xerox PARC technical
report CSL-83-3, October 1983.
BSTM79 Boggs, D., J. Shoch, E. Taft, and R. Metcalfe, "Pup: An
Internetwork Architecture", IEEE Transactions on
Communications, COM-28(4), pages 612-624. April 1980.
DIX82 Digital Equipment Corp., Intel Corp., Xerox Corp., "The
Ethernet, a Local Area Network: Data Link and Physical Layer
Specifications", September 1982.
CLZ87 Clark, D., M. Lambert, and L. Zhang, "NETBLT: A high
throughput transport protocol", In Proceedings of ACM SIGCOMM
'87 Workshop, pages 353-359, 1987.
CM87 Chang J., and M. Maxemchuck. "Atomic broadcast", ACM
Transactions on Computer Systems, 2(3):251-273, August 1987.
Cri88 Cristian, F., "Reaching agreement on processor group
membership in synchronous distributed systems", In
Proceedings of the 18th International Conference on Fault-
Tolerant Computing. IEEE TOCS, 1988.
Dee89 Deering, S., "Host Extensions for IP Multicasting", RFC 1112,
Stanford University, August 1989.
Armstrong, Freier & Marzullo [Page 36]
RFC 1301 Multicast Transport Protocol February 1992
Fre84 Freier, A., "Compatability and interoperability", Open letter
to XNS Interest Group, Xerox Systems Developement Division,
December 13, 1984.
JB89 Joseph T., and K. Birman, "Reliable Broadcast Protocols",
pages 294-318, ACM Press, New York, 1989.
Pos81 Postel, J., "Transmission Control Protocol - DARPA Internet
Program Protocol Specification", RFC 793, DARPA, September
1981.
Xer81 Xerox Corp., "Internet Transport Protocols", Xerox System
Integration Standard 028112, Stamford, Connecticut. December
1981.
Footnotes
[1] The network layer is not specified by MTP. One of the goals is to
specify a transport that can be implemented with equal functionality
on many network architectures.
[2] There's only one such multicast connection identifier per web. If
there are multiple processes on the same machine participating in a
web, the transport must descriminate between those processes by using
the connnection identifier.
[3] Determining the network service access point (NSAP) for a given
instantiation of a web is not addressed by this protocol. This
document may define some policy, but the actual means are left for
other mechanisms.
[4] Best effort delivery is also known as highly reliable delivery.
It is somewhat unique that the qualifying adjective highly weakens
the definition of reliable in this context.
[5] The resource being flow controlled is packets carrying client
data. Consequently, full data units provide the greatest efficiency.
[6] There seems to be an opportunity to suppress retransmissions to
networks that were not represented in the set of naks received.
Security Considerations
Security issues are not discussed in this memo.
Armstrong, Freier & Marzullo [Page 37]
RFC 1301 Multicast Transport Protocol February 1992
Authors' Addresses
Susan M. Armstrong
Xerox Webster Research Center
800 Phillips Rd. MS 128-27E
Webster, NY 14580
Phone: (716) 422-6437
EMail: armstrong@wrc.xerox.com
Alan O. Freier
Apple Computer, Inc.
20525 Mariani Ave. MS 3-PK
Cupertino, CA 95014
Phone: (408) 974-9196
EMail: freier@apple.com
Keith A. Marzullo
Cornell University
Department of Computer Science
Upson Hall
Ithaca, NY 14853-7501
Phone: (607) 255-9188
EMail: marzullo@cs.cornell.edu
Keith Marzullo is supported in part by the Defense Advanced
Research Projects Agency (DoD) under NASA Ames grant number NAG
2-593, Contract N00140-87-C-8904. The views, opinions and
findings contained in this report are those of the authors and
should not be construed as an official Department of Defense
position, policy, or decision.
Armstrong, Freier & Marzullo [Page 38]
|