1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 2886 2887 2888 2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 2929 2930 2931 2932 2933 2934 2935 2936 2937 2938 2939 2940 2941 2942 2943 2944 2945 2946 2947 2948 2949 2950 2951 2952 2953 2954 2955 2956 2957 2958 2959 2960 2961 2962 2963 2964 2965 2966 2967 2968 2969 2970 2971 2972 2973 2974 2975 2976 2977 2978 2979 2980 2981 2982 2983 2984 2985 2986 2987 2988 2989 2990 2991 2992 2993 2994 2995 2996 2997 2998 2999 3000 3001 3002 3003 3004 3005 3006 3007 3008 3009 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 3020 3021 3022 3023 3024 3025 3026 3027 3028 3029 3030 3031 3032 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 3052 3053 3054 3055 3056 3057 3058 3059 3060 3061 3062 3063 3064 3065 3066 3067 3068 3069 3070 3071 3072 3073 3074 3075 3076 3077 3078 3079 3080 3081 3082 3083 3084 3085 3086 3087 3088 3089 3090 3091 3092 3093 3094 3095 3096 3097 3098 3099 3100 3101 3102 3103 3104 3105 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 3117 3118 3119 3120 3121 3122 3123 3124 3125 3126 3127 3128 3129 3130 3131 3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 3143 3144 3145 3146 3147 3148 3149 3150 3151 3152 3153 3154 3155 3156 3157 3158 3159 3160 3161 3162 3163 3164 3165 3166 3167 3168 3169 3170 3171 3172 3173 3174 3175 3176 3177 3178 3179 3180 3181 3182 3183 3184 3185 3186 3187 3188 3189 3190 3191 3192 3193 3194 3195 3196 3197 3198 3199 3200 3201 3202 3203 3204 3205 3206 3207 3208 3209 3210 3211 3212 3213 3214 3215 3216 3217 3218 3219 3220 3221 3222 3223 3224 3225 3226 3227 3228 3229 3230 3231 3232 3233 3234 3235 3236 3237 3238 3239 3240 3241 3242 3243 3244 3245 3246 3247 3248 3249 3250 3251 3252 3253 3254 3255 3256 3257 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 3273 3274 3275 3276 3277 3278 3279 3280 3281 3282 3283 3284 3285 3286 3287 3288 3289 3290 3291 3292 3293 3294 3295 3296 3297 3298 3299 3300 3301 3302 3303 3304 3305 3306 3307 3308 3309 3310 3311 3312 3313 3314 3315 3316 3317 3318 3319 3320 3321 3322 3323 3324 3325 3326 3327 3328 3329 3330 3331 3332 3333 3334 3335 3336 3337 3338 3339 3340 3341 3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 3355 3356 3357 3358 3359 3360 3361 3362 3363 3364 3365 3366 3367 3368 3369 3370 3371 3372 3373 3374 3375 3376 3377 3378 3379 3380 3381 3382 3383 3384 3385 3386 3387 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 3407 3408 3409 3410 3411 3412 3413 3414 3415 3416 3417 3418 3419 3420 3421 3422 3423 3424 3425 3426 3427 3428 3429 3430 3431 3432 3433 3434 3435 3436 3437 3438 3439 3440 3441 3442 3443 3444 3445 3446 3447 3448 3449 3450 3451 3452 3453 3454 3455 3456 3457 3458 3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 3470 3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 3489 3490 3491 3492 3493 3494 3495 3496 3497 3498 3499 3500 3501 3502 3503 3504 3505 3506 3507 3508 3509 3510 3511 3512 3513 3514 3515 3516 3517 3518 3519 3520 3521 3522 3523 3524 3525 3526 3527 3528 3529 3530 3531 3532 3533 3534 3535 3536 3537 3538 3539 3540 3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 3552 3553 3554 3555 3556 3557 3558 3559 3560 3561 3562 3563 3564 3565 3566 3567 3568 3569 3570 3571 3572 3573 3574 3575 3576 3577 3578 3579 3580 3581 3582 3583 3584 3585 3586 3587 3588 3589 3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 3637 3638 3639 3640 3641 3642 3643 3644 3645 3646 3647 3648 3649 3650 3651 3652 3653 3654
|
<!DOCTYPE html>
<html lang="en" class="RFC">
<head>
<meta charset="utf-8">
<meta content="Common,Latin" name="scripts">
<meta content="initial-scale=1.0" name="viewport">
<title>RFC 9330: Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: Architecture</title>
<meta content="Bob Briscoe" name="author">
<meta content="Koen De Schepper" name="author">
<meta content="Marcelo Bagnulo" name="author">
<meta content="Greg White" name="author">
<meta content="
This document describes the L4S architecture, which enables Internet
applications to achieve low queuing latency, low congestion loss, and scalable
throughput control. L4S is based on the insight that the root cause of
queuing delay is in the capacity-seeking congestion controllers of
senders, not in the queue itself. With the L4S architecture, all Internet
applications could (but do not have to) transition away from congestion
control algorithms that cause substantial queuing delay and instead adopt a new class
of congestion controls that can seek capacity with very little queuing.
These are aided by a modified form of Explicit Congestion Notification
(ECN) from the network. With this new architecture, applications can
have both low latency and high throughput.
The architecture primarily concerns incremental deployment. It
defines mechanisms that allow the new class of L4S congestion controls
to coexist with 'Classic' congestion controls in a shared network. The
aim is for L4S latency and throughput to be usually much better (and
rarely worse) while typically not impacting Classic performance.
" name="description">
<meta content="xml2rfc 3.15.3" name="generator">
<meta content="Performance" name="keyword">
<meta content="Queuing Delay" name="keyword">
<meta content="One Way Delay" name="keyword">
<meta content="Round-Trip Time" name="keyword">
<meta content="RTT" name="keyword">
<meta content="Jitter" name="keyword">
<meta content="Congestion Control" name="keyword">
<meta content="Congestion Avoidance" name="keyword">
<meta content="Quality of Service" name="keyword">
<meta content="QoS" name="keyword">
<meta content="Quality of Experience" name="keyword">
<meta content="QoE" name="keyword">
<meta content="Active Queue Management" name="keyword">
<meta content="AQM" name="keyword">
<meta content="Explicit Congestion Notification" name="keyword">
<meta content="ECN" name="keyword">
<meta content="Pacing" name="keyword">
<meta content="Burstiness" name="keyword">
<meta content="9330" name="rfc.number">
<!-- Generator version information:
xml2rfc 3.15.3
Python 3.9.15
appdirs 1.4.4
ConfigArgParse 1.5.3
google-i18n-address 2.5.1
html5lib 1.1
intervaltree 3.1.0
Jinja2 3.1.2
lxml 4.9.0
MarkupSafe 2.1.1
pycountry 22.3.5
PyYAML 6.0
requests 2.28.0
setuptools 44.1.1
six 1.16.0
wcwidth 0.2.5
weasyprint 56.1
-->
<link href="rfc9330.xml" rel="alternate" type="application/rfc+xml">
<link href="#copyright" rel="license">
<style type="text/css">/*
NOTE: Changes at the bottom of this file overrides some earlier settings.
Once the style has stabilized and has been adopted as an official RFC style,
this can be consolidated so that style settings occur only in one place, but
for now the contents of this file consists first of the initial CSS work as
provided to the RFC Formatter (xml2rfc) work, followed by itemized and
commented changes found necessary during the development of the v3
formatters.
*/
/* fonts */
@import url('https://fonts.googleapis.com/css?family=Noto+Sans'); /* Sans-serif */
@import url('https://fonts.googleapis.com/css?family=Noto+Serif'); /* Serif (print) */
@import url('https://fonts.googleapis.com/css?family=Roboto+Mono'); /* Monospace */
:root {
--font-sans: 'Noto Sans', Arial, Helvetica, sans-serif;
--font-serif: 'Noto Serif', 'Times', 'Times New Roman', serif;
--font-mono: 'Roboto Mono', Courier, 'Courier New', monospace;
}
@viewport {
zoom: 1.0;
width: extend-to-zoom;
}
@-ms-viewport {
width: extend-to-zoom;
zoom: 1.0;
}
/* general and mobile first */
html {
}
body {
max-width: 90%;
margin: 1.5em auto;
color: #222;
background-color: #fff;
font-size: 14px;
font-family: var(--font-sans);
line-height: 1.6;
scroll-behavior: smooth;
}
.ears {
display: none;
}
/* headings */
#title, h1, h2, h3, h4, h5, h6 {
margin: 1em 0 0.5em;
font-weight: bold;
line-height: 1.3;
}
#title {
clear: both;
border-bottom: 1px solid #ddd;
margin: 0 0 0.5em 0;
padding: 1em 0 0.5em;
}
.author {
padding-bottom: 4px;
}
h1 {
font-size: 26px;
margin: 1em 0;
}
h2 {
font-size: 22px;
margin-top: -20px; /* provide offset for in-page anchors */
padding-top: 33px;
}
h3 {
font-size: 18px;
margin-top: -36px; /* provide offset for in-page anchors */
padding-top: 42px;
}
h4 {
font-size: 16px;
margin-top: -36px; /* provide offset for in-page anchors */
padding-top: 42px;
}
h5, h6 {
font-size: 14px;
}
#n-copyright-notice {
border-bottom: 1px solid #ddd;
padding-bottom: 1em;
margin-bottom: 1em;
}
/* general structure */
p {
padding: 0;
margin: 0 0 1em 0;
text-align: left;
}
div, span {
position: relative;
}
div {
margin: 0;
}
.alignRight.art-text {
background-color: #f9f9f9;
border: 1px solid #eee;
border-radius: 3px;
padding: 1em 1em 0;
margin-bottom: 1.5em;
}
.alignRight.art-text pre {
padding: 0;
}
.alignRight {
margin: 1em 0;
}
.alignRight > *:first-child {
border: none;
margin: 0;
float: right;
clear: both;
}
.alignRight > *:nth-child(2) {
clear: both;
display: block;
border: none;
}
svg {
display: block;
}
svg[font-family~="serif" i], svg [font-family~="serif" i] {
font-family: var(--font-serif);
}
svg[font-family~="sans-serif" i], svg [font-family~="sans-serif" i] {
font-family: var(--font-sans);
}
svg[font-family~="monospace" i], svg [font-family~="monospace" i] {
font-family: var(--font-mono);
}
.alignCenter.art-text {
background-color: #f9f9f9;
border: 1px solid #eee;
border-radius: 3px;
padding: 1em 1em 0;
margin-bottom: 1.5em;
}
.alignCenter.art-text pre {
padding: 0;
}
.alignCenter {
margin: 1em 0;
}
.alignCenter > *:first-child {
display: table;
border: none;
margin: 0 auto;
}
/* lists */
ol, ul {
padding: 0;
margin: 0 0 1em 2em;
}
ol ol, ul ul, ol ul, ul ol {
margin-left: 1em;
}
li {
margin: 0 0 0.25em 0;
}
.ulCompact li {
margin: 0;
}
ul.empty, .ulEmpty {
list-style-type: none;
}
ul.empty li, .ulEmpty li {
margin-top: 0.5em;
}
ul.ulBare, li.ulBare {
margin-left: 0em !important;
}
ul.compact, .ulCompact,
ol.compact, .olCompact {
line-height: 100%;
margin: 0 0 0 2em;
}
/* definition lists */
dl {
}
dl > dt {
float: left;
margin-right: 1em;
}
/*
dl.nohang > dt {
float: none;
}
*/
dl > dd {
margin-bottom: .8em;
min-height: 1.3em;
}
dl.compact > dd, .dlCompact > dd {
margin-bottom: 0em;
}
dl > dd > dl {
margin-top: 0.5em;
margin-bottom: 0em;
}
/* links */
a {
text-decoration: none;
}
a[href] {
color: #22e; /* Arlen: WCAG 2019 */
}
a[href]:hover {
background-color: #f2f2f2;
}
figcaption a[href],
a[href].selfRef {
color: #222;
}
/* XXX probably not this:
a.selfRef:hover {
background-color: transparent;
cursor: default;
} */
/* Figures */
tt, code, pre {
background-color: #f9f9f9;
font-family: var(--font-mono);
}
pre {
border: 1px solid #eee;
margin: 0;
padding: 1em;
}
img {
max-width: 100%;
}
figure {
margin: 0;
}
figure blockquote {
margin: 0.8em 0.4em 0.4em;
}
figcaption {
font-style: italic;
margin: 0 0 1em 0;
}
@media screen {
pre {
overflow-x: auto;
max-width: 100%;
max-width: calc(100% - 22px);
}
}
/* aside, blockquote */
aside, blockquote {
margin-left: 0;
padding: 1.2em 2em;
}
blockquote {
background-color: #f9f9f9;
color: #111; /* Arlen: WCAG 2019 */
border: 1px solid #ddd;
border-radius: 3px;
margin: 1em 0;
}
cite {
display: block;
text-align: right;
font-style: italic;
}
/* tables */
table {
width: 100%;
margin: 0 0 1em;
border-collapse: collapse;
border: 1px solid #eee;
}
th, td {
text-align: left;
vertical-align: top;
padding: 0.5em 0.75em;
}
th {
text-align: left;
background-color: #e9e9e9;
}
tr:nth-child(2n+1) > td {
background-color: #f5f5f5;
}
table caption {
font-style: italic;
margin: 0;
padding: 0;
text-align: left;
}
table p {
/* XXX to avoid bottom margin on table row signifiers. If paragraphs should
be allowed within tables more generally, it would be far better to select on a class. */
margin: 0;
}
/* pilcrow */
a.pilcrow {
color: #666; /* Arlen: AHDJ 2019 */
text-decoration: none;
visibility: hidden;
user-select: none;
-ms-user-select: none;
-o-user-select:none;
-moz-user-select: none;
-khtml-user-select: none;
-webkit-user-select: none;
-webkit-touch-callout: none;
}
@media screen {
aside:hover > a.pilcrow,
p:hover > a.pilcrow,
blockquote:hover > a.pilcrow,
div:hover > a.pilcrow,
li:hover > a.pilcrow,
pre:hover > a.pilcrow {
visibility: visible;
}
a.pilcrow:hover {
background-color: transparent;
}
}
/* misc */
hr {
border: 0;
border-top: 1px solid #eee;
}
.bcp14 {
font-variant: small-caps;
}
.role {
font-variant: all-small-caps;
}
/* info block */
#identifiers {
margin: 0;
font-size: 0.9em;
}
#identifiers dt {
width: 3em;
clear: left;
}
#identifiers dd {
float: left;
margin-bottom: 0;
}
/* Fix PDF info block run off issue */
@media print {
#identifiers dd {
float: none;
}
}
#identifiers .authors .author {
display: inline-block;
margin-right: 1.5em;
}
#identifiers .authors .org {
font-style: italic;
}
/* The prepared/rendered info at the very bottom of the page */
.docInfo {
color: #666; /* Arlen: WCAG 2019 */
font-size: 0.9em;
font-style: italic;
margin-top: 2em;
}
.docInfo .prepared {
float: left;
}
.docInfo .prepared {
float: right;
}
/* table of contents */
#toc {
padding: 0.75em 0 2em 0;
margin-bottom: 1em;
}
nav.toc ul {
margin: 0 0.5em 0 0;
padding: 0;
list-style: none;
}
nav.toc li {
line-height: 1.3em;
margin: 0.75em 0;
padding-left: 1.2em;
text-indent: -1.2em;
}
/* references */
.references dt {
text-align: right;
font-weight: bold;
min-width: 7em;
}
.references dd {
margin-left: 8em;
overflow: auto;
}
.refInstance {
margin-bottom: 1.25em;
}
.references .ascii {
margin-bottom: 0.25em;
}
/* index */
.index ul {
margin: 0 0 0 1em;
padding: 0;
list-style: none;
}
.index ul ul {
margin: 0;
}
.index li {
margin: 0;
text-indent: -2em;
padding-left: 2em;
padding-bottom: 5px;
}
.indexIndex {
margin: 0.5em 0 1em;
}
.index a {
font-weight: 700;
}
/* make the index two-column on all but the smallest screens */
@media (min-width: 600px) {
.index ul {
-moz-column-count: 2;
-moz-column-gap: 20px;
}
.index ul ul {
-moz-column-count: 1;
-moz-column-gap: 0;
}
}
/* authors */
address.vcard {
font-style: normal;
margin: 1em 0;
}
address.vcard .nameRole {
font-weight: 700;
margin-left: 0;
}
address.vcard .label {
font-family: var(--font-sans);
margin: 0.5em 0;
}
address.vcard .type {
display: none;
}
.alternative-contact {
margin: 1.5em 0 1em;
}
hr.addr {
border-top: 1px dashed;
margin: 0;
color: #ddd;
max-width: calc(100% - 16px);
}
/* temporary notes */
.rfcEditorRemove::before {
position: absolute;
top: 0.2em;
right: 0.2em;
padding: 0.2em;
content: "The RFC Editor will remove this note";
color: #9e2a00; /* Arlen: WCAG 2019 */
background-color: #ffd; /* Arlen: WCAG 2019 */
}
.rfcEditorRemove {
position: relative;
padding-top: 1.8em;
background-color: #ffd; /* Arlen: WCAG 2019 */
border-radius: 3px;
}
.cref {
background-color: #ffd; /* Arlen: WCAG 2019 */
padding: 2px 4px;
}
.crefSource {
font-style: italic;
}
/* alternative layout for smaller screens */
@media screen and (max-width: 1023px) {
body {
padding-top: 2em;
}
#title {
padding: 1em 0;
}
h1 {
font-size: 24px;
}
h2 {
font-size: 20px;
margin-top: -18px; /* provide offset for in-page anchors */
padding-top: 38px;
}
#identifiers dd {
max-width: 60%;
}
#toc {
position: fixed;
z-index: 2;
top: 0;
right: 0;
padding: 0;
margin: 0;
background-color: inherit;
border-bottom: 1px solid #ccc;
}
#toc h2 {
margin: -1px 0 0 0;
padding: 4px 0 4px 6px;
padding-right: 1em;
min-width: 190px;
font-size: 1.1em;
text-align: right;
background-color: #444;
color: white;
cursor: pointer;
}
#toc h2::before { /* css hamburger */
float: right;
position: relative;
width: 1em;
height: 1px;
left: -164px;
margin: 6px 0 0 0;
background: white none repeat scroll 0 0;
box-shadow: 0 4px 0 0 white, 0 8px 0 0 white;
content: "";
}
#toc nav {
display: none;
padding: 0.5em 1em 1em;
overflow: auto;
height: calc(100vh - 48px);
border-left: 1px solid #ddd;
}
}
/* alternative layout for wide screens */
@media screen and (min-width: 1024px) {
body {
max-width: 724px;
margin: 42px auto;
padding-left: 1.5em;
padding-right: 29em;
}
#toc {
position: fixed;
top: 42px;
right: 42px;
width: 25%;
margin: 0;
padding: 0 1em;
z-index: 1;
}
#toc h2 {
border-top: none;
border-bottom: 1px solid #ddd;
font-size: 1em;
font-weight: normal;
margin: 0;
padding: 0.25em 1em 1em 0;
}
#toc nav {
display: block;
height: calc(90vh - 84px);
bottom: 0;
padding: 0.5em 0 0;
overflow: auto;
}
img { /* future proofing */
max-width: 100%;
height: auto;
}
}
/* pagination */
@media print {
body {
width: 100%;
}
p {
orphans: 3;
widows: 3;
}
#n-copyright-notice {
border-bottom: none;
}
#toc, #n-introduction {
page-break-before: always;
}
#toc {
border-top: none;
padding-top: 0;
}
figure, pre {
page-break-inside: avoid;
}
figure {
overflow: scroll;
}
.breakable pre {
break-inside: auto;
}
h1, h2, h3, h4, h5, h6 {
page-break-after: avoid;
}
h2+*, h3+*, h4+*, h5+*, h6+* {
page-break-before: avoid;
}
pre {
white-space: pre-wrap;
word-wrap: break-word;
font-size: 10pt;
}
table {
border: 1px solid #ddd;
}
td {
border-top: 1px solid #ddd;
}
}
/* This is commented out here, as the string-set: doesn't
pass W3C validation currently */
/*
.ears thead .left {
string-set: ears-top-left content();
}
.ears thead .center {
string-set: ears-top-center content();
}
.ears thead .right {
string-set: ears-top-right content();
}
.ears tfoot .left {
string-set: ears-bottom-left content();
}
.ears tfoot .center {
string-set: ears-bottom-center content();
}
.ears tfoot .right {
string-set: ears-bottom-right content();
}
*/
@page :first {
padding-top: 0;
@top-left {
content: normal;
border: none;
}
@top-center {
content: normal;
border: none;
}
@top-right {
content: normal;
border: none;
}
}
@page {
size: A4;
margin-bottom: 45mm;
padding-top: 20px;
/* The following is commented out here, but set appropriately by in code, as
the content depends on the document */
/*
@top-left {
content: 'Internet-Draft';
vertical-align: bottom;
border-bottom: solid 1px #ccc;
}
@top-left {
content: string(ears-top-left);
vertical-align: bottom;
border-bottom: solid 1px #ccc;
}
@top-center {
content: string(ears-top-center);
vertical-align: bottom;
border-bottom: solid 1px #ccc;
}
@top-right {
content: string(ears-top-right);
vertical-align: bottom;
border-bottom: solid 1px #ccc;
}
@bottom-left {
content: string(ears-bottom-left);
vertical-align: top;
border-top: solid 1px #ccc;
}
@bottom-center {
content: string(ears-bottom-center);
vertical-align: top;
border-top: solid 1px #ccc;
}
@bottom-right {
content: '[Page ' counter(page) ']';
vertical-align: top;
border-top: solid 1px #ccc;
}
*/
}
/* Changes introduced to fix issues found during implementation */
/* Make sure links are clickable even if overlapped by following H* */
a {
z-index: 2;
}
/* Separate body from document info even without intervening H1 */
section {
clear: both;
}
/* Top align author divs, to avoid names without organization dropping level with org names */
.author {
vertical-align: top;
}
/* Leave room in document info to show Internet-Draft on one line */
#identifiers dt {
width: 8em;
}
/* Don't waste quite as much whitespace between label and value in doc info */
#identifiers dd {
margin-left: 1em;
}
/* Give floating toc a background color (needed when it's a div inside section */
#toc {
background-color: white;
}
/* Make the collapsed ToC header render white on gray also when it's a link */
@media screen and (max-width: 1023px) {
#toc h2 a,
#toc h2 a:link,
#toc h2 a:focus,
#toc h2 a:hover,
#toc a.toplink,
#toc a.toplink:hover {
color: white;
background-color: #444;
text-decoration: none;
}
}
/* Give the bottom of the ToC some whitespace */
@media screen and (min-width: 1024px) {
#toc {
padding: 0 0 1em 1em;
}
}
/* Style section numbers with more space between number and title */
.section-number {
padding-right: 0.5em;
}
/* prevent monospace from becoming overly large */
tt, code, pre {
font-size: 95%;
}
/* Fix the height/width aspect for ascii art*/
.sourcecode pre,
.art-text pre {
line-height: 1.12;
}
/* Add styling for a link in the ToC that points to the top of the document */
a.toplink {
float: right;
margin-right: 0.5em;
}
/* Fix the dl styling to match the RFC 7992 attributes */
dl > dt,
dl.dlParallel > dt {
float: left;
margin-right: 1em;
}
dl.dlNewline > dt {
float: none;
}
/* Provide styling for table cell text alignment */
table td.text-left,
table th.text-left {
text-align: left;
}
table td.text-center,
table th.text-center {
text-align: center;
}
table td.text-right,
table th.text-right {
text-align: right;
}
/* Make the alternative author contact information look less like just another
author, and group it closer with the primary author contact information */
.alternative-contact {
margin: 0.5em 0 0.25em 0;
}
address .non-ascii {
margin: 0 0 0 2em;
}
/* With it being possible to set tables with alignment
left, center, and right, { width: 100%; } does not make sense */
table {
width: auto;
}
/* Avoid reference text that sits in a block with very wide left margin,
because of a long floating dt label.*/
.references dd {
overflow: visible;
}
/* Control caption placement */
caption {
caption-side: bottom;
}
/* Limit the width of the author address vcard, so names in right-to-left
script don't end up on the other side of the page. */
address.vcard {
max-width: 30em;
margin-right: auto;
}
/* For address alignment dependent on LTR or RTL scripts */
address div.left {
text-align: left;
}
address div.right {
text-align: right;
}
/* Provide table alignment support. We can't use the alignX classes above
since they do unwanted things with caption and other styling. */
table.right {
margin-left: auto;
margin-right: 0;
}
table.center {
margin-left: auto;
margin-right: auto;
}
table.left {
margin-left: 0;
margin-right: auto;
}
/* Give the table caption label the same styling as the figcaption */
caption a[href] {
color: #222;
}
@media print {
.toplink {
display: none;
}
/* avoid overwriting the top border line with the ToC header */
#toc {
padding-top: 1px;
}
/* Avoid page breaks inside dl and author address entries */
.vcard {
page-break-inside: avoid;
}
}
/* Tweak the bcp14 keyword presentation */
.bcp14 {
font-variant: small-caps;
font-weight: bold;
font-size: 0.9em;
}
/* Tweak the invisible space above H* in order not to overlay links in text above */
h2 {
margin-top: -18px; /* provide offset for in-page anchors */
padding-top: 31px;
}
h3 {
margin-top: -18px; /* provide offset for in-page anchors */
padding-top: 24px;
}
h4 {
margin-top: -18px; /* provide offset for in-page anchors */
padding-top: 24px;
}
/* Float artwork pilcrow to the right */
@media screen {
.artwork a.pilcrow {
display: block;
line-height: 0.7;
margin-top: 0.15em;
}
}
/* Make pilcrows on dd visible */
@media screen {
dd:hover > a.pilcrow {
visibility: visible;
}
}
/* Make the placement of figcaption match that of a table's caption
by removing the figure's added bottom margin */
.alignLeft.art-text,
.alignCenter.art-text,
.alignRight.art-text {
margin-bottom: 0;
}
.alignLeft,
.alignCenter,
.alignRight {
margin: 1em 0 0 0;
}
/* In print, the pilcrow won't show on hover, so prevent it from taking up space,
possibly even requiring a new line */
@media print {
a.pilcrow {
display: none;
}
}
/* Styling for the external metadata */
div#external-metadata {
background-color: #eee;
padding: 0.5em;
margin-bottom: 0.5em;
display: none;
}
div#internal-metadata {
padding: 0.5em; /* to match the external-metadata padding */
}
/* Styling for title RFC Number */
h1#rfcnum {
clear: both;
margin: 0 0 -1em;
padding: 1em 0 0 0;
}
/* Make .olPercent look the same as <ol><li> */
dl.olPercent > dd {
margin-bottom: 0.25em;
min-height: initial;
}
/* Give aside some styling to set it apart */
aside {
border-left: 1px solid #ddd;
margin: 1em 0 1em 2em;
padding: 0.2em 2em;
}
aside > dl,
aside > ol,
aside > ul,
aside > table,
aside > p {
margin-bottom: 0.5em;
}
/* Additional page break settings */
@media print {
figcaption, table caption {
page-break-before: avoid;
}
}
/* Font size adjustments for print */
@media print {
body { font-size: 10pt; line-height: normal; max-width: 96%; }
h1 { font-size: 1.72em; padding-top: 1.5em; } /* 1*1.2*1.2*1.2 */
h2 { font-size: 1.44em; padding-top: 1.5em; } /* 1*1.2*1.2 */
h3 { font-size: 1.2em; padding-top: 1.5em; } /* 1*1.2 */
h4 { font-size: 1em; padding-top: 1.5em; }
h5, h6 { font-size: 1em; margin: initial; padding: 0.5em 0 0.3em; }
}
/* Sourcecode margin in print, when there's no pilcrow */
@media print {
.artwork,
.artwork > pre,
.sourcecode {
margin-bottom: 1em;
}
}
/* Avoid narrow tables forcing too narrow table captions, which may render badly */
table {
min-width: 20em;
}
/* ol type a */
ol.type-a { list-style-type: lower-alpha; }
ol.type-A { list-style-type: upper-alpha; }
ol.type-i { list-style-type: lower-roman; }
ol.type-I { list-style-type: lower-roman; }
/* Apply the print table and row borders in general, on request from the RPC,
and increase the contrast between border and odd row background slightly */
table {
border: 1px solid #ddd;
}
td {
border-top: 1px solid #ddd;
}
tr {
break-inside: avoid;
}
tr:nth-child(2n+1) > td {
background-color: #f8f8f8;
}
/* Use style rules to govern display of the TOC. */
@media screen and (max-width: 1023px) {
#toc nav { display: none; }
#toc.active nav { display: block; }
}
/* Add support for keepWithNext */
.keepWithNext {
break-after: avoid-page;
break-after: avoid-page;
}
/* Add support for keepWithPrevious */
.keepWithPrevious {
break-before: avoid-page;
}
/* Change the approach to avoiding breaks inside artwork etc. */
figure, pre, table, .artwork, .sourcecode {
break-before: auto;
break-after: auto;
}
/* Avoid breaks between <dt> and <dd> */
dl {
break-before: auto;
break-inside: auto;
}
dt {
break-before: auto;
break-after: avoid-page;
}
dd {
break-before: avoid-page;
break-after: auto;
orphans: 3;
widows: 3
}
span.break, dd.break {
margin-bottom: 0;
min-height: 0;
break-before: auto;
break-inside: auto;
break-after: auto;
}
/* Undo break-before ToC */
@media print {
#toc {
break-before: auto;
}
}
/* Text in compact lists should not get extra bottom margin space,
since that would makes the list not compact */
ul.compact p, .ulCompact p,
ol.compact p, .olCompact p {
margin: 0;
}
/* But the list as a whole needs the extra space at the end */
section ul.compact,
section .ulCompact,
section ol.compact,
section .olCompact {
margin-bottom: 1em; /* same as p not within ul.compact etc. */
}
/* The tt and code background above interferes with for instance table cell
backgrounds. Changed to something a bit more selective. */
tt, code {
background-color: transparent;
}
p tt, p code, li tt, li code {
background-color: #f8f8f8;
}
/* Tweak the pre margin -- 0px doesn't come out well */
pre {
margin-top: 0.5px;
}
/* Tweak the compact list text */
ul.compact, .ulCompact,
ol.compact, .olCompact,
dl.compact, .dlCompact {
line-height: normal;
}
/* Don't add top margin for nested lists */
li > ul, li > ol, li > dl,
dd > ul, dd > ol, dd > dl,
dl > dd > dl {
margin-top: initial;
}
/* Elements that should not be rendered on the same line as a <dt> */
/* This should match the element list in writer.text.TextWriter.render_dl() */
dd > div.artwork:first-child,
dd > aside:first-child,
dd > figure:first-child,
dd > ol:first-child,
dd > div.sourcecode:first-child,
dd > table:first-child,
dd > ul:first-child {
clear: left;
}
/* fix for weird browser behaviour when <dd/> is empty */
dt+dd:empty::before{
content: "\00a0";
}
/* Make paragraph spacing inside <li> smaller than in body text, to fit better within the list */
li > p {
margin-bottom: 0.5em
}
/* Don't let p margin spill out from inside list items */
li > p:last-of-type:only-child {
margin-bottom: 0;
}
</style>
<link href="rfc-local.css" rel="stylesheet" type="text/css">
<link href="https://dx.doi.org/10.17487/rfc9330" rel="alternate">
<link href="urn:issn:2070-1721" rel="alternate">
<link href="https://datatracker.ietf.org/doc/draft-ietf-tsvwg-l4s-arch-20" rel="prev">
</head>
<body class="xml2rfc">
<script src="https://www.rfc-editor.org/js/metadata.min.js"></script>
<table class="ears">
<thead><tr>
<td class="left">RFC 9330</td>
<td class="center">L4S Architecture</td>
<td class="right">January 2023</td>
</tr></thead>
<tfoot><tr>
<td class="left">Briscoe, et al.</td>
<td class="center">Informational</td>
<td class="right">[Page]</td>
</tr></tfoot>
</table>
<div id="external-metadata" class="document-information"></div>
<div id="internal-metadata" class="document-information">
<dl id="identifiers">
<dt class="label-stream">Stream:</dt>
<dd class="stream">Internet Engineering Task Force (IETF)</dd>
<dt class="label-rfc">RFC:</dt>
<dd class="rfc"><a href="https://www.rfc-editor.org/rfc/rfc9330" class="eref">9330</a></dd>
<dt class="label-category">Category:</dt>
<dd class="category">Informational</dd>
<dt class="label-published">Published:</dt>
<dd class="published">
<time datetime="2023-01" class="published">January 2023</time>
</dd>
<dt class="label-issn">ISSN:</dt>
<dd class="issn">2070-1721</dd>
<dt class="label-authors">Authors:</dt>
<dd class="authors">
<div class="author">
<div class="author-name">B. Briscoe, <span class="editor">Ed.</span>
</div>
<div class="org">Independent</div>
</div>
<div class="author">
<div class="author-name">K. De Schepper</div>
<div class="org">Nokia Bell Labs</div>
</div>
<div class="author">
<div class="author-name">M. Bagnulo</div>
<div class="org">Universidad Carlos III de Madrid</div>
</div>
<div class="author">
<div class="author-name">G. White</div>
<div class="org">CableLabs</div>
</div>
</dd>
</dl>
</div>
<h1 id="rfcnum">RFC 9330</h1>
<h1 id="title">Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: Architecture</h1>
<section id="section-abstract">
<h2 id="abstract"><a href="#abstract" class="selfRef">Abstract</a></h2>
<p id="section-abstract-1">This document describes the L4S architecture, which enables Internet
applications to achieve low queuing latency, low congestion loss, and scalable
throughput control. L4S is based on the insight that the root cause of
queuing delay is in the capacity-seeking congestion controllers of
senders, not in the queue itself. With the L4S architecture, all Internet
applications could (but do not have to) transition away from congestion
control algorithms that cause substantial queuing delay and instead adopt a new class
of congestion controls that can seek capacity with very little queuing.
These are aided by a modified form of Explicit Congestion Notification
(ECN) from the network. With this new architecture, applications can
have both low latency and high throughput.<a href="#section-abstract-1" class="pilcrow">¶</a></p>
<p id="section-abstract-2">The architecture primarily concerns incremental deployment. It
defines mechanisms that allow the new class of L4S congestion controls
to coexist with 'Classic' congestion controls in a shared network. The
aim is for L4S latency and throughput to be usually much better (and
rarely worse) while typically not impacting Classic performance.<a href="#section-abstract-2" class="pilcrow">¶</a></p>
</section>
<div id="status-of-memo">
<section id="section-boilerplate.1">
<h2 id="name-status-of-this-memo">
<a href="#name-status-of-this-memo" class="section-name selfRef">Status of This Memo</a>
</h2>
<p id="section-boilerplate.1-1">
This document is not an Internet Standards Track specification; it is
published for informational purposes.<a href="#section-boilerplate.1-1" class="pilcrow">¶</a></p>
<p id="section-boilerplate.1-2">
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are candidates for any level of Internet
Standard; see Section 2 of RFC 7841.<a href="#section-boilerplate.1-2" class="pilcrow">¶</a></p>
<p id="section-boilerplate.1-3">
Information about the current status of this document, any
errata, and how to provide feedback on it may be obtained at
<span><a href="https://www.rfc-editor.org/info/rfc9330">https://www.rfc-editor.org/info/rfc9330</a></span>.<a href="#section-boilerplate.1-3" class="pilcrow">¶</a></p>
</section>
</div>
<div id="copyright">
<section id="section-boilerplate.2">
<h2 id="name-copyright-notice">
<a href="#name-copyright-notice" class="section-name selfRef">Copyright Notice</a>
</h2>
<p id="section-boilerplate.2-1">
Copyright (c) 2023 IETF Trust and the persons identified as the
document authors. All rights reserved.<a href="#section-boilerplate.2-1" class="pilcrow">¶</a></p>
<p id="section-boilerplate.2-2">
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(<span><a href="https://trustee.ietf.org/license-info">https://trustee.ietf.org/license-info</a></span>) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
document must include Revised BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Revised BSD License.<a href="#section-boilerplate.2-2" class="pilcrow">¶</a></p>
</section>
</div>
<div id="toc">
<section id="section-toc.1">
<a href="#" onclick="scroll(0,0)" class="toplink">▲</a><h2 id="name-table-of-contents">
<a href="#name-table-of-contents" class="section-name selfRef">Table of Contents</a>
</h2>
<nav class="toc"><ul class="compact toc ulBare ulEmpty">
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.1">
<p id="section-toc.1-1.1.1" class="keepWithNext"><a href="#section-1" class="auto internal xref">1</a>. <a href="#name-introduction" class="internal xref">Introduction</a></p>
<ul class="compact toc ulBare ulEmpty">
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.1.2.1">
<p id="section-toc.1-1.1.2.1.1" class="keepWithNext"><a href="#section-1.1" class="auto internal xref">1.1</a>. <a href="#name-document-roadmap" class="internal xref">Document Roadmap</a></p>
</li>
</ul>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.2">
<p id="section-toc.1-1.2.1" class="keepWithNext"><a href="#section-2" class="auto internal xref">2</a>. <a href="#name-l4s-architecture-overview" class="internal xref">L4S Architecture Overview</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.3">
<p id="section-toc.1-1.3.1"><a href="#section-3" class="auto internal xref">3</a>. <a href="#name-terminology" class="internal xref">Terminology</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.4">
<p id="section-toc.1-1.4.1"><a href="#section-4" class="auto internal xref">4</a>. <a href="#name-l4s-architecture-components" class="internal xref">L4S Architecture Components</a></p>
<ul class="compact toc ulBare ulEmpty">
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.4.2.1">
<p id="section-toc.1-1.4.2.1.1"><a href="#section-4.1" class="auto internal xref">4.1</a>. <a href="#name-protocol-mechanisms" class="internal xref">Protocol Mechanisms</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.4.2.2">
<p id="section-toc.1-1.4.2.2.1"><a href="#section-4.2" class="auto internal xref">4.2</a>. <a href="#name-network-components" class="internal xref">Network Components</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.4.2.3">
<p id="section-toc.1-1.4.2.3.1"><a href="#section-4.3" class="auto internal xref">4.3</a>. <a href="#name-host-mechanisms" class="internal xref">Host Mechanisms</a></p>
</li>
</ul>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.5">
<p id="section-toc.1-1.5.1"><a href="#section-5" class="auto internal xref">5</a>. <a href="#name-rationale" class="internal xref">Rationale</a></p>
<ul class="compact toc ulBare ulEmpty">
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.5.2.1">
<p id="section-toc.1-1.5.2.1.1"><a href="#section-5.1" class="auto internal xref">5.1</a>. <a href="#name-why-these-primary-component" class="internal xref">Why These Primary Components?</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.5.2.2">
<p id="section-toc.1-1.5.2.2.1"><a href="#section-5.2" class="auto internal xref">5.2</a>. <a href="#name-what-l4s-adds-to-existing-a" class="internal xref">What L4S Adds to Existing Approaches</a></p>
</li>
</ul>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.6">
<p id="section-toc.1-1.6.1"><a href="#section-6" class="auto internal xref">6</a>. <a href="#name-applicability" class="internal xref">Applicability</a></p>
<ul class="compact toc ulBare ulEmpty">
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.6.2.1">
<p id="section-toc.1-1.6.2.1.1"><a href="#section-6.1" class="auto internal xref">6.1</a>. <a href="#name-applications" class="internal xref">Applications</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.6.2.2">
<p id="section-toc.1-1.6.2.2.1"><a href="#section-6.2" class="auto internal xref">6.2</a>. <a href="#name-use-cases" class="internal xref">Use Cases</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.6.2.3">
<p id="section-toc.1-1.6.2.3.1"><a href="#section-6.3" class="auto internal xref">6.3</a>. <a href="#name-applicability-with-specific" class="internal xref">Applicability with Specific Link Technologies</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.6.2.4">
<p id="section-toc.1-1.6.2.4.1"><a href="#section-6.4" class="auto internal xref">6.4</a>. <a href="#name-deployment-considerations" class="internal xref">Deployment Considerations</a></p>
<ul class="compact toc ulBare ulEmpty">
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.6.2.4.2.1">
<p id="section-toc.1-1.6.2.4.2.1.1"><a href="#section-6.4.1" class="auto internal xref">6.4.1</a>. <a href="#name-deployment-topology" class="internal xref">Deployment Topology</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.6.2.4.2.2">
<p id="section-toc.1-1.6.2.4.2.2.1"><a href="#section-6.4.2" class="auto internal xref">6.4.2</a>. <a href="#name-deployment-sequences" class="internal xref">Deployment Sequences</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.6.2.4.2.3">
<p id="section-toc.1-1.6.2.4.2.3.1"><a href="#section-6.4.3" class="auto internal xref">6.4.3</a>. <a href="#name-l4s-flow-but-non-ecn-bottle" class="internal xref">L4S Flow but Non-ECN Bottleneck</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.6.2.4.2.4">
<p id="section-toc.1-1.6.2.4.2.4.1"><a href="#section-6.4.4" class="auto internal xref">6.4.4</a>. <a href="#name-l4s-flow-but-classic-ecn-bo" class="internal xref">L4S Flow but Classic ECN Bottleneck</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.6.2.4.2.5">
<p id="section-toc.1-1.6.2.4.2.5.1"><a href="#section-6.4.5" class="auto internal xref">6.4.5</a>. <a href="#name-l4s-aqm-deployment-within-t" class="internal xref">L4S AQM Deployment within Tunnels</a></p>
</li>
</ul>
</li>
</ul>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.7">
<p id="section-toc.1-1.7.1"><a href="#section-7" class="auto internal xref">7</a>. <a href="#name-iana-considerations" class="internal xref">IANA Considerations</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.8">
<p id="section-toc.1-1.8.1"><a href="#section-8" class="auto internal xref">8</a>. <a href="#name-security-considerations" class="internal xref">Security Considerations</a></p>
<ul class="compact toc ulBare ulEmpty">
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.8.2.1">
<p id="section-toc.1-1.8.2.1.1"><a href="#section-8.1" class="auto internal xref">8.1</a>. <a href="#name-traffic-rate-non-policing" class="internal xref">Traffic Rate (Non-)Policing</a></p>
<ul class="compact toc ulBare ulEmpty">
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.8.2.1.2.1">
<p id="section-toc.1-1.8.2.1.2.1.1"><a href="#section-8.1.1" class="auto internal xref">8.1.1</a>. <a href="#name-non-policing-rate-per-flow" class="internal xref">(Non-)Policing Rate per Flow</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.8.2.1.2.2">
<p id="section-toc.1-1.8.2.1.2.2.1"><a href="#section-8.1.2" class="auto internal xref">8.1.2</a>. <a href="#name-non-policing-l4s-service-ra" class="internal xref">(Non-)Policing L4S Service Rate</a></p>
</li>
</ul>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.8.2.2">
<p id="section-toc.1-1.8.2.2.1"><a href="#section-8.2" class="auto internal xref">8.2</a>. <a href="#name-latency-friendliness" class="internal xref">'Latency Friendliness'</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.8.2.3">
<p id="section-toc.1-1.8.2.3.1"><a href="#section-8.3" class="auto internal xref">8.3</a>. <a href="#name-interaction-between-rate-po" class="internal xref">Interaction between Rate Policing and L4S</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.8.2.4">
<p id="section-toc.1-1.8.2.4.1"><a href="#section-8.4" class="auto internal xref">8.4</a>. <a href="#name-ecn-integrity" class="internal xref">ECN Integrity</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.8.2.5">
<p id="section-toc.1-1.8.2.5.1"><a href="#section-8.5" class="auto internal xref">8.5</a>. <a href="#name-privacy-considerations" class="internal xref">Privacy Considerations</a></p>
</li>
</ul>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.9">
<p id="section-toc.1-1.9.1"><a href="#section-9" class="auto internal xref">9</a>. <a href="#name-informative-references" class="internal xref">Informative References</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.10">
<p id="section-toc.1-1.10.1"><a href="#appendix-A" class="auto internal xref"></a><a href="#name-acknowledgements" class="internal xref">Acknowledgements</a></p>
</li>
<li class="compact toc ulBare ulEmpty" id="section-toc.1-1.11">
<p id="section-toc.1-1.11.1"><a href="#appendix-B" class="auto internal xref"></a><a href="#name-authors-addresses" class="internal xref">Authors' Addresses</a></p>
</li>
</ul>
</nav>
</section>
</div>
<div id="l4sps_intro">
<section id="section-1">
<h2 id="name-introduction">
<a href="#section-1" class="section-number selfRef">1. </a><a href="#name-introduction" class="section-name selfRef">Introduction</a>
</h2>
<p id="section-1-1">At any one time, it is increasingly common for all of the traffic in
a bottleneck link (e.g., a household's Internet access or Wi-Fi) to come from
applications that prefer low delay: interactive web, web services,
voice, conversational video, interactive video, interactive remote
presence, instant messaging, online and cloud-rendered gaming, remote desktop, cloud-based
applications, cloud-rendered virtual reality or augmented reality, and video-assisted remote control of machinery and
industrial processes. In the last decade or so, much has been done to
reduce propagation delay by placing caches or servers closer to users.
However, queuing remains a major, albeit intermittent, component of
latency. For instance, spikes of hundreds of milliseconds are not
uncommon, even with state-of-the-art Active Queue Management
(AQM) <span>[<a href="#COBALT" class="cite xref">COBALT</a>]</span> <span>[<a href="#DOCSIS3AQM" class="cite xref">DOCSIS3AQM</a>]</span>. A Classic AQM in an
access network bottleneck is typically configured to buffer the sawteeth of
lone flows, which can cause peak overall
network delay to roughly double during a long-running flow, relative to
expected base (unloaded) path delay <span>[<a href="#BufferSize" class="cite xref">BufferSize</a>]</span>.
Low loss is also important because, for interactive applications, losses
translate into even longer retransmission delays.<a href="#section-1-1" class="pilcrow">¶</a></p>
<p id="section-1-2">It has been demonstrated that, once access network bit rates reach
levels now common in the developed world, increasing link capacity
offers diminishing returns if latency (delay) is not addressed <span>[<a href="#Dukkipati06" class="cite xref">Dukkipati06</a>]</span> <span>[<a href="#Rajiullah15" class="cite xref">Rajiullah15</a>]</span>. Therefore, the
goal is an Internet service with very low queuing latency, very low
loss, and scalable throughput. Very low queuing latency means less
than 1 millisecond (ms) on average and less than about 2 ms at
the 99th percentile. End-to-end delay above 50 ms <span>[<a href="#Raaen14" class="cite xref">Raaen14</a>]</span>, or even above 20 ms <span>[<a href="#NASA04" class="cite xref">NASA04</a>]</span>,
starts to feel unnatural for more demanding interactive applications. Therefore,
removing unnecessary delay variability increases the reach of these
applications (the distance over which they are comfortable to use) and/or
provides additional latency budget that can be used for enhanced processing. This
document describes the L4S architecture for achieving these goals.<a href="#section-1-2" class="pilcrow">¶</a></p>
<p id="section-1-3">Differentiated services (Diffserv) offers Expedited Forwarding
(EF) <span>[<a href="#RFC3246" class="cite xref">RFC3246</a>]</span> for some packets at the expense of
others, but this makes no difference when all (or most) of the traffic
at a bottleneck at any one time requires low latency. In contrast, L4S
still works well when all traffic is L4S -- a service that gives without
taking needs none of the configuration or management baggage (traffic
policing or traffic contracts) associated with favouring some traffic
flows over others.<a href="#section-1-3" class="pilcrow">¶</a></p>
<p id="section-1-4">Queuing delay degrades performance intermittently <span>[<a href="#Hohlfeld14" class="cite xref">Hohlfeld14</a>]</span>.
It occurs i) when a large enough capacity-seeking
(e.g., TCP) flow is running alongside the user's traffic in the
bottleneck link, which is typically in the access network, or ii) when the
low latency application is itself a large capacity-seeking or adaptive
rate flow (e.g., interactive video).
At these times, the performance
improvement from L4S must be sufficient for network operators to be motivated
to deploy it.<a href="#section-1-4" class="pilcrow">¶</a></p>
<p id="section-1-5">Active Queue Management (AQM) is part of the solution to queuing
under load. AQM improves performance for all traffic, but there is a
limit to how much queuing delay can be reduced by solely changing the
network without addressing the root of the problem.<a href="#section-1-5" class="pilcrow">¶</a></p>
<p id="section-1-6">The root of the problem is the presence of standard congestion
control (Reno <span>[<a href="#RFC5681" class="cite xref">RFC5681</a>]</span>) or compatible variants
(e.g., CUBIC <span>[<a href="#RFC8312" class="cite xref">RFC8312</a>]</span>) that are used in TCP and
in other transports, such as QUIC <span>[<a href="#RFC9000" class="cite xref">RFC9000</a>]</span>.
We shall use
the term 'Classic' for these Reno-friendly congestion controls.
Classic
congestion controls induce relatively large sawtooth-shaped excursions
of queue occupancy. So if a network operator naively
attempts to reduce queuing delay by configuring an AQM to operate at a
shallower queue, a Classic congestion control will significantly
underutilize the link at the bottom of every sawtooth. These sawteeth have
also been growing in duration as flow rate scales (see <a href="#l4sps_why_primary_components" class="auto internal xref">Section 5.1</a>
and <span>[<a href="#RFC3649" class="cite xref">RFC3649</a>]</span>).<a href="#section-1-6" class="pilcrow">¶</a></p>
<p id="section-1-7">It has been demonstrated that, if the sending host replaces a Classic
congestion control with a 'Scalable' alternative, the performance under load of all the above
interactive applications can be significantly improved once a suitable AQM is
deployed in the network.
Taking the example solution cited below that uses Data Center TCP (DCTCP)
<span>[<a href="#RFC8257" class="cite xref">RFC8257</a>]</span> and a Dual-Queue Coupled AQM <span>[<a href="#RFC9332" class="cite xref">RFC9332</a>]</span> on a DSL or Ethernet link,
queuing delay under heavy load is roughly 1-2 ms at
the 99th percentile without losing link utilization <span>[<a href="#L4Seval22" class="cite xref">L4Seval22</a>]</span> <span>[<a href="#DualPI2Linux" class="cite xref">DualPI2Linux</a>]</span> (for other link types,
see <a href="#l4sarch_link-specifics" class="auto internal xref">Section 6.3</a>).
This compares with
5-20 ms on <em>average</em> with a Classic
congestion control and current state-of-the-art AQMs, such as
Flow Queue CoDel <span>[<a href="#RFC8290" class="cite xref">RFC8290</a>]</span>, Proportional Integral controller Enhanced (PIE) <span>[<a href="#RFC8033" class="cite xref">RFC8033</a>]</span>, or DOCSIS PIE <span>[<a href="#RFC8034" class="cite xref">RFC8034</a>]</span> and about
20-30 ms at the 99th percentile <span>[<a href="#DualPI2Linux" class="cite xref">DualPI2Linux</a>]</span>.<a href="#section-1-7" class="pilcrow">¶</a></p>
<p id="section-1-8">L4S is designed for incremental deployment. It is possible to deploy
the L4S service at a bottleneck link alongside the existing best efforts
service <span>[<a href="#DualPI2Linux" class="cite xref">DualPI2Linux</a>]</span> so that unmodified
applications can start using it as soon as the sender's stack is
updated. Access networks are typically designed with one link as the
bottleneck for each site (which might be a home, small enterprise, or
mobile device), so deployment at either or both ends of this link should
give nearly all the benefit in the respective direction.
With some
transport protocols, namely TCP <span>[<a href="#I-D.ietf-tcpm-accurate-ecn" class="cite xref">ACCECN</a>]</span>, the sender has to check that
the receiver has been suitably updated to give more accurate feedback,
whereas with more recent transport protocols, such as QUIC <span>[<a href="#RFC9000" class="cite xref">RFC9000</a>]</span> and Datagram Congestion Control Protocol (DCCP) <span>[<a href="#RFC4340" class="cite xref">RFC4340</a>]</span>, all
receivers have always been suitable.<a href="#section-1-8" class="pilcrow">¶</a></p>
<p id="section-1-9">This document presents the L4S architecture. It consists of three
components: network support to isolate L4S traffic from Classic traffic;
protocol features that allow network elements to identify L4S traffic;
and host support for L4S congestion controls. The protocol is defined
separately in <span>[<a href="#RFC9331" class="cite xref">RFC9331</a>]</span> as an experimental
change to Explicit Congestion Notification (ECN). This document
describes and justifies the component parts and how they interact to
provide the low latency, low loss, and scalable Internet service. It also
details the approach to incremental deployment, as briefly summarized
above.<a href="#section-1-9" class="pilcrow">¶</a></p>
<section id="section-1.1">
<h3 id="name-document-roadmap">
<a href="#section-1.1" class="section-number selfRef">1.1. </a><a href="#name-document-roadmap" class="section-name selfRef">Document Roadmap</a>
</h3>
<p id="section-1.1-1">This document describes the L4S architecture in three passes. First,
the brief overview in <a href="#l4s-arch_arch_overview" class="auto internal xref">Section 2</a> gives the very high-level idea and states the main
components with minimal rationale. This is only intended to give some
context for the terminology definitions that follow in <a href="#l4sps_Terminology" class="auto internal xref">Section 3</a> and to explain the structure of the rest
of the document. Then, <a href="#l4sps_components" class="auto internal xref">Section 4</a> goes into more
detail on each component with some rationale but still mostly stating
what the architecture is, rather than why. Finally, <a href="#l4sps_rationale" class="auto internal xref">Section 5</a> justifies why each element of the solution
was chosen (<a href="#l4sps_why_primary_components" class="auto internal xref">Section 5.1</a>) and why
these choices were different from other solutions (<a href="#l4sps_why-not" class="auto internal xref">Section 5.2</a>).<a href="#section-1.1-1" class="pilcrow">¶</a></p>
<p id="section-1.1-2">After the architecture has been described, <a href="#l4sarch_applicability" class="auto internal xref">Section 6</a>
clarifies its applicability by describing the applications and use cases
that motivated the design, the challenges applying the architecture to
various link technologies, and various incremental deployment models
(including the two main deployment topologies, different sequences for
incremental deployment, and various interactions with preexisting
approaches). The document
ends with the usual tailpieces, including extensive discussion of
traffic policing and other security considerations in <a href="#l4sps_Security_Considerations" class="auto internal xref">Section 8</a>.<a href="#section-1.1-2" class="pilcrow">¶</a></p>
</section>
</section>
</div>
<div id="l4s-arch_arch_overview">
<section id="section-2">
<h2 id="name-l4s-architecture-overview">
<a href="#section-2" class="section-number selfRef">2. </a><a href="#name-l4s-architecture-overview" class="section-name selfRef">L4S Architecture Overview</a>
</h2>
<p id="section-2-1">Below, we outline the three main components to the L4S architecture:
1) the Scalable congestion control on the sending host; 2) the AQM at
the network bottleneck; and 3) the protocol between them.<a href="#section-2-1" class="pilcrow">¶</a></p>
<p id="section-2-2">But first, the main point to grasp is that low latency is not
provided by the network; low latency results from the careful behaviour
of the Scalable congestion controllers used by L4S senders. The network
does have a role, primarily to isolate the low latency of the carefully
behaving L4S traffic from the higher queuing delay needed by traffic
with preexisting Classic behaviour. The network also alters the way it
signals queue growth to the transport. It uses the Explicit Congestion
Notification (ECN) protocol, but it signals the very start of queue
growth immediately, without the smoothing delay typical of Classic
AQMs. Because ECN support is essential for L4S, senders use the ECN
field as the protocol that allows the network to identify which packets
are L4S and which are Classic.<a href="#section-2-2" class="pilcrow">¶</a></p>
<span class="break"></span><dl class="olPercent" id="section-2-3">
<dt>1)</dt>
<dd id="section-2-3.1">
<p id="section-2-3.1.1">Host:<a href="#section-2-3.1.1" class="pilcrow">¶</a></p>
<p id="section-2-3.1.2">Scalable congestion controls already exist. They solve the scaling
problem with Classic congestion controls, such as Reno or
CUBIC. Because flow rate has scaled since TCP congestion control was
first designed in 1988, assuming the flow lasts long enough, it now
takes hundreds of round trips (and growing) to recover after a
congestion signal (whether a loss or an ECN mark), as shown in the
examples in <a href="#l4sps_why_primary_components" class="auto internal xref">Section 5.1</a> and <span>[<a href="#RFC3649" class="cite xref">RFC3649</a>]</span>. Therefore, control of queuing and utilization
becomes very slack, and the slightest disturbances (e.g., from new
flows starting) prevent a high rate from being attained.<a href="#section-2-3.1.2" class="pilcrow">¶</a></p>
<p id="section-2-3.1.3">With a Scalable congestion control, the average time from one
congestion signal to the next (the recovery time) remains invariant as
flow rate scales, all other factors being equal. This maintains
the same degree of control over queuing and utilization, whatever the
flow rate, as well as ensuring that high throughput is more robust to
disturbances. The Scalable control used most widely (in controlled
environments) is DCTCP <span>[<a href="#RFC8257" class="cite xref">RFC8257</a>]</span>, which has been implemented and deployed in
Windows Server Editions (since 2012), in Linux, and in
FreeBSD.
Although DCTCP as-is functions well over wide-area round-trip
times (RTTs), most implementations lack certain safety features that would be
necessary for use outside controlled environments, like data centres
(see <a href="#l4sarch_sec_non-l4s-neck" class="auto internal xref">Section 6.4.3</a>). Therefore,
Scalable congestion control needs to be implemented in TCP and other
transport protocols (QUIC, Stream Control Transmission Protocol (SCTP), RTP/RTCP, RTP Media Congestion Avoidance Techniques (RMCAT), etc.).
Indeed,
between the present document being drafted and published, the
following Scalable congestion controls were implemented: Prague over TCP and QUIC
<span>[<a href="#I-D.briscoe-iccrg-prague-congestion-control" class="cite xref">PRAGUE-CC</a>]</span> <span>[<a href="#PragueLinux" class="cite xref">PragueLinux</a>]</span>, an L4S
variant of the RMCAT SCReAM controller <span>[<a href="#SCReAM-L4S" class="cite xref">SCReAM-L4S</a>]</span>, and the L4S ECN part of Bottleneck Bandwidth and Round-trip propagation time (BBRv2) <span>[<a href="#BBRv2" class="cite xref">BBRv2</a>]</span> intended for TCP and QUIC transports.<a href="#section-2-3.1.3" class="pilcrow">¶</a></p>
</dd>
<dd class="break"></dd>
<dt>2)</dt>
<dd id="section-2-3.2">
<p id="section-2-3.2.1">Network:<a href="#section-2-3.2.1" class="pilcrow">¶</a></p>
<p id="section-2-3.2.2">L4S traffic needs to be isolated from the queuing latency of
Classic traffic. One queue per application flow (FQ) is one way to
achieve this, e.g., FQ-CoDel <span>[<a href="#RFC8290" class="cite xref">RFC8290</a>]</span>. However, using just two queues is sufficient and
does not require inspection of transport layer headers in the network,
which is not always possible (see <a href="#l4sps_why-not" class="auto internal xref">Section 5.2</a>). With just two queues, it might seem impossible to
know how much capacity to schedule for each queue without inspecting
how many flows at any one time are using each. And it would be
undesirable to arbitrarily divide access network capacity into two
partitions. The Dual-Queue Coupled AQM was developed as a minimal
complexity solution to this problem. It acts like a 'semi-permeable'
membrane that partitions latency but not bandwidth. As such, the two
queues are for transitioning from Classic to L4S behaviour, not bandwidth
prioritization.<a href="#section-2-3.2.2" class="pilcrow">¶</a></p>
<p id="section-2-3.2.3"><a href="#l4sps_components" class="auto internal xref">Section 4</a> gives a high-level
explanation of how the per-flow queue (FQ) and DualQ variants of
L4S work, and <span>[<a href="#RFC9332" class="cite xref">RFC9332</a>]</span> gives a full explanation of the DualQ Coupled AQM
framework. A specific marking algorithm is not mandated for L4S
AQMs. Appendices of <span>[<a href="#RFC9332" class="cite xref">RFC9332</a>]</span> give non-normative examples that have been
implemented and evaluated and give recommended default parameter
settings. It is expected that L4S experiments will improve knowledge
of parameter settings and whether the set of marking algorithms needs
to be limited.<a href="#section-2-3.2.3" class="pilcrow">¶</a></p>
</dd>
<dd class="break"></dd>
<dt>3)</dt>
<dd id="section-2-3.3">
<p id="section-2-3.3.1">Protocol:<a href="#section-2-3.3.1" class="pilcrow">¶</a></p>
<p id="section-2-3.3.2">A sending host needs to distinguish L4S and Classic packets with an
identifier so that the network can classify them into their separate
treatments. The L4S identifier spec <span>[<a href="#RFC9331" class="cite xref">RFC9331</a>]</span> concludes that
all alternatives involve compromises, but the ECT(1) and Congestion Experienced (CE) codepoints
of the ECN field represent a workable solution. As already explained,
the network also uses ECN to immediately signal the very start of
queue growth to the transport.<a href="#section-2-3.3.2" class="pilcrow">¶</a></p>
</dd>
<dd class="break"></dd>
</dl>
</section>
</div>
<div id="l4sps_Terminology">
<section id="section-3">
<h2 id="name-terminology">
<a href="#section-3" class="section-number selfRef">3. </a><a href="#name-terminology" class="section-name selfRef">Terminology</a>
</h2>
<span class="break"></span><dl class="dlParallel" id="section-3-1">
<dt id="section-3-1.1">Classic Congestion Control:</dt>
<dd style="margin-left: 1.5em" id="section-3-1.2">A congestion control
behaviour that can coexist with standard Reno <span>[<a href="#RFC5681" class="cite xref">RFC5681</a>]</span> without causing significantly negative impact on
its flow rate <span>[<a href="#RFC5033" class="cite xref">RFC5033</a>]</span>. The scaling problem
with Classic congestion control is explained, with examples, in
<a href="#l4sps_why_primary_components" class="auto internal xref">Section 5.1</a> and in <span>[<a href="#RFC3649" class="cite xref">RFC3649</a>]</span>.<a href="#section-3-1.2" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-3-1.3">Scalable Congestion Control:</dt>
<dd style="margin-left: 1.5em" id="section-3-1.4">A congestion control
where the average time from one congestion signal to the next (the
recovery time) remains invariant as flow rate scales, all other
factors being equal.
For instance, DCTCP averages 2 congestion
signals per round trip, whatever the flow rate, as do other recently
developed Scalable congestion controls, e.g., Relentless
TCP <span>[<a href="#I-D.mathis-iccrg-relentless-tcp" class="cite xref">RELENTLESS</a>]</span>, Prague for TCP and QUIC <span>[<a href="#I-D.briscoe-iccrg-prague-congestion-control" class="cite xref">PRAGUE-CC</a>]</span> <span>[<a href="#PragueLinux" class="cite xref">PragueLinux</a>]</span>, BBRv2 <span>[<a href="#BBRv2" class="cite xref">BBRv2</a>]</span> <span>[<a href="#I-D.cardwell-iccrg-bbr-congestion-control" class="cite xref">BBR-CC</a>]</span>, and the L4S
variant of SCReAM for real-time media <span>[<a href="#SCReAM-L4S" class="cite xref">SCReAM-L4S</a>]</span> <span>[<a href="#RFC8298" class="cite xref">RFC8298</a>]</span>. See
<span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-4.3" class="relref">Section 4.3</a> of [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span> for more
explanation.<a href="#section-3-1.4" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-3-1.5">Classic Service:</dt>
<dd style="margin-left: 1.5em" id="section-3-1.6">The Classic service is intended for
all the congestion control behaviours that coexist with
Reno <span>[<a href="#RFC5681" class="cite xref">RFC5681</a>]</span> (e.g., Reno itself,
CUBIC <span>[<a href="#RFC8312" class="cite xref">RFC8312</a>]</span>, Compound <span>[<a href="#I-D.sridharan-tcpm-ctcp" class="cite xref">CTCP</a>]</span>, and TFRC <span>[<a href="#RFC5348" class="cite xref">RFC5348</a>]</span>). The term 'Classic queue' means a queue
providing the Classic service.<a href="#section-3-1.6" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-3-1.7">Low Latency, Low Loss, and Scalable throughput (L4S) service:</dt>
<dd style="margin-left: 1.5em" id="section-3-1.8">
<p id="section-3-1.8.1">The
'L4S' service is intended for traffic from Scalable congestion
control algorithms, such as the Prague congestion control <span>[<a href="#I-D.briscoe-iccrg-prague-congestion-control" class="cite xref">PRAGUE-CC</a>]</span>, which was
derived from DCTCP <span>[<a href="#RFC8257" class="cite xref">RFC8257</a>]</span>. The L4S service
is for more general traffic than just Prague -- it allows the
set of congestion controls with similar scaling properties to Prague
to evolve, such as the examples listed above (Relentless, SCReAM, etc.).
The term 'L4S queue' means a queue providing the L4S service.<a href="#section-3-1.8.1" class="pilcrow">¶</a></p>
<p id="section-3-1.8.2">The terms Classic or L4S can also qualify other
nouns, such as 'queue', 'codepoint', 'identifier', 'classification',
'packet', and 'flow'. For example, an L4S packet means a packet with an
L4S identifier sent from an L4S congestion control.<a href="#section-3-1.8.2" class="pilcrow">¶</a></p>
<p id="section-3-1.8.3">Both Classic and L4S services can cope with a
proportion of unresponsive or less-responsive traffic as well but,
in the L4S case, its rate has to be smooth enough or low enough to
not build a queue (e.g., DNS, Voice over IP (VoIP), game sync datagrams,
etc.).<a href="#section-3-1.8.3" class="pilcrow">¶</a></p>
</dd>
<dd class="break"></dd>
<dt id="section-3-1.9">Reno-friendly:</dt>
<dd style="margin-left: 1.5em" id="section-3-1.10">The subset of Classic traffic that is
friendly to the standard Reno congestion control defined for TCP in
<span>[<a href="#RFC5681" class="cite xref">RFC5681</a>]</span>. The TFRC spec <span>[<a href="#RFC5348" class="cite xref">RFC5348</a>]</span> indirectly implies that 'friendly' is defined as
"generally within a factor of two of the sending rate of a TCP flow
under the same conditions". Reno-friendly is used here in place of
'TCP-friendly', given the latter has become imprecise, because the
TCP protocol is now used with so many different congestion control
behaviours, and Reno is used in non-TCP transports, such as
QUIC <span>[<a href="#RFC9000" class="cite xref">RFC9000</a>]</span>.<a href="#section-3-1.10" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-3-1.11">Classic ECN:</dt>
<dd style="margin-left: 1.5em" id="section-3-1.12">
<p id="section-3-1.12.1">The original Explicit Congestion
Notification (ECN) protocol <span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span> that
requires ECN signals to be treated as equivalent to drops, both when
generated in the network and when responded to by the sender.<a href="#section-3-1.12.1" class="pilcrow">¶</a></p>
<p id="section-3-1.12.2">For L4S, the names used for the four codepoints of the 2-bit
IP-ECN field are unchanged from those defined in the ECN spec
<span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span>, i.e., Not-ECT, ECT(0),
ECT(1), and CE, where ECT stands for ECN-Capable Transport and CE
stands for Congestion Experienced. A packet marked with the CE
codepoint is termed 'ECN-marked' or sometimes just 'marked' where
the context makes ECN obvious.<a href="#section-3-1.12.2" class="pilcrow">¶</a></p>
</dd>
<dd class="break"></dd>
<dt id="section-3-1.13">Site:</dt>
<dd style="margin-left: 1.5em" id="section-3-1.14">A home, mobile device, small enterprise, or
campus where the network bottleneck is typically the access link to
the site. Not all network arrangements fit this model, but it is a
useful, widely applicable generalization.<a href="#section-3-1.14" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-3-1.15">Traffic Policing:</dt>
<dd style="margin-left: 1.5em" id="section-3-1.16">Limiting traffic by dropping packets
or shifting them to a lower service class (as opposed to introducing
delay, which is termed 'traffic shaping'). Policing can involve
limiting the average rate and/or burst size. Policing focused on
limiting queuing but not the average flow rate is termed 'congestion
policing', 'latency policing', 'burst policing', or 'queue protection' in
this document. Otherwise, the term rate policing is used.<a href="#section-3-1.16" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
</dl>
</section>
</div>
<div id="l4sps_components">
<section id="section-4">
<h2 id="name-l4s-architecture-components">
<a href="#section-4" class="section-number selfRef">4. </a><a href="#name-l4s-architecture-components" class="section-name selfRef">L4S Architecture Components</a>
</h2>
<p id="section-4-1">The L4S architecture is composed of the elements in the following
three subsections.<a href="#section-4-1" class="pilcrow">¶</a></p>
<div id="l4sps_protocol_components">
<section id="section-4.1">
<h3 id="name-protocol-mechanisms">
<a href="#section-4.1" class="section-number selfRef">4.1. </a><a href="#name-protocol-mechanisms" class="section-name selfRef">Protocol Mechanisms</a>
</h3>
<p id="section-4.1-1">The L4S architecture involves: a) unassignment of the previous use
of the identifier; b) reassignment of the same identifier; and c)
optional further identifiers:<a href="#section-4.1-1" class="pilcrow">¶</a></p>
<ol start="1" type="a" class="normal type-a" id="section-4.1-2">
<li id="section-4.1-2.1">
<p id="section-4.1-2.1.1">An essential aspect of a Scalable congestion control is the use
of explicit congestion signals. Classic ECN <span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span> requires an ECN signal to be treated as
equivalent to drop, both when it is generated in the network and
when it is responded to by hosts. L4S needs networks and hosts to
support a more fine-grained meaning for each ECN signal that is
less severe than a drop, so that the L4S signals:<a href="#section-4.1-2.1.1" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-4.1-2.1.2.1">can be much more frequent and<a href="#section-4.1-2.1.2.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-4.1-2.1.2.2">can be signalled immediately, without the significant delay
required to smooth out fluctuations in the queue.<a href="#section-4.1-2.1.2.2" class="pilcrow">¶</a>
</li>
</ul>
<p id="section-4.1-2.1.3">To enable L4S, the Standards Track Classic ECN
spec <span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span> has had to be updated to allow
L4S packets to depart from the 'equivalent-to-drop' constraint.
<span>[<a href="#RFC8311" class="cite xref">RFC8311</a>]</span> is a Standards Track update to
relax specific requirements in <span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span>
(and certain other Standards
Track RFCs), which clears the way for the experimental changes
proposed for L4S. Also, the ECT(1) codepoint was previously
assigned as the experimental ECN nonce <span>[<a href="#RFC3540" class="cite xref">RFC3540</a>]</span>, which <span>[<a href="#RFC8311" class="cite xref">RFC8311</a>]</span> recategorizes as historic to
make the codepoint available again.<a href="#section-4.1-2.1.3" class="pilcrow">¶</a></p>
</li>
<li id="section-4.1-2.2">
<p id="section-4.1-2.2.1"><span>[<a href="#RFC9331" class="cite xref">RFC9331</a>]</span> specifies that
ECT(1) is used as the identifier to classify L4S packets into a
separate treatment from Classic packets. This satisfies the
requirement for identifying an alternative ECN treatment in <span>[<a href="#RFC4774" class="cite xref">RFC4774</a>]</span>.<a href="#section-4.1-2.2.1" class="pilcrow">¶</a></p>
<p id="section-4.1-2.2.2">The CE codepoint is
used to indicate Congestion Experienced by both L4S and Classic
treatments. This raises the concern that a Classic AQM earlier on
the path might have marked some ECT(0) packets as CE. Then, these
packets will be erroneously classified into the L4S queue.
<span><a href="https://www.rfc-editor.org/rfc/rfc9331#appendix-B" class="relref">Appendix B</a> of [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span> explains why five unlikely
eventualities all have to coincide for this to have any
detrimental effect, which even then would only involve a
vanishingly small likelihood of a spurious retransmission.<a href="#section-4.1-2.2.2" class="pilcrow">¶</a></p>
</li>
<li id="section-4.1-2.3">A network operator might wish to include certain unresponsive,
non-L4S traffic in the L4S queue if it is deemed to be paced smoothly
enough and at a low enough rate not to build a queue, for
instance, VoIP, low rate datagrams to sync online games,
relatively low rate application-limited traffic, DNS, Lightweight Directory Access Protocol (LDAP), etc.
This traffic would need to be tagged with specific identifiers,
e.g., a low-latency Diffserv codepoint such as Expedited
Forwarding (EF) <span>[<a href="#RFC3246" class="cite xref">RFC3246</a>]</span>, Non-Queue-Building
(NQB) <span>[<a href="#I-D.ietf-tsvwg-nqb" class="cite xref">NQB-PHB</a>]</span>, or
operator-specific identifiers.<a href="#section-4.1-2.3" class="pilcrow">¶</a>
</li>
</ol>
</section>
</div>
<div id="l4sps_network_components">
<section id="section-4.2">
<h3 id="name-network-components">
<a href="#section-4.2" class="section-number selfRef">4.2. </a><a href="#name-network-components" class="section-name selfRef">Network Components</a>
</h3>
<p id="section-4.2-1">The L4S architecture aims to provide low latency without the <em>need</em> for per-flow operations in network
components. Nonetheless, the architecture does not preclude per-flow
solutions. The following bullets describe the known arrangements: a)
the DualQ Coupled AQM with an L4S AQM in one queue coupled from a
Classic AQM in the other; b) per-flow queues with an instance of a
Classic and an L4S AQM in each queue; and c) Dual queues with per-flow
AQMs but no per-flow queues:<a href="#section-4.2-1" class="pilcrow">¶</a></p>
<ol start="1" type="a" class="normal type-a" id="section-4.2-2">
<li id="section-4.2-2.1">
<p id="section-4.2-2.1.1">The Dual-Queue Coupled AQM (illustrated in <a href="#l4sps_fig_components" class="auto internal xref">Figure 1</a>) achieves the 'semi-permeable'
membrane property mentioned earlier as follows:<a href="#section-4.2-2.1.1" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-4.2-2.1.2.1">Latency isolation: Two separate queues are used to isolate
L4S queuing delay from the larger queue that Classic traffic
needs to maintain full utilization.<a href="#section-4.2-2.1.2.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-4.2-2.1.2.2">Bandwidth pooling: The two queues act as if they are a
single pool of bandwidth in which flows of either type get
roughly equal throughput without the scheduler needing to
identify any flows. This is achieved by having an AQM in each
queue, but the Classic AQM provides a congestion signal to
both queues in a manner that ensures a consistent response
from the two classes of congestion control. Specifically, the
Classic AQM generates a drop/mark probability based on
congestion in its own queue, which it uses both to drop/mark
packets in its own queue and to affect the marking probability
in the L4S queue. The strength of the coupling of the
congestion signalling between the two queues is enough to make
the L4S flows slow down to leave the right amount of capacity
for the Classic flows (as they would if they were the same
type of traffic sharing the same queue).<a href="#section-4.2-2.1.2.2" class="pilcrow">¶</a>
</li>
</ul>
<p id="section-4.2-2.1.3">Then, the scheduler can serve the L4S queue with priority
(denoted by the '1' on the higher priority input), because the L4S
traffic isn't offering up enough traffic to use all the priority
that it is given. Therefore:<a href="#section-4.2-2.1.3" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-4.2-2.1.4.1">for latency isolation on short timescales (sub-round-trip),
the prioritization of the L4S queue protects its low latency
by allowing bursts to dissipate quickly;<a href="#section-4.2-2.1.4.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-4.2-2.1.4.2">but for bandwidth pooling on longer timescales (round-trip
and longer), the Classic queue creates an equal and opposite
pressure against the L4S traffic to ensure that neither has
priority when it comes to bandwidth -- the tension between
prioritizing L4S and coupling the marking from the Classic AQM
results in approximate per-flow fairness.<a href="#section-4.2-2.1.4.2" class="pilcrow">¶</a>
</li>
</ul>
<p id="section-4.2-2.1.5">To protect against the prioritization of persistent L4S traffic
deadlocking the Classic queue for a while in some implementations,
it is advisable for the priority to be conditional, not
strict (see <span><a href="https://www.rfc-editor.org/rfc/rfc9332#appendix-A" class="relref">Appendix A</a> of the DualQ spec [<a href="#RFC9332" class="cite xref">RFC9332</a>]</span>).<a href="#section-4.2-2.1.5" class="pilcrow">¶</a></p>
<p id="section-4.2-2.1.6">When there is no Classic traffic, the L4S
queue's own AQM comes into play. It starts congestion
marking with a very shallow queue, so L4S traffic maintains very
low queuing delay.<a href="#section-4.2-2.1.6" class="pilcrow">¶</a></p>
<p id="section-4.2-2.1.7">If either queue becomes persistently overloaded, drop of some
ECN-capable packets is introduced, as recommended in <span><a href="https://www.rfc-editor.org/rfc/rfc3168#section-7" class="relref">Section 7</a> of the ECN
spec [<a href="#RFC3168" class="cite xref">RFC3168</a>]</span> and <span><a href="https://www.rfc-editor.org/rfc/rfc7567#section-4.2.1" class="relref">Section 4.2.1</a> of the AQM recommendations [<a href="#RFC7567" class="cite xref">RFC7567</a>]</span>. The trade-offs with different approaches
are discussed in <span><a href="https://www.rfc-editor.org/rfc/rfc9332#section-4.2.3" class="relref">Section 4.2.3</a> of the DualQ
spec [<a href="#RFC9332" class="cite xref">RFC9332</a>]</span> (not shown in the figure here).<a href="#section-4.2-2.1.7" class="pilcrow">¶</a></p>
<p id="section-4.2-2.1.8">The Dual-Queue Coupled AQM has been specified as
generically as possible <span>[<a href="#RFC9332" class="cite xref">RFC9332</a>]</span> without specifying the
particular AQMs to use in the two queues so that designers are
free to implement diverse ideas. Informational appendices in that
document give pseudocode examples of two different specific AQM
approaches: one called DualPI2 (pronounced Dual PI
Squared) <span>[<a href="#DualPI2Linux" class="cite xref">DualPI2Linux</a>]</span> that uses the PI2
variant of PIE and a zero-config variant of Random Early Detection (RED) called Curvy RED.
A DualQ Coupled AQM based on PIE has also been specified and
implemented for Low Latency DOCSIS <span>[<a href="#DOCSIS3.1" class="cite xref">DOCSIS3.1</a>]</span>.<a href="#section-4.2-2.1.8" class="pilcrow">¶</a></p>
<span id="name-components-of-an-l4s-dualq-"></span><div id="l4sps_fig_components">
<figure id="figure-1">
<div class="alignCenter art-text artwork" id="section-4.2-2.1.9.1">
<pre>
(3) (2)
.-------^------..------------^------------------.
,-(1)-----. _____
; ________ : L4S -------. | |
:|Scalable| : _\ ||__\_|mark |
:| sender | : __________ / / || / |_____|\ _________
:|________|\; | |/ -------' ^ \1|condit'nl|
`---------'\_| IP-ECN | Coupling : \|priority |_\
________ / |Classifier| : /|scheduler| /
|Classic |/ |__________|\ -------. __:__ / |_________|
| sender | \_\ || | ||__\_|mark/|/
|________| / || | || / |drop |
Classic -------' |_____|
(1) Scalable sending host
(2) Isolation in separate network queues
(3) Packet identification protocol
</pre>
</div>
<figcaption><a href="#figure-1" class="selfRef">Figure 1</a>:
<a href="#name-components-of-an-l4s-dualq-" class="selfRef">Components of an L4S DualQ Coupled AQM Solution</a>
</figcaption></figure>
</div>
</li>
<li id="section-4.2-2.2">Per-Flow Queues and AQMs: A scheduler with per-flow queues, such
as FQ-CoDel or FQ-PIE, can be used for L4S. For instance, within
each queue of an FQ-CoDel system, as well as a CoDel AQM, there is
typically also the option of ECN marking at an immediate
(unsmoothed) shallow threshold to support use in data centres (see
<span><a href="https://www.rfc-editor.org/rfc/rfc8290#section-5.2.7" class="relref">Section 5.2.7</a> of the FQ-CoDel spec [<a href="#RFC8290" class="cite xref">RFC8290</a>]</span>). In
Linux, this has been modified so that the shallow threshold can be
solely applied to ECT(1) packets <span>[<a href="#FQ_CoDel_Thresh" class="cite xref">FQ_CoDel_Thresh</a>]</span>. Then, if there is a flow of Not-ECT or
ECT(0) packets in the per-flow queue, the Classic AQM
(e.g., CoDel) is applied; whereas, if there is a flow of ECT(1)
packets in the queue, the shallower (typically sub-millisecond)
threshold is applied.
In addition, ECT(0) and Not-ECT packets
could potentially be classified into a separate flow queue from
ECT(1) and CE packets to avoid them mixing if they share a common
flow identifier (e.g., in a VPN).<a href="#section-4.2-2.2" class="pilcrow">¶</a>
</li>
<li id="section-4.2-2.3">
<p id="section-4.2-2.3.1">Dual queues but per-flow AQMs: It should also be possible to
use dual queues for isolation but with per-flow marking to
control flow rates (instead of the coupled per-queue marking of
the Dual-Queue Coupled AQM). One of the two queues would be for
isolating L4S packets, which would be classified by the ECN
codepoint. Flow rates could be controlled by flow-specific
marking. The policy goal of the marking could be to differentiate
flow rates (e.g., <span>[<a href="#Nadas20" class="cite xref">Nadas20</a>]</span>, which requires
additional signalling of a per-flow 'value') or to equalize
flow rates (perhaps in a similar way to Approx Fair
CoDel <span>[<a href="#AFCD" class="cite xref">AFCD</a>]</span> <span>[<a href="#I-D.morton-tsvwg-codel-approx-fair" class="cite xref">CODEL-APPROX-FAIR</a>]</span> but with two queues
not one).<a href="#section-4.2-2.3.1" class="pilcrow">¶</a></p>
<p id="section-4.2-2.3.2">Note that, whenever the term
'DualQ' is used loosely without saying whether marking is
per queue or per flow, it means a dual-queue AQM with per-queue
marking.<a href="#section-4.2-2.3.2" class="pilcrow">¶</a></p>
</li>
</ol>
</section>
</div>
<div id="l4sps_host_components">
<section id="section-4.3">
<h3 id="name-host-mechanisms">
<a href="#section-4.3" class="section-number selfRef">4.3. </a><a href="#name-host-mechanisms" class="section-name selfRef">Host Mechanisms</a>
</h3>
<p id="section-4.3-1">The L4S architecture includes two main mechanisms in the end host
that we enumerate next:<a href="#section-4.3-1" class="pilcrow">¶</a></p>
<ol start="1" type="a" class="normal type-a" id="section-4.3-2">
<li id="section-4.3-2.1">
<p id="section-4.3-2.1.1">Scalable congestion control at the sender: <a href="#l4s-arch_arch_overview" class="auto internal xref">Section 2</a> defines a Scalable congestion
control as one where the average time from one congestion signal
to the next (the recovery time) remains invariant as flow rate
scales, all other factors being equal. DCTCP is the most
widely used example. It has been documented as an informational
record of the protocol currently in use in controlled
environments <span>[<a href="#RFC8257" class="cite xref">RFC8257</a>]</span>. A list of safety
and performance improvements for a Scalable congestion control to
be usable on the public Internet has been drawn up (see the so-called
'Prague L4S requirements' in <span><a href="https://www.rfc-editor.org/rfc/rfc9331#appendix-A" class="relref">Appendix A</a> of [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>).
The subset that involve
risk of harm to others have been captured as normative
requirements in <span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-4" class="relref">Section 4</a> of [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>. TCP Prague <span>[<a href="#I-D.briscoe-iccrg-prague-congestion-control" class="cite xref">PRAGUE-CC</a>]</span> has been
implemented in Linux as a reference implementation to address
these requirements <span>[<a href="#PragueLinux" class="cite xref">PragueLinux</a>]</span>.<a href="#section-4.3-2.1.1" class="pilcrow">¶</a></p>
<p id="section-4.3-2.1.2">Transport protocols other than TCP use various
congestion controls that are designed to be friendly with Reno.
Before they can use the L4S service, they will need to be updated
to implement a Scalable congestion response, which they will have
to indicate by using the ECT(1) codepoint. Scalable variants are
under consideration for more recent transport protocols
(e.g., QUIC), and the L4S ECN part of BBRv2 <span>[<a href="#BBRv2" class="cite xref">BBRv2</a>]</span> <span>[<a href="#I-D.cardwell-iccrg-bbr-congestion-control" class="cite xref">BBR-CC</a>]</span> is a Scalable
congestion control intended for the TCP and QUIC transports,
amongst others. Also, an L4S variant of the RMCAT SCReAM
controller <span>[<a href="#RFC8298" class="cite xref">RFC8298</a>]</span> has been
implemented <span>[<a href="#SCReAM-L4S" class="cite xref">SCReAM-L4S</a>]</span> for media transported
over RTP.<a href="#section-4.3-2.1.2" class="pilcrow">¶</a></p>
<p id="section-4.3-2.1.3"> <span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-4.3" class="relref">Section 4.3</a> of the L4S ECN spec [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span> defines
Scalable congestion control in more detail and specifies the
requirements that an L4S Scalable congestion control has to comply
with.<a href="#section-4.3-2.1.3" class="pilcrow">¶</a></p>
</li>
<li id="section-4.3-2.2">
<p id="section-4.3-2.2.1">The ECN feedback in some transport protocols is already
sufficiently fine-grained for L4S (specifically DCCP <span>[<a href="#RFC4340" class="cite xref">RFC4340</a>]</span> and QUIC <span>[<a href="#RFC9000" class="cite xref">RFC9000</a>]</span>). But
others either require updates or are in the process of being
updated:<a href="#section-4.3-2.2.1" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-4.3-2.2.2.1">For the case of TCP, the feedback protocol for ECN embeds
the assumption from Classic ECN <span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span>
that an ECN mark is equivalent to a drop, making it unusable
for a Scalable TCP. Therefore, the implementation of TCP
receivers will have to be upgraded <span>[<a href="#RFC7560" class="cite xref">RFC7560</a>]</span>.
Work to standardize and implement more
accurate ECN feedback for TCP (AccECN) is in
progress <span>[<a href="#I-D.ietf-tcpm-accurate-ecn" class="cite xref">ACCECN</a>]</span>
<span>[<a href="#PragueLinux" class="cite xref">PragueLinux</a>]</span>.<a href="#section-4.3-2.2.2.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-4.3-2.2.2.2">ECN feedback was only roughly sketched in the appendix of
the now obsoleted second specification of SCTP <span>[<a href="#RFC4960" class="cite xref">RFC4960</a>]</span>, while a fuller specification was proposed
in a long-expired document <span>[<a href="#I-D.stewart-tsvwg-sctpecn" class="cite xref">ECN-SCTP</a>]</span>. A new design would need
to be implemented and deployed before SCTP could support
L4S.<a href="#section-4.3-2.2.2.2" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-4.3-2.2.2.3">For RTP, sufficient ECN feedback was defined in <span>[<a href="#RFC6679" class="cite xref">RFC6679</a>]</span>, but <span>[<a href="#RFC8888" class="cite xref">RFC8888</a>]</span> defines the
latest Standards Track improvements.<a href="#section-4.3-2.2.2.3" class="pilcrow">¶</a>
</li>
</ul>
</li>
</ol>
</section>
</div>
</section>
</div>
<div id="l4sps_rationale">
<section id="section-5">
<h2 id="name-rationale">
<a href="#section-5" class="section-number selfRef">5. </a><a href="#name-rationale" class="section-name selfRef">Rationale</a>
</h2>
<div id="l4sps_why_primary_components">
<section id="section-5.1">
<h3 id="name-why-these-primary-component">
<a href="#section-5.1" class="section-number selfRef">5.1. </a><a href="#name-why-these-primary-component" class="section-name selfRef">Why These Primary Components?</a>
</h3>
<span class="break"></span><dl class="dlParallel" id="section-5.1-1">
<dt id="section-5.1-1.1">Explicit congestion signalling (protocol):</dt>
<dd style="margin-left: 1.5em" id="section-5.1-1.2">
<p id="section-5.1-1.2.1">Explicit
congestion signalling is a key part of the L4S approach. In
contrast, use of drop as a congestion signal creates tension
because drop is both an impairment (less would be better) and a
useful signal (more would be better):<a href="#section-5.1-1.2.1" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-5.1-1.2.2.1">Explicit congestion signals can be used many times per
round trip to keep tight control without any impairment.
Under heavy load, even more explicit signals can be applied
so that the queue can be kept short whatever the load. In
contrast, Classic AQMs have to introduce very high packet drop
at high load to keep the queue short. By using ECN, an L4S
congestion control's sawtooth reduction can be smaller and
therefore return to the operating point more often, without
worrying that more sawteeth will cause more signals. The
consequent smaller amplitude sawteeth fit between an empty
queue and a very shallow marking threshold (~1 ms in the
public Internet), so queue delay variation can be very low,
without risk of underutilization.<a href="#section-5.1-1.2.2.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-5.1-1.2.2.2">Explicit congestion signals can be emitted immediately to
track fluctuations of the queue. L4S shifts smoothing from the
network to the host. The network doesn't know the round-trip
times (RTTs) of any of the flows. So if the network is responsible
for smoothing (as in the Classic approach), it has to assume a
worst case RTT, otherwise long RTT flows would become
unstable. This delays Classic congestion signals by 100-200
ms. In contrast, each host knows its own RTT. So,
in the L4S approach, the host can smooth each flow over its
own RTT, introducing no more smoothing delay than strictly
necessary (usually only a few milliseconds). A host can also
choose not to introduce any smoothing delay if appropriate,
e.g., during flow start-up.<a href="#section-5.1-1.2.2.2" class="pilcrow">¶</a>
</li>
</ul>
<p id="section-5.1-1.2.3">Neither of the above are feasible if explicit congestion
signalling has to be considered 'equivalent to drop' (as was
required with Classic ECN <span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span>), because
drop is an impairment as well as a signal. So drop cannot be
excessively frequent, and drop cannot be immediate; otherwise, too
many drops would turn out to have been due to only a transient
fluctuation in the queue that would not have warranted dropping a
packet in hindsight. Therefore, in an L4S AQM, the L4S queue uses
a new L4S variant of ECN that is not equivalent to drop (see
<span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-5.2" class="relref">Section 5.2</a> of the L4S ECN spec [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>), while the Classic queue
uses either Classic ECN <span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span> or drop,
which are still equivalent to each other.<a href="#section-5.1-1.2.3" class="pilcrow">¶</a></p>
<p id="section-5.1-1.2.4">Before
Classic ECN was standardized, there were various proposals to give
an ECN mark a different meaning from drop. However, there was no
particular reason to agree on any one of the alternative meanings,
so 'equivalent to drop' was the only compromise that could be
reached. <span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span> contains a statement that:<a href="#section-5.1-1.2.4" class="pilcrow">¶</a></p>
<ul class="normal ulEmpty">
<li class="normal ulEmpty" id="section-5.1-1.2.5.1">
<p style="margin-left: 0.5em" id="section-5.1-1.2.5.1.1">An environment where all end nodes were
ECN-Capable could allow new criteria to be developed for
setting the CE codepoint, and new congestion control
mechanisms for end-node reaction to CE packets. However, this
is a research issue, and as such is not addressed in this
document.<a href="#section-5.1-1.2.5.1.1" class="pilcrow">¶</a></p>
</li>
</ul>
</dd>
<dd class="break"></dd>
<dt id="section-5.1-1.3">Latency isolation (network):</dt>
<dd style="margin-left: 1.5em" id="section-5.1-1.4">L4S congestion controls
keep queue delay low, whereas Classic congestion controls need a
queue of the order of the RTT to avoid underutilization. One
queue cannot have two lengths; therefore, L4S traffic needs to be
isolated in a separate queue (e.g., DualQ) or queues
(e.g., FQ).<a href="#section-5.1-1.4" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-5.1-1.5">Coupled congestion notification:</dt>
<dd style="margin-left: 1.5em" id="section-5.1-1.6">Coupling the
congestion notification between two queues as in the DualQ Coupled
AQM is not necessarily essential, but it is a simple way to allow
senders to determine their rate packet by packet, rather than be
overridden by a network scheduler. An alternative is for a network
scheduler to control the rate of each application flow (see the
discussion in <a href="#l4sps_why-not" class="auto internal xref">Section 5.2</a>).<a href="#section-5.1-1.6" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-5.1-1.7">L4S packet identifier (protocol):</dt>
<dd style="margin-left: 1.5em" id="section-5.1-1.8">Once there are at
least two treatments in the network, hosts need an identifier at
the IP layer to distinguish which treatment they intend to
use.<a href="#section-5.1-1.8" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-5.1-1.9">Scalable congestion notification:</dt>
<dd style="margin-left: 1.5em" id="section-5.1-1.10">A Scalable
congestion control in the host keeps the signalling frequency from
the network high, whatever the flow rate, so that queue delay
variations can be small when conditions are stable, and rate can
track variations in available capacity as rapidly as possible
otherwise.<a href="#section-5.1-1.10" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-5.1-1.11">Low loss:</dt>
<dd style="margin-left: 1.5em" id="section-5.1-1.12">Latency is not the only concern of L4S.
The 'Low Loss' part of the name denotes that L4S generally
achieves zero congestion loss due to its use of ECN. Otherwise,
loss would itself cause delay, particularly for short flows, due
to retransmission delay <span>[<a href="#RFC2884" class="cite xref">RFC2884</a>]</span>.<a href="#section-5.1-1.12" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-5.1-1.13">Scalable throughput:</dt>
<dd style="margin-left: 1.5em" id="section-5.1-1.14">
<p id="section-5.1-1.14.1">The 'Scalable throughput' part
of the name denotes that the per-flow throughput of Scalable
congestion controls should scale indefinitely, avoiding the
imminent scaling problems with Reno-friendly congestion control
algorithms <span>[<a href="#RFC3649" class="cite xref">RFC3649</a>]</span>. It was known when TCP
congestion avoidance was first developed in 1988 that it would not
scale to high bandwidth-delay products (see footnote 6 in <span>[<a href="#TCP-CA" class="cite xref">TCP-CA</a>]</span>). Today, regular broadband flow rates over WAN
distances are already beyond the scaling range of Classic Reno
congestion control. So 'less unscalable' CUBIC <span>[<a href="#RFC8312" class="cite xref">RFC8312</a>]</span> and Compound <span>[<a href="#I-D.sridharan-tcpm-ctcp" class="cite xref">CTCP</a>]</span> variants of TCP have been
successfully deployed. However, these are now approaching their
scaling limits.<a href="#section-5.1-1.14.1" class="pilcrow">¶</a></p>
<p id="section-5.1-1.14.2">For instance, we will
consider a scenario with a maximum RTT of 30 ms at the peak
of each sawtooth. As Reno packet rate scales 8 times from 1,250 to
10,000 packet/s (from 15 to 120 Mb/s with 1500 B
packets), the time to recover from a congestion event rises
proportionately by 8 times as well, from 422 ms to 3.38 s. It
is clearly problematic for a congestion control to take multiple
seconds to recover from each congestion event. CUBIC <span>[<a href="#RFC8312" class="cite xref">RFC8312</a>]</span> was developed to be less unscalable, but it is
approaching its scaling limit; with the same max RTT of
30 ms, at 120 Mb/s, CUBIC is still fully in its
Reno-friendly mode, so it takes about 4.3 s to recover.
However, once flow rate scales by 8 times again to 960 Mb/s it
enters true CUBIC mode, with a recovery time of 12.2 s. From
then on, each further scaling by 8 times doubles CUBIC's recovery time
(because the cube root of 8 is 2), e.g., at 7.68 Gb/s, the
recovery time is 24.3 s. In contrast, a Scalable congestion
control like DCTCP or Prague induces 2 congestion signals per
round trip on average, which remains invariant for any flow rate,
keeping dynamic control very tight.<a href="#section-5.1-1.14.2" class="pilcrow">¶</a></p>
<p id="section-5.1-1.14.3">For a
feel of where the global average lone-flow download sits on this
scale at the time of writing (2021), according to <span>[<a href="#BDPdata" class="cite xref">BDPdata</a>]</span>, the global average fixed access capacity was 103
Mb/s in 2020 and the average base RTT to a CDN was 25 to 34 ms in 2019.
Averaging of per-country data was weighted by Internet user
population (data collected globally is necessarily of variable
quality, but the paper does double-check that the outcome compares
well against a second source). So a lone CUBIC flow would at best
take about 200 round trips (5 s) to recover from each of its
sawtooth reductions, if the flow even lasted that long. This is
described as 'at best' because it assumes everyone uses an AQM,
whereas in reality, most users still have a (probably bloated)
tail-drop buffer.
In the tail-drop case, the likely average recovery
time would be at least 4 times 5 s, if not more, because RTT under load
would be at least double that of an AQM, and the recovery time of Reno-friendly flows depends
on the square of RTT.<a href="#section-5.1-1.14.3" class="pilcrow">¶</a></p>
<p id="section-5.1-1.14.4">Although work on
scaling congestion controls tends to start with TCP as the
transport, the above is not intended to exclude other transports
(e.g., SCTP and QUIC) or less elastic algorithms
(e.g., RMCAT), which all tend to adopt the same or similar
developments.<a href="#section-5.1-1.14.4" class="pilcrow">¶</a></p>
</dd>
<dd class="break"></dd>
</dl>
</section>
</div>
<div id="l4sps_why-not">
<section id="section-5.2">
<h3 id="name-what-l4s-adds-to-existing-a">
<a href="#section-5.2" class="section-number selfRef">5.2. </a><a href="#name-what-l4s-adds-to-existing-a" class="section-name selfRef">What L4S Adds to Existing Approaches</a>
</h3>
<p id="section-5.2-1">All the following approaches address some part of the same problem
space as L4S. In each case, it is shown that L4S complements them or
improves on them, rather than being a mutually exclusive
alternative:<a href="#section-5.2-1" class="pilcrow">¶</a></p>
<span class="break"></span><dl class="dlParallel" id="section-5.2-2">
<dt id="section-5.2-2.1">Diffserv:</dt>
<dd style="margin-left: 1.5em" id="section-5.2-2.2">
<p id="section-5.2-2.2.1">Diffserv addresses the problem of
bandwidth apportionment for important traffic as well as queuing
latency for delay-sensitive traffic. Of these, L4S solely
addresses the problem of queuing latency. Diffserv will still be
necessary where important traffic requires priority (e.g., for
commercial reasons or for protection of critical infrastructure
traffic) -- see <span>[<a href="#I-D.briscoe-tsvwg-l4s-diffserv" class="cite xref">L4S-DIFFSERV</a>]</span>.
Nonetheless, the L4S approach can provide low latency for all
traffic within each Diffserv class (including the case where there
is only the one default Diffserv class).<a href="#section-5.2-2.2.1" class="pilcrow">¶</a></p>
<p id="section-5.2-2.2.2">Also, Diffserv can only provide a latency benefit
if a small subset of the traffic on a bottleneck link requests low
latency. As already explained, it has no effect when all the
applications in use at one time at a single site (e.g., a home, small
business, or mobile device) require low latency. In contrast,
because L4S works for all traffic, it needs none of the management
baggage (traffic policing or traffic contracts) associated with
favouring some packets over others. This lack of management
baggage ought to give L4S a better chance of end-to-end
deployment.<a href="#section-5.2-2.2.2" class="pilcrow">¶</a></p>
<p id="section-5.2-2.2.3">In particular, if networks do not trust end systems to identify which
packets should be favoured, they assign packets to Diffserv classes
themselves. However, the techniques available to such networks, like
inspection of flow identifiers or deeper inspection of application
signatures, do not always sit well with encryption of the layers above
IP <span>[<a href="#RFC8404" class="cite xref">RFC8404</a>]</span>. In these cases, users
can have either privacy or Quality of Service (QoS), but not both.<a href="#section-5.2-2.2.3" class="pilcrow">¶</a></p>
<p id="section-5.2-2.2.4">As with Diffserv,
the L4S identifier is in the IP header. But, in contrast to
Diffserv, the L4S identifier does not convey a want or a need for
a certain level of quality. Rather, it promises a certain
behaviour (Scalable congestion response), which networks can
objectively verify if they need to. This is because low delay
depends on collective host behaviour, whereas bandwidth priority
depends on network behaviour.<a href="#section-5.2-2.2.4" class="pilcrow">¶</a></p>
</dd>
<dd class="break"></dd>
<dt id="section-5.2-2.3">State-of-the-art AQMs:</dt>
<dd style="margin-left: 1.5em" id="section-5.2-2.4">AQMs for Classic traffic, such as PIE and FQ-CoDel,
give a significant reduction in queuing delay relative to no AQM
at all. L4S is intended to complement these AQMs and should not
distract from the need to deploy them as widely as possible.
Nonetheless, AQMs alone cannot reduce queuing delay too far
without significantly reducing link utilization, because the root
cause of the problem is on the host -- where Classic congestion
controls use large sawtoothing rate variations. The L4S approach
resolves this tension between delay and utilization by enabling
hosts to minimize the amplitude of their sawteeth. A single-queue
Classic AQM is not sufficient to allow hosts to use small sawteeth
for two reasons: i) smaller sawteeth would not get lower delay in
an AQM designed for larger amplitude Classic sawteeth, because a
queue can only have one length at a time and ii) much smaller
sawteeth implies much more frequent sawteeth, so L4S flows would
drive a Classic AQM into a high level of ECN-marking, which would
appear as heavy congestion to Classic flows, which in turn would
greatly reduce their rate as a result (see <a href="#l4sarch_sec_classic-ecn-neck" class="auto internal xref">Section 6.4.4</a>).<a href="#section-5.2-2.4" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-5.2-2.5">Per-flow queuing or marking:</dt>
<dd style="margin-left: 1.5em" id="section-5.2-2.6">
<p id="section-5.2-2.6.1">Similarly, per-flow
approaches, such as FQ-CoDel or Approx Fair CoDel <span>[<a href="#AFCD" class="cite xref">AFCD</a>]</span>, are not incompatible with the L4S approach.
However, per-flow queuing alone is not enough -- it only isolates
the queuing of one flow from others, not from itself. Per-flow
implementations need to have support for Scalable congestion
control added, which has already been done for FQ-CoDel in Linux
(see <span><a href="https://www.rfc-editor.org/rfc/rfc8290#section-5.2.7" class="relref">Section 5.2.7</a> of [<a href="#RFC8290" class="cite xref">RFC8290</a>]</span> and <span>[<a href="#FQ_CoDel_Thresh" class="cite xref">FQ_CoDel_Thresh</a>]</span>). Without this simple modification,
per-flow AQMs, like FQ-CoDel, would still not be able to support
applications that need both very low delay and high bandwidth,
e.g., video-based control of remote procedures or interactive
cloud-based video (see Note <a href="#l4sarch_note_app_shuffle" class="auto internal xref">1</a> below).<a href="#section-5.2-2.6.1" class="pilcrow">¶</a></p>
<p id="section-5.2-2.6.2">Although per-flow techniques are not incompatible
with L4S, it is important to have the DualQ alternative. This is
because handling end-to-end (layer 4) flows in the network (layer
3 or 2) precludes some important end-to-end functions. For
instance:<a href="#section-5.2-2.6.2" class="pilcrow">¶</a></p>
<ol start="1" type="a" class="normal type-a" id="section-5.2-2.6.3">
<li id="section-5.2-2.6.3.1">
<p id="section-5.2-2.6.3.1.1">Per-flow forms of L4S, like FQ-CoDel, are incompatible with
full end-to-end encryption of transport layer identifiers for
privacy and confidentiality (e.g., IPsec or encrypted VPN
tunnels, as opposed to DTLS over UDP), because they require
packet inspection to access the end-to-end transport flow
identifiers.<a href="#section-5.2-2.6.3.1.1" class="pilcrow">¶</a></p>
<p id="section-5.2-2.6.3.1.2">In contrast, the DualQ
form of L4S requires no deeper inspection than the IP layer.
So as long as operators take the DualQ approach, their users
can have both very low queuing delay and full end-to-end
encryption <span>[<a href="#RFC8404" class="cite xref">RFC8404</a>]</span>.<a href="#section-5.2-2.6.3.1.2" class="pilcrow">¶</a></p>
</li>
<li id="section-5.2-2.6.3.2">
<p id="section-5.2-2.6.3.2.1">With per-flow forms of L4S, the network takes over control of
the relative rates of each application flow. Some see it as
an advantage that the network will prevent some flows running
faster than others. Others consider it an inherent part of the
Internet's appeal that applications can control their rate
while taking account of the needs of others via congestion
signals.
They maintain that this has allowed applications with
interesting rate behaviours to evolve, for instance: i) a variable
bit-rate video that varies around an equal share, rather than
being forced to remain equal at every instant or ii) end-to-end
scavenger behaviours <span>[<a href="#RFC6817" class="cite xref">RFC6817</a>]</span> that use
less than an equal share of capacity <span>[<a href="#LEDBAT_AQM" class="cite xref">LEDBAT_AQM</a>]</span>.<a href="#section-5.2-2.6.3.2.1" class="pilcrow">¶</a></p>
<p id="section-5.2-2.6.3.2.2">The L4S
architecture does not require the IETF to commit to one
approach over the other, because it supports both so that the
'market' can decide. Nonetheless, in the spirit of 'Do one
thing and do it well' <span>[<a href="#McIlroy78" class="cite xref">McIlroy78</a>]</span>, the
DualQ option provides low delay without prejudging the issue
of flow-rate control. Then, flow rate policing can be added
separately if desired. In contrast to scheduling, a policer would allow application control up to a
point, but the network would still be able to set the point at
which it intervened to prevent one flow completely starving
another.<a href="#section-5.2-2.6.3.2.2" class="pilcrow">¶</a></p>
</li>
</ol>
<p id="section-5.2-2.6.4">Note:<a href="#section-5.2-2.6.4" class="pilcrow">¶</a></p>
<ol start="1" type="1" class="normal type-1" id="section-5.2-2.6.5">
<li id="section-5.2-2.6.5.1">
<div id="l4sarch_note_app_shuffle">It might seem that
self-inflicted queuing delay within a per-flow queue should
not be counted, because if the delay wasn't in the network, it
would just shift to the sender. However, modern adaptive
applications, e.g., HTTP/2 <span>[<a href="#RFC9113" class="cite xref">RFC9113</a>]</span>
or some interactive media applications (see <a href="#l4sarch_apps" class="auto internal xref">Section 6.1</a>), can keep low latency objects at the
front of their local send queue by shuffling priorities of
other objects dependent on the progress of other transfers
(for example, see <span>[<a href="#lowat" class="cite xref">lowat</a>]</span>). They cannot shuffle
objects once they have released them into the network.<a href="#l4sarch_note_app_shuffle" class="pilcrow">¶</a>
</div>
</li>
</ol>
</dd>
<dd class="break"></dd>
<dt id="section-5.2-2.7">Alternative Back-off ECN (ABE):</dt>
<dd style="margin-left: 1.5em" id="section-5.2-2.8">Here again, L4S is
not an alternative to ABE but a complement that introduces much
lower queuing delay. ABE <span>[<a href="#RFC8511" class="cite xref">RFC8511</a>]</span> alters the
host behaviour in response to ECN marking to utilize a link better
and give ECN flows faster throughput. It uses ECT(0) and assumes
the network still treats ECN and drop the same. Therefore, ABE
exploits any lower queuing delay that AQMs can provide. But, as
explained above, AQMs still cannot reduce queuing delay too much
without losing link utilization (to allow for other, non-ABE,
flows).<a href="#section-5.2-2.8" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-5.2-2.9">BBR:</dt>
<dd style="margin-left: 1.5em" id="section-5.2-2.10">
<p id="section-5.2-2.10.1">Bottleneck Bandwidth and Round-trip propagation
time (BBR) <span>[<a href="#I-D.cardwell-iccrg-bbr-congestion-control" class="cite xref">BBR-CC</a>]</span> controls
queuing delay end-to-end without needing any special logic in the
network, such as an AQM. So it works pretty much on any path. BBR
keeps queuing delay reasonably low, but perhaps not quite as low
as with state-of-the-art AQMs, such as PIE or FQ-CoDel, and
certainly nowhere near as low as with L4S. Queuing delay is also
not consistently low, due to BBR's regular bandwidth probing
spikes and its aggressive flow start-up phase.<a href="#section-5.2-2.10.1" class="pilcrow">¶</a></p>
<p id="section-5.2-2.10.2">L4S complements BBR. Indeed, BBRv2 can use L4S ECN
where available and a Scalable L4S congestion control behaviour in
response to any ECN signalling from the path <span>[<a href="#BBRv2" class="cite xref">BBRv2</a>]</span>. The L4S ECN signal complements the delay-based
congestion control aspects of BBR with an explicit indication that
hosts can use, both to converge on a fair rate and to keep below a
shallow queue target set by the network. Without L4S ECN, both
these aspects need to be assumed or estimated.<a href="#section-5.2-2.10.2" class="pilcrow">¶</a></p>
</dd>
<dd class="break"></dd>
</dl>
</section>
</div>
</section>
</div>
<div id="l4sarch_applicability">
<section id="section-6">
<h2 id="name-applicability">
<a href="#section-6" class="section-number selfRef">6. </a><a href="#name-applicability" class="section-name selfRef">Applicability</a>
</h2>
<div id="l4sarch_apps">
<section id="section-6.1">
<h3 id="name-applications">
<a href="#section-6.1" class="section-number selfRef">6.1. </a><a href="#name-applications" class="section-name selfRef">Applications</a>
</h3>
<p id="section-6.1-1">A transport layer that solves the current latency issues will
provide new service, product, and application opportunities.<a href="#section-6.1-1" class="pilcrow">¶</a></p>
<p id="section-6.1-2">With the L4S approach, the following existing applications also
experience significantly better quality of experience under load:<a href="#section-6.1-2" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-6.1-3.1">gaming, including cloud-based gaming;<a href="#section-6.1-3.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.1-3.2">VoIP;<a href="#section-6.1-3.2" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.1-3.3">video conferencing;<a href="#section-6.1-3.3" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.1-3.4">web browsing;<a href="#section-6.1-3.4" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.1-3.5">(adaptive) video streaming; and<a href="#section-6.1-3.5" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.1-3.6">instant messaging.<a href="#section-6.1-3.6" class="pilcrow">¶</a>
</li>
</ul>
<p id="section-6.1-4">The significantly lower queuing latency also enables some
interactive application functions to be offloaded to the cloud that
would hardly even be usable today, including:<a href="#section-6.1-4" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-6.1-5.1">cloud-based interactive video and<a href="#section-6.1-5.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.1-5.2">cloud-based virtual and augmented reality.<a href="#section-6.1-5.2" class="pilcrow">¶</a>
</li>
</ul>
<p id="section-6.1-6">The above two applications have been successfully demonstrated with
L4S, both running together over a 40 Mb/s broadband access link
loaded up with the numerous other latency-sensitive applications in
the previous list, as well as numerous downloads, with all sharing the same
bottleneck queue simultaneously <span>[<a href="#L4Sdemo16" class="cite xref">L4Sdemo16</a>]</span> <span>[<a href="#L4Sdemo16-Video" class="cite xref">L4Sdemo16-Video</a>]</span>. For
the former, a panoramic video of a football stadium could be swiped
and pinched so that, on the fly, a proxy in the cloud could generate a
sub-window of the match video under the finger-gesture control of each
user. For the latter, a virtual reality headset displayed a viewport
taken from a 360-degree camera in a racing car. The user's head
movements controlled the viewport extracted by a cloud-based proxy. In
both cases, with a 7 ms end-to-end base delay, the additional
queuing delay of roughly 1 ms was so low that it seemed the video
was generated locally.<a href="#section-6.1-6" class="pilcrow">¶</a></p>
<p id="section-6.1-7">Using a swiping finger gesture or head movement to pan a video are
extremely latency-demanding actions -- far more demanding than
VoIP -- because human vision can detect extremely low delays of the
order of single milliseconds when delay is translated into a visual
lag between a video and a reference point (the finger or the
orientation of the head sensed by the balance system in the inner ear,
i.e., the vestibular system). With an alternative AQM, the video
noticeably lagged behind the finger gestures and head movements.<a href="#section-6.1-7" class="pilcrow">¶</a></p>
<p id="section-6.1-8">Without the low queuing delay of L4S, cloud-based applications like
these would not be credible without significantly more access-network bandwidth
(to deliver all possible areas of the video that might be viewed) and
more local processing, which would increase the weight and power
consumption of head-mounted displays. When all interactive processing
can be done in the cloud, only the data to be rendered for the end
user needs to be sent.<a href="#section-6.1-8" class="pilcrow">¶</a></p>
<p id="section-6.1-9">Other low latency high bandwidth applications, such as:<a href="#section-6.1-9" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-6.1-10.1">interactive remote presence and<a href="#section-6.1-10.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.1-10.2">video-assisted remote control of machinery or industrial
processes<a href="#section-6.1-10.2" class="pilcrow">¶</a>
</li>
</ul>
<p id="section-6.1-11">are not credible at all without very low queuing delay. No
amount of extra access bandwidth or local processing can make up for
lost time.<a href="#section-6.1-11" class="pilcrow">¶</a></p>
</section>
</div>
<section id="section-6.2">
<h3 id="name-use-cases">
<a href="#section-6.2" class="section-number selfRef">6.2. </a><a href="#name-use-cases" class="section-name selfRef">Use Cases</a>
</h3>
<p id="section-6.2-1">The following use cases for L4S are being considered by various
interested parties:<a href="#section-6.2-1" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-6.2-2.1">where the bottleneck is one of various types of access network,
e.g., DSL, Passive Optical Networks (PONs), DOCSIS cable,
mobile, satellite; or where it's a Wi-Fi link (see <a href="#l4sarch_link-specifics" class="auto internal xref">Section 6.3</a> for
some technology-specific details)<a href="#section-6.2-2.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.2-2.2">
<p id="section-6.2-2.2.1">private networks of heterogeneous data centres, where there is
no single administrator that can arrange for all the simultaneous
changes to senders, receivers, and networks needed to deploy
DCTCP:<a href="#section-6.2-2.2.1" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-6.2-2.2.2.1">a set of private data centres interconnected over a wide
area with separate administrations but within the same
company<a href="#section-6.2-2.2.2.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.2-2.2.2.2">a set of data centres operated by separate companies
interconnected by a community of interest network
(e.g., for the finance sector)<a href="#section-6.2-2.2.2.2" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.2-2.2.2.3">multi-tenant (cloud) data centres where tenants choose
their operating system stack (Infrastructure as a Service
(IaaS))<a href="#section-6.2-2.2.2.3" class="pilcrow">¶</a>
</li>
</ul>
</li>
<li class="normal" id="section-6.2-2.3">
<p id="section-6.2-2.3.1">different types of transport (or application) congestion
control:<a href="#section-6.2-2.3.1" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-6.2-2.3.2.1">elastic (TCP/SCTP);<a href="#section-6.2-2.3.2.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.2-2.3.2.2">real-time (RTP, RMCAT); and<a href="#section-6.2-2.3.2.2" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.2-2.3.2.3">query-response (DNS/LDAP).<a href="#section-6.2-2.3.2.3" class="pilcrow">¶</a>
</li>
</ul>
</li>
<li class="normal" id="section-6.2-2.4">
<p id="section-6.2-2.4.1">where low delay QoS is required but without
inspecting or intervening above the IP layer <span>[<a href="#RFC8404" class="cite xref">RFC8404</a>]</span>:<a href="#section-6.2-2.4.1" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-6.2-2.4.2.1">Mobile and other networks have tended to inspect higher
layers in order to guess application QoS requirements.
However, with growing demand for support of privacy and
encryption, L4S offers an alternative. There is no need to
select which traffic to favour for queuing when L4S can give
favourable queuing to all traffic.<a href="#section-6.2-2.4.2.1" class="pilcrow">¶</a>
</li>
</ul>
</li>
<li class="normal" id="section-6.2-2.5">If queuing delay is minimized, applications with a fixed delay
budget can communicate over longer distances or via more circuitous paths, e.g., longer
chains of service functions <span>[<a href="#RFC7665" class="cite xref">RFC7665</a>]</span> or of onion
routers.<a href="#section-6.2-2.5" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.2-2.6">If delay jitter is minimized, it is possible to reduce the
dejitter buffers on the receiving end of video streaming, which
should improve the interactive experience.<a href="#section-6.2-2.6" class="pilcrow">¶</a>
</li>
</ul>
</section>
<div id="l4sarch_link-specifics">
<section id="section-6.3">
<h3 id="name-applicability-with-specific">
<a href="#section-6.3" class="section-number selfRef">6.3. </a><a href="#name-applicability-with-specific" class="section-name selfRef">Applicability with Specific Link Technologies</a>
</h3>
<p id="section-6.3-1">Certain link technologies aggregate data from multiple packets into
bursts and buffer incoming packets while building each burst. Wi-Fi,
PON, and cable all involve such packet aggregation, whereas fixed
Ethernet and DSL do not. No sender, whether L4S or not, can do
anything to reduce the buffering needed for packet aggregation. So an
AQM should not count this buffering as part of the queue that it
controls, given no amount of congestion signals will reduce it.<a href="#section-6.3-1" class="pilcrow">¶</a></p>
<p id="section-6.3-2">Certain link technologies also add buffering for other reasons,
specifically:<a href="#section-6.3-2" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-6.3-3.1">Radio links (cellular, Wi-Fi, or satellite) that are distant from
the source are particularly challenging. The radio link capacity
can vary rapidly by orders of magnitude, so it is considered
desirable to hold a standing queue that can utilize sudden
increases of capacity.<a href="#section-6.3-3.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.3-3.2">Cellular networks are further complicated by a perceived need
to buffer in order to make hand-overs imperceptible.<a href="#section-6.3-3.2" class="pilcrow">¶</a>
</li>
</ul>
<p id="section-6.3-4">L4S cannot remove the need for all these different forms of
buffering. However, by removing 'the longest pole in the tent'
(buffering for the large sawteeth of Classic congestion controls), L4S
exposes all these 'shorter poles' to greater scrutiny.<a href="#section-6.3-4" class="pilcrow">¶</a></p>
<p id="section-6.3-5">Until now, the buffering needed for these additional reasons tended
to be over-specified -- with the excuse that none were 'the longest
pole in the tent'. But having removed the 'longest pole', it becomes
worthwhile to minimize them, for instance, reducing packet aggregation
burst sizes and MAC scheduling intervals.<a href="#section-6.3-5" class="pilcrow">¶</a></p>
<p id="section-6.3-6">Also, certain link types, particularly radio-based links, are far
more prone to transmission losses. <a href="#l4sarch_sec_non-l4s-neck" class="auto internal xref">Section 6.4.3</a> explains how an L4S response to
loss has to be as drastic as a Classic response. Nonetheless, research
referred to in the same section has demonstrated potential for
considerably more effective loss repair at the link layer, due to the
relaxed ordering constraints of L4S packets.<a href="#section-6.3-6" class="pilcrow">¶</a></p>
</section>
</div>
<section id="section-6.4">
<h3 id="name-deployment-considerations">
<a href="#section-6.4" class="section-number selfRef">6.4. </a><a href="#name-deployment-considerations" class="section-name selfRef">Deployment Considerations</a>
</h3>
<p id="section-6.4-1">L4S AQMs, whether DualQ <span>[<a href="#RFC9332" class="cite xref">RFC9332</a>]</span> or FQ <span>[<a href="#RFC8290" class="cite xref">RFC8290</a>]</span>, are in themselves an incremental deployment
mechanism for L4S -- so that L4S traffic can coexist with existing
Classic (Reno-friendly) traffic. <a href="#l4sarch_deploy_top" class="auto internal xref">Section 6.4.1</a>
explains why only deploying an L4S AQM in one node at each end of the
access link will realize nearly all the benefit of L4S.<a href="#section-6.4-1" class="pilcrow">¶</a></p>
<p id="section-6.4-2">L4S involves both the network and end systems, so <a href="#l4s_arch_deploy_seq" class="auto internal xref">Section 6.4.2</a> suggests some typical sequences to
deploy each part and why there will be an immediate and significant
benefit after deploying just one part.<a href="#section-6.4-2" class="pilcrow">¶</a></p>
<p id="section-6.4-3">Sections <a href="#l4sarch_sec_non-l4s-neck" class="auto internal xref">6.4.3</a> and <a href="#l4sarch_sec_classic-ecn-neck" class="auto internal xref">6.4.4</a> describe the converse
incremental deployment case where there is no L4S AQM at the network
bottleneck, so any L4S flow traversing this bottleneck has to take
care in case it is competing with Classic traffic.<a href="#section-6.4-3" class="pilcrow">¶</a></p>
<div id="l4sarch_deploy_top">
<section id="section-6.4.1">
<h4 id="name-deployment-topology">
<a href="#section-6.4.1" class="section-number selfRef">6.4.1. </a><a href="#name-deployment-topology" class="section-name selfRef">Deployment Topology</a>
</h4>
<p id="section-6.4.1-1">L4S AQMs will not have to be deployed throughout the Internet
before L4S can benefit anyone. Operators of public Internet access
networks typically design their networks so that the bottleneck will
nearly always occur at one known (logical) link. This confines the
cost of queue management technology to one place.<a href="#section-6.4.1-1" class="pilcrow">¶</a></p>
<p id="section-6.4.1-2">The case of mesh networks is different and will be discussed
later in this section.
However, the known-bottleneck case is generally
true for Internet access to all sorts of different 'sites', where
the word 'site' includes home networks, small- to medium-sized
campus or enterprise networks and even cellular devices (<a href="#l4sarch_fig_access_topology" class="auto internal xref">Figure 2</a>).
Also, this known-bottleneck
case tends to be applicable whatever the access link technology,
whether xDSL, cable, PON, cellular, line of sight wireless, or
satellite.<a href="#section-6.4.1-2" class="pilcrow">¶</a></p>
<p id="section-6.4.1-3">Therefore, the full benefit of the L4S service should be
available in the downstream direction when an L4S AQM is deployed at
the ingress to this bottleneck link. And similarly, the full
upstream service will typically be available once an L4S AQM is deployed at
the ingress into the upstream link. (Of course, multihomed sites
would only see the full benefit once all their access links were
covered.)<a href="#section-6.4.1-3" class="pilcrow">¶</a></p>
<span id="name-likely-location-of-dualq-dq"></span><div id="l4sarch_fig_access_topology">
<figure id="figure-2">
<div class="alignLeft art-text artwork" id="section-6.4.1-4.1">
<pre>
______
( )
__ __ ( )
|DQ\________/DQ|( enterprise )
___ |__/ \__| ( /campus )
( ) (______)
( ) ___||_
+----+ ( ) __ __ / \
| DC |-----( Core )|DQ\_______________/DQ|| home |
+----+ ( ) |__/ \__||______|
(_____) __
|DQ\__/\ __ ,===.
|__/ \ ____/DQ||| ||mobile
\/ \__|||_||device
| o |
`---'
</pre>
</div>
<figcaption><a href="#figure-2" class="selfRef">Figure 2</a>:
<a href="#name-likely-location-of-dualq-dq" class="selfRef">Likely Location of DualQ (DQ) Deployments in Common Access Topologies</a>
</figcaption></figure>
</div>
<p id="section-6.4.1-5">Deployment in mesh topologies depends on how overbooked the core
is. If the core is non-blocking, or at least generously provisioned
so that the edges are nearly always the bottlenecks, it would only
be necessary to deploy an L4S AQM at the edge bottlenecks.
For
example, some data-centre networks are designed with the bottleneck
in the hypervisor or host Network Interface Controllers (NICs), while others
bottleneck at the
top-of-rack switch (both the output ports facing hosts and those
facing the core).<a href="#section-6.4.1-5" class="pilcrow">¶</a></p>
<p id="section-6.4.1-6">An L4S AQM would often next be needed where the Wi-Fi links in a
home sometimes become the bottleneck. Also an L4S AQM would
eventually need to be deployed at any other persistent
bottlenecks, such as network interconnections, e.g., some public
Internet exchange points and the ingress and egress to WAN links
interconnecting data centres.<a href="#section-6.4.1-6" class="pilcrow">¶</a></p>
</section>
</div>
<div id="l4s_arch_deploy_seq">
<section id="section-6.4.2">
<h4 id="name-deployment-sequences">
<a href="#section-6.4.2" class="section-number selfRef">6.4.2. </a><a href="#name-deployment-sequences" class="section-name selfRef">Deployment Sequences</a>
</h4>
<p id="section-6.4.2-1">For any one L4S flow to provide benefit, it requires three (or
sometimes two) parts to have been deployed: i) the congestion
control at the sender; ii) the AQM at the bottleneck; and iii) older
transports (namely TCP) need upgraded receiver feedback too. This
was the same deployment problem that ECN faced <span>[<a href="#RFC8170" class="cite xref">RFC8170</a>]</span>, so we have learned from that experience.<a href="#section-6.4.2-1" class="pilcrow">¶</a></p>
<p id="section-6.4.2-2">Firstly, L4S deployment exploits the fact that DCTCP already
exists on many Internet hosts (e.g., Windows, FreeBSD, and Linux), both
servers and clients. Therefore, an L4S AQM can be deployed at a
network bottleneck to immediately give a working deployment of all
the L4S parts for testing, as long as the ECT(0) codepoint is
switched to ECT(1). DCTCP needs some safety concerns to be fixed for
general use over the public Internet (see <span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-4.3" class="relref">Section 4.3</a> of the L4S ECN spec [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>), but DCTCP is
not on by default, so these issues can be managed within controlled
deployments or controlled trials.<a href="#section-6.4.2-2" class="pilcrow">¶</a></p>
<p id="section-6.4.2-3">Secondly, the performance improvement with L4S is so significant
that it enables new interactive services and products that were not
previously possible. It is much easier for companies to initiate new
work on deployment if there is budget for a new product trial.
In contrast, if there were only an incremental performance improvement
(as with Classic ECN), spending on deployment tends to be much
harder to justify.<a href="#section-6.4.2-3" class="pilcrow">¶</a></p>
<p id="section-6.4.2-4">Thirdly, the L4S identifier is defined so that network
operators can initially enable L4S exclusively for certain customers or
certain applications. However, this is carefully defined so that it does
not compromise future evolution towards L4S as an Internet-wide
service. This is because the L4S identifier is defined not only as
the end-to-end ECN field, but it can also optionally be combined
with any other packet header or some status of a customer or their
access link (see <span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-5.4" class="relref">Section 5.4</a> of [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>). Operators could do this
anyway, even if it were not blessed by the IETF. However, it is best
for the IETF to specify that, if they use their own local
identifier, it must be in combination with the IETF's identifier, ECT(1).
Then, if an operator has opted for an exclusive local-use approach,
they only have to remove this extra rule later to make the service
work across the Internet -- it will already traverse middleboxes, peerings,
etc.<a href="#section-6.4.2-4" class="pilcrow">¶</a></p>
<span id="name-example-l4s-deployment-sequ"></span><div id="l4s_arch_fig_deploy_seq">
<figure id="figure-3">
<div class="alignLeft art-text artwork" id="section-6.4.2-5.1">
<pre>
+-+--------------------+----------------------+---------------------+
| | Servers or proxies | Access link | Clients |
+-+--------------------+----------------------+---------------------+
|0| DCTCP (existing) | | DCTCP (existing) |
+-+--------------------+----------------------+---------------------+
|1| |Add L4S AQM downstream| |
| | WORKS DOWNSTREAM FOR CONTROLLED DEPLOYMENTS/TRIALS |
+-+--------------------+----------------------+---------------------+
|2| Upgrade DCTCP to | |Replace DCTCP feedb'k|
| | TCP Prague | | with AccECN |
| | FULLY WORKS DOWNSTREAM |
+-+--------------------+----------------------+---------------------+
| | | | Upgrade DCTCP to |
|3| | Add L4S AQM upstream | TCP Prague |
| | | | |
| | FULLY WORKS UPSTREAM AND DOWNSTREAM |
+-+--------------------+----------------------+---------------------+
</pre>
</div>
<figcaption><a href="#figure-3" class="selfRef">Figure 3</a>:
<a href="#name-example-l4s-deployment-sequ" class="selfRef">Example L4S Deployment Sequence</a>
</figcaption></figure>
</div>
<p id="section-6.4.2-6"><a href="#l4s_arch_fig_deploy_seq" class="auto internal xref">Figure 3</a> illustrates some example
sequences in which the parts of L4S might be deployed. It consists
of the following stages, preceded by a presumption that DCTCP is
already installed at both ends:<a href="#section-6.4.2-6" class="pilcrow">¶</a></p>
<ol start="1" type="1" class="normal type-1" id="section-6.4.2-7">
<li id="section-6.4.2-7.1">
<p id="section-6.4.2-7.1.1">DCTCP is not applicable for use over the public Internet, so
it is emphasized here that any DCTCP flow has to be completely
contained within a controlled trial environment.<a href="#section-6.4.2-7.1.1" class="pilcrow">¶</a></p>
<p id="section-6.4.2-7.1.2">Within this trial environment, once an L4S AQM
has been deployed, the trial DCTCP flow will experience
immediate benefit, without any other deployment being needed. In
this example, downstream deployment is first, but in other
scenarios, the upstream might be deployed first. If no AQM at all
was previously deployed for the downstream access, an L4S AQM
greatly improves the Classic service (as well as adding the L4S
service). If an AQM was already deployed, the Classic service
will be unchanged (and L4S will add an improvement on top).<a href="#section-6.4.2-7.1.2" class="pilcrow">¶</a></p>
</li>
<li id="section-6.4.2-7.2">
<p id="section-6.4.2-7.2.1">In this stage, the name 'TCP Prague' <span>[<a href="#I-D.briscoe-iccrg-prague-congestion-control" class="cite xref">PRAGUE-CC</a>]</span> is used
to represent a variant of DCTCP that is designed to be used in a
production Internet environment (that is, it has to comply with
all the requirements in <span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-4" class="relref">Section 4</a> of the L4S ECN spec [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>, which then means it can be
used over the public Internet). If the application is primarily
unidirectional, 'TCP Prague' at the sending end will provide all
the benefit needed, as long as the receiving end supports Accurate ECN (AccECN)
feedback <span>[<a href="#I-D.ietf-tcpm-accurate-ecn" class="cite xref">ACCECN</a>]</span>.<a href="#section-6.4.2-7.2.1" class="pilcrow">¶</a></p>
<p id="section-6.4.2-7.2.2">For TCP transports,
AccECN feedback is needed at the other
end, but it is a generic ECN feedback facility that is already
planned to be deployed for other purposes, e.g., DCTCP and BBR.
The two ends can be deployed in either order because, in TCP,
an L4S congestion control only enables itself if it has
negotiated the use of AccECN feedback with the other end during
the connection handshake. Thus, deployment of TCP Prague on a
server enables L4S trials to move to a production service in one
direction, wherever AccECN is deployed at the other end. This
stage might be further motivated by the performance improvements
of TCP Prague relative to DCTCP (see <span><a href="https://www.rfc-editor.org/rfc/rfc9331#appendix-A.2" class="relref">Appendix A.2</a> of the L4S ECN spec [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>).<a href="#section-6.4.2-7.2.2" class="pilcrow">¶</a></p>
<p id="section-6.4.2-7.2.3">Unlike TCP, from the outset, QUIC ECN
feedback <span>[<a href="#RFC9000" class="cite xref">RFC9000</a>]</span> has supported L4S.
Therefore, if the transport is QUIC, one-ended deployment of a
Prague congestion control at this stage is simple and
sufficient.<a href="#section-6.4.2-7.2.3" class="pilcrow">¶</a></p>
<p id="section-6.4.2-7.2.4">For QUIC, if a proxy sits in
the path between multiple origin servers and the access
bottlenecks to multiple clients, then upgrading the proxy with a
Scalable congestion control would provide the benefits of L4S
over all the clients' downstream bottlenecks in one go --
whether or not all the origin servers were upgraded. Conversely,
where a proxy has not been upgraded, the clients served by it
will not benefit from L4S at all in the downstream, even when
any origin server behind the proxy has been upgraded to support
L4S.<a href="#section-6.4.2-7.2.4" class="pilcrow">¶</a></p>
<p id="section-6.4.2-7.2.5">For TCP, a proxy upgraded to support
'TCP Prague' would provide the benefits of L4S downstream to all
clients that support AccECN (whether or not they support L4S as
well). And in the upstream, the proxy would also support AccECN
as a receiver, so that any client deploying its own L4S support
would benefit in the upstream direction, irrespective of whether
any origin server beyond the proxy supported AccECN.<a href="#section-6.4.2-7.2.5" class="pilcrow">¶</a></p>
</li>
<li id="section-6.4.2-7.3">This is a two-move stage to enable L4S upstream. An L4S AQM
or TCP Prague can be deployed in either order as already
explained. To motivate the first of two independent moves, the
deferred benefit of enabling new services after the second move
has to be worth it to cover the first mover's investment risk.
As explained already, the potential for new interactive services
provides this motivation. An L4S AQM also improves the upstream
Classic service significantly if no other AQM has already been
deployed.<a href="#section-6.4.2-7.3" class="pilcrow">¶</a>
</li>
</ol>
<p id="section-6.4.2-8">Note that other deployment sequences might occur. For
instance, the upstream might be deployed first; a non-TCP protocol
might be used end to end, e.g., QUIC and RTP; a body, such as the
3GPP, might require L4S to be implemented in 5G user equipment; or
other random acts of kindness might arise.<a href="#section-6.4.2-8" class="pilcrow">¶</a></p>
</section>
</div>
<div id="l4sarch_sec_non-l4s-neck">
<section id="section-6.4.3">
<h4 id="name-l4s-flow-but-non-ecn-bottle">
<a href="#section-6.4.3" class="section-number selfRef">6.4.3. </a><a href="#name-l4s-flow-but-non-ecn-bottle" class="section-name selfRef">L4S Flow but Non-ECN Bottleneck</a>
</h4>
<p id="section-6.4.3-1">If L4S is enabled between two hosts, the L4S sender is required
to coexist safely with Reno in response to any drop (see <span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-4.3" class="relref">Section 4.3</a> of the L4S ECN spec [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>).<a href="#section-6.4.3-1" class="pilcrow">¶</a></p>
<p id="section-6.4.3-2">Unfortunately, as well as protecting Classic traffic, this rule
degrades the L4S service whenever there is any loss, even if the
cause is not persistent congestion at a bottleneck, for example:<a href="#section-6.4.3-2" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-6.4.3-3.1">congestion loss at other transient bottlenecks, e.g., due
to bursts in shallower queues;<a href="#section-6.4.3-3.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.4.3-3.2">transmission errors, e.g., due to electrical
interference; and<a href="#section-6.4.3-3.2" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.4.3-3.3">rate policing.<a href="#section-6.4.3-3.3" class="pilcrow">¶</a>
</li>
</ul>
<p id="section-6.4.3-4">Three complementary approaches are in progress to address this
issue, but they are all currently research:<a href="#section-6.4.3-4" class="pilcrow">¶</a></p>
<ul class="normal">
<li class="normal" id="section-6.4.3-5.1">In Prague congestion control, ignore certain losses deemed
unlikely to be due to congestion (using some ideas from
BBR <span>[<a href="#I-D.cardwell-iccrg-bbr-congestion-control" class="cite xref">BBR-CC</a>]</span> regarding
isolated losses). This could mask any of the above types of loss
while still coexisting with drop-based congestion controls.<a href="#section-6.4.3-5.1" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.4.3-5.2">A combination of Recent Acknowledgement (RACK) <span>[<a href="#RFC8985" class="cite xref">RFC8985</a>]</span>, L4S, and link retransmission without
resequencing could repair transmission errors without the head
of line blocking delay usually associated with link-layer
retransmission <span>[<a href="#UnorderedLTE" class="cite xref">UnorderedLTE</a>]</span> <span>[<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>.<a href="#section-6.4.3-5.2" class="pilcrow">¶</a>
</li>
<li class="normal" id="section-6.4.3-5.3">Hybrid ECN/drop rate policers (see <a href="#l4s_arch_sec_policing" class="auto internal xref">Section 8.3</a>).<a href="#section-6.4.3-5.3" class="pilcrow">¶</a>
</li>
</ul>
<p id="section-6.4.3-6">L4S deployment scenarios that minimize these issues
(e.g., over wireline networks) can proceed in parallel to this
research, in the expectation that research success could continually
widen L4S applicability.<a href="#section-6.4.3-6" class="pilcrow">¶</a></p>
</section>
</div>
<div id="l4sarch_sec_classic-ecn-neck">
<section id="section-6.4.4">
<h4 id="name-l4s-flow-but-classic-ecn-bo">
<a href="#section-6.4.4" class="section-number selfRef">6.4.4. </a><a href="#name-l4s-flow-but-classic-ecn-bo" class="section-name selfRef">L4S Flow but Classic ECN Bottleneck</a>
</h4>
<p id="section-6.4.4-1">Classic ECN support is starting to materialize on the Internet as
an increased level of CE marking. It is hard to detect whether this
is all due to the addition of support for ECN in implementations of
FQ-CoDel and/or FQ-COBALT, which is not generally problematic,
because flow queue (FQ) scheduling inherently prevents a flow from
exceeding the 'fair' rate irrespective of its aggressiveness.
However, some of this Classic ECN marking might be due to
single-queue ECN deployment. This case is discussed in
<span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-4.3" class="relref">Section 4.3</a> of the L4S ECN spec [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>.<a href="#section-6.4.4-1" class="pilcrow">¶</a></p>
</section>
</div>
<section id="section-6.4.5">
<h4 id="name-l4s-aqm-deployment-within-t">
<a href="#section-6.4.5" class="section-number selfRef">6.4.5. </a><a href="#name-l4s-aqm-deployment-within-t" class="section-name selfRef">L4S AQM Deployment within Tunnels</a>
</h4>
<p id="section-6.4.5-1">An L4S AQM uses the ECN field to signal congestion. So in common
with Classic ECN, if the AQM is within a tunnel or at a lower layer,
correct functioning of ECN signalling requires standards-compliant propagation
of the ECN field up the layers <span>[<a href="#RFC6040" class="cite xref">RFC6040</a>]</span> <span>[<a href="#I-D.ietf-tsvwg-rfc6040update-shim" class="cite xref">ECN-SHIM</a>]</span> <span>[<a href="#I-D.ietf-tsvwg-ecn-encap-guidelines" class="cite xref">ECN-ENCAP</a>]</span>.<a href="#section-6.4.5-1" class="pilcrow">¶</a></p>
</section>
</section>
</section>
</div>
<div id="l4sps_IANA">
<section id="section-7">
<h2 id="name-iana-considerations">
<a href="#section-7" class="section-number selfRef">7. </a><a href="#name-iana-considerations" class="section-name selfRef">IANA Considerations</a>
</h2>
<p id="section-7-1">This document has no IANA actions.<a href="#section-7-1" class="pilcrow">¶</a></p>
</section>
</div>
<div id="l4sps_Security_Considerations">
<section id="section-8">
<h2 id="name-security-considerations">
<a href="#section-8" class="section-number selfRef">8. </a><a href="#name-security-considerations" class="section-name selfRef">Security Considerations</a>
</h2>
<section id="section-8.1">
<h3 id="name-traffic-rate-non-policing">
<a href="#section-8.1" class="section-number selfRef">8.1. </a><a href="#name-traffic-rate-non-policing" class="section-name selfRef">Traffic Rate (Non-)Policing</a>
</h3>
<section id="section-8.1.1">
<h4 id="name-non-policing-rate-per-flow">
<a href="#section-8.1.1" class="section-number selfRef">8.1.1. </a><a href="#name-non-policing-rate-per-flow" class="section-name selfRef">(Non-)Policing Rate per Flow</a>
</h4>
<p id="section-8.1.1-1">In the current Internet, ISPs usually enforce separation between
the capacity of shared links assigned to different 'sites'
(e.g., households, businesses, or mobile users -- see terminology
in <a href="#l4sps_Terminology" class="auto internal xref">Section 3</a>) using some form of
scheduler <span>[<a href="#RFC0970" class="cite xref">RFC0970</a>]</span>. And they use various
techniques, like redirection to traffic scrubbing facilities, to deal
with flooding attacks. However, there has never been a universal
need to police the rate of individual application flows -- the
Internet has generally always relied on self-restraint of congestion
controls at senders for sharing intra-'site' capacity.<a href="#section-8.1.1-1" class="pilcrow">¶</a></p>
<p id="section-8.1.1-2">L4S has been designed not to upset this status quo. If a DualQ is
used to provide L4S service, <span><a href="https://www.rfc-editor.org/rfc/rfc9332#section-4.2" class="relref">Section 4.2</a> of [<a href="#RFC9332" class="cite xref">RFC9332</a>]</span> explains how it is
designed to give no more rate advantage to unresponsive flows than a
single-queue AQM would, whether or not there is traffic
overload.<a href="#section-8.1.1-2" class="pilcrow">¶</a></p>
<p id="section-8.1.1-3">Also, in case per-flow rate policing is ever required, it can be
added because it is orthogonal to the distinction between L4S and
Classic. As explained in <a href="#l4sps_why-not" class="auto internal xref">Section 5.2</a>, the DualQ
variant of L4S provides low delay without prejudging the issue of
flow-rate control. So if flow-rate control is needed,
per-flow queuing (FQ) with L4S support can be used instead, or flow
rate policing can be added as a modular addition to a DualQ.
However, per-flow rate control is not usually deployed as a security
mechanism, because an active attacker can just shard its traffic
over more flow identifiers if the rate of each is restricted.<a href="#section-8.1.1-3" class="pilcrow">¶</a></p>
</section>
<section id="section-8.1.2">
<h4 id="name-non-policing-l4s-service-ra">
<a href="#section-8.1.2" class="section-number selfRef">8.1.2. </a><a href="#name-non-policing-l4s-service-ra" class="section-name selfRef">(Non-)Policing L4S Service Rate</a>
</h4>
<p id="section-8.1.2-1"><a href="#l4sps_why-not" class="auto internal xref">Section 5.2</a> explains how Diffserv only makes a
difference if some packets get less favourable treatment than
others, which typically requires traffic rate policing for a low
latency class. In contrast, it should not be necessary to
rate-police access to the L4S service to protect the Classic
service, because L4S is designed to reduce delay without harming the
delay or rate of any Classic traffic.<a href="#section-8.1.2-1" class="pilcrow">¶</a></p>
<p id="section-8.1.2-2">During early deployment (and perhaps always), some networks will
not offer the L4S service. In general, these networks should not
need to police L4S traffic. They are required (by both the ECN
spec <span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span> and the L4S ECN spec <span>[<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>) not to change the L4S
identifier, which would interfere with end-to-end congestion
control. If they already treat ECN traffic as Not-ECT, they can
merely treat L4S traffic as Not-ECT too. At a bottleneck, such
networks will introduce some queuing and dropping. When a Scalable
congestion control detects a drop, it will have to respond safely
with respect to Classic congestion controls (as required in
<span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-4.3" class="relref">Section 4.3</a> of [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>). This will
degrade the L4S service to be no better (but never worse) than
Classic best efforts whenever a non-ECN bottleneck is encountered
on a path (see <a href="#l4sarch_sec_non-l4s-neck" class="auto internal xref">Section 6.4.3</a>).<a href="#section-8.1.2-2" class="pilcrow">¶</a></p>
<p id="section-8.1.2-3">In cases that are expected to be rare, networks that solely
support Classic ECN <span>[<a href="#RFC3168" class="cite xref">RFC3168</a>]</span> in a single queue
bottleneck might opt to police L4S traffic so as to protect
competing Classic ECN traffic (for instance, see
<span><a href="https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-l4sops-03#section-6.1.3" class="relref">Section 6.1.3</a> of the L4S operational guidance [<a href="#I-D.ietf-tsvwg-l4sops" class="cite xref">L4SOPS</a>]</span>). However, <span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-4.3" class="relref">Section 4.3</a> of the L4S ECN spec [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span> recommends
that the sender adapts its congestion response to properly coexist
with Classic ECN flows, i.e., reverting to the self-restraint
approach.<a href="#section-8.1.2-3" class="pilcrow">¶</a></p>
<p id="section-8.1.2-4">Certain network operators might choose to restrict access to the
L4S service, perhaps only to selected premium customers as a
value-added service. Their packet classifier (item 2 in <a href="#l4sps_fig_components" class="auto internal xref">Figure 1</a>) could identify such customers
against some other field (e.g., source address range), as well as
classifying on the ECN field. If only the ECN L4S identifier
matched, but not (say) the source address, the classifier could
direct these packets (from non-premium customers) into the Classic
queue. Explaining clearly how operators can use additional local
classifiers (see <span><a href="https://www.rfc-editor.org/rfc/rfc9331#section-5.4" class="relref">Section 5.4</a> of [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>) is intended to remove any
motivation to clear the L4S identifier. Then at least the L4S ECN
identifier will be more likely to survive end to end, even though the
service may not be supported at every hop.
Such local arrangements
would only require simple registered/not-registered packet
classification, rather than the managed, application-specific
traffic policing against customer-specific traffic contracts that
Diffserv uses.<a href="#section-8.1.2-4" class="pilcrow">¶</a></p>
</section>
</section>
<section id="section-8.2">
<h3 id="name-latency-friendliness">
<a href="#section-8.2" class="section-number selfRef">8.2. </a><a href="#name-latency-friendliness" class="section-name selfRef">'Latency Friendliness'</a>
</h3>
<p id="section-8.2-1">Like the Classic service, the L4S service relies on self-restraint to
limit the rate in response to congestion. In addition, the L4S
service requires self-restraint in terms of limiting latency
(burstiness). It is hoped that self-interest and guidance on dynamic
behaviour (especially flow start-up, which might need to be
standardized) will be sufficient to prevent transports from sending
excessive bursts of L4S traffic, given the application's own latency
will suffer most from such behaviour.<a href="#section-8.2-1" class="pilcrow">¶</a></p>
<p id="section-8.2-2">Because the L4S service can reduce delay without discernibly
increasing the delay of any Classic traffic, it should not be
necessary to police L4S traffic to protect the delay of Classic traffic.
However, whether burst policing becomes necessary to protect other L4S
traffic remains to be seen. Without it, there will be potential for
attacks on the low latency of the L4S service.<a href="#section-8.2-2" class="pilcrow">¶</a></p>
<p id="section-8.2-3">If needed, various arrangements could be used to address this
concern:<a href="#section-8.2-3" class="pilcrow">¶</a></p>
<span class="break"></span><dl class="dlParallel" id="section-8.2-4">
<dt id="section-8.2-4.1">Local bottleneck queue protection:</dt>
<dd style="margin-left: 1.5em" id="section-8.2-4.2">A per-flow
(5-tuple) queue protection function <span>[<a href="#I-D.briscoe-docsis-q-protection" class="cite xref">DOCSIS-Q-PROT</a>]</span> has been developed for
the low latency queue in DOCSIS, which has adopted the DualQ L4S
architecture. It protects the low latency service from any
queue-building flows that accidentally or maliciously classify
themselves into the low latency queue. It is designed to score
flows based solely on their contribution to queuing (not flow rate
in itself). Then, if the shared low latency queue is at risk of
exceeding a threshold, the function redirects enough packets of
the highest scoring flow(s) into the Classic queue to preserve low
latency.<a href="#section-8.2-4.2" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-8.2-4.3">Distributed traffic scrubbing:</dt>
<dd style="margin-left: 1.5em" id="section-8.2-4.4">Rather than policing
locally at each bottleneck, it may only be necessary to address
problems reactively, e.g., punitively target any deployments
of new bursty malware, in a similar way to how traffic from
flooding attack sources is rerouted via scrubbing facilities.<a href="#section-8.2-4.4" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-8.2-4.5">Local bottleneck per-flow scheduling:</dt>
<dd style="margin-left: 1.5em" id="section-8.2-4.6">Per-flow
scheduling should inherently isolate non-bursty flows from bursty flows
(see <a href="#l4sps_why-not" class="auto internal xref">Section 5.2</a> for discussion of the merits
of per-flow scheduling relative to per-flow policing).<a href="#section-8.2-4.6" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-8.2-4.7">Distributed access subnet queue protection:</dt>
<dd style="margin-left: 1.5em" id="section-8.2-4.8">Per-flow
queue protection could be arranged for a queue structure
distributed across a subnet intercommunicating using lower layer
control messages (see Section 2.1.4 of <span>[<a href="#QDyn" class="cite xref">QDyn</a>]</span>). For
instance, in a radio access network, user equipment already sends
regular buffer status reports to a radio network controller, which
could use this information to remotely police individual
flows.<a href="#section-8.2-4.8" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-8.2-4.9">Distributed Congestion Exposure to ingress policers:</dt>
<dd style="margin-left: 1.5em" id="section-8.2-4.10">The
Congestion Exposure (ConEx) architecture <span>[<a href="#RFC7713" class="cite xref">RFC7713</a>]</span> uses an egress audit to motivate senders to
truthfully signal path congestion in-band, where it can be used by
ingress policers. An edge-to-edge variant of this architecture is
also possible.<a href="#section-8.2-4.10" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-8.2-4.11">Distributed domain-edge traffic conditioning:</dt>
<dd style="margin-left: 1.5em" id="section-8.2-4.12">An
architecture similar to Diffserv <span>[<a href="#RFC2475" class="cite xref">RFC2475</a>]</span> may
be preferred, where traffic is proactively conditioned on entry to
a domain, rather than reactively policed only if it leads to
queuing once combined with other traffic at a bottleneck.<a href="#section-8.2-4.12" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
<dt id="section-8.2-4.13">Distributed core network queue protection:</dt>
<dd style="margin-left: 1.5em" id="section-8.2-4.14">The
policing function could be divided between per-flow mechanisms at
the network ingress that characterize the burstiness of each flow
into a signal carried with the traffic and per-class mechanisms
at bottlenecks that act on these signals if queuing actually
occurs once the traffic converges. This would be somewhat similar
to <span>[<a href="#Nadas20" class="cite xref">Nadas20</a>]</span>, which is in turn similar to the idea
behind core stateless fair queuing.<a href="#section-8.2-4.14" class="pilcrow">¶</a>
</dd>
<dd class="break"></dd>
</dl>
<p id="section-8.2-5">No single one of these possible queue protection capabilities is
considered an essential part of the L4S architecture, which works
without any of them under non-attack conditions (much as the Internet
normally works without per-flow rate policing).
Indeed, even where
latency policers are deployed, under normal circumstances, they would
not intervene, and if operators found they were not necessary, they
could disable them. Part of the L4S experiment will be to see whether
such a function is necessary and which arrangements are most
appropriate to the size of the problem.<a href="#section-8.2-5" class="pilcrow">¶</a></p>
</section>
<div id="l4s_arch_sec_policing">
<section id="section-8.3">
<h3 id="name-interaction-between-rate-po">
<a href="#section-8.3" class="section-number selfRef">8.3. </a><a href="#name-interaction-between-rate-po" class="section-name selfRef">Interaction between Rate Policing and L4S</a>
</h3>
<p id="section-8.3-1">As mentioned in <a href="#l4sps_why-not" class="auto internal xref">Section 5.2</a>, L4S should remove
the need for low latency Diffserv classes. However, those Diffserv
classes that give certain applications or users priority over
capacity would still be applicable in certain scenarios
(e.g., corporate networks). Then, within such Diffserv classes,
L4S would often be applicable to give traffic low latency and low loss
as well. Within such a Diffserv class, the bandwidth available to a
user or application is often limited by a rate policer. Similarly, in
the default Diffserv class, rate policers are sometimes used to
partition shared capacity.<a href="#section-8.3-1" class="pilcrow">¶</a></p>
<p id="section-8.3-2">A Classic rate policer drops any packets exceeding a set rate,
usually also giving a burst allowance (variants exist where the
policer re-marks noncompliant traffic to a discard-eligible Diffserv
codepoint, so they can be dropped elsewhere during contention).
Whenever L4S traffic encounters one of these rate policers, it will
experience drops and the source will have to fall back to a Classic
congestion control, thus losing the benefits of L4S (<a href="#l4sarch_sec_non-l4s-neck" class="auto internal xref">Section 6.4.3</a>). So in networks that already use
rate policers and plan to deploy L4S, it will be preferable to
redesign these rate policers to be more friendly to the L4S
service.<a href="#section-8.3-2" class="pilcrow">¶</a></p>
<p id="section-8.3-3">L4S-friendly rate policing is currently a research area (note that
this is not the same as latency policing). It might be achieved by
setting a threshold where ECN marking is introduced, such that it is
just under the policed rate or just under the burst allowance where
drop is introduced. For instance, the two-rate, three-colour
marker <span>[<a href="#RFC2698" class="cite xref">RFC2698</a>]</span> or a PCN threshold and
excess-rate marker <span>[<a href="#RFC5670" class="cite xref">RFC5670</a>]</span> could mark ECN at the
lower rate and drop at the higher. Or an existing rate policer could
have congestion-rate policing added, e.g., using the 'local'
(non-ConEx) variant of the ConEx aggregate congestion
policer <span>[<a href="#I-D.briscoe-conex-policing" class="cite xref">CONG-POLICING</a>]</span>. It might
also be possible to design Scalable congestion controls to respond
less catastrophically to loss that has not been preceded by a period
of increasing delay.<a href="#section-8.3-3" class="pilcrow">¶</a></p>
<p id="section-8.3-4">The design of L4S-friendly rate policers will require a separate,
dedicated document. For further discussion of the interaction between
L4S and Diffserv, see <span>[<a href="#I-D.briscoe-tsvwg-l4s-diffserv" class="cite xref">L4S-DIFFSERV</a>]</span>.<a href="#section-8.3-4" class="pilcrow">¶</a></p>
</section>
</div>
<section id="section-8.4">
<h3 id="name-ecn-integrity">
<a href="#section-8.4" class="section-number selfRef">8.4. </a><a href="#name-ecn-integrity" class="section-name selfRef">ECN Integrity</a>
</h3>
<p id="section-8.4-1">Various ways have been developed to protect the integrity of the
congestion feedback loop (whether signalled by loss, Classic ECN, or
L4S ECN) against misbehaviour by the receiver, sender, or network (or
all three). Brief details of each, including applicability, pros, and
cons, are given in <span><a href="https://www.rfc-editor.org/rfc/rfc9331#appendix-C.1" class="relref">Appendix C.1</a> of the L4S ECN spec [<a href="#RFC9331" class="cite xref">RFC9331</a>]</span>.<a href="#section-8.4-1" class="pilcrow">¶</a></p>
</section>
<section id="section-8.5">
<h3 id="name-privacy-considerations">
<a href="#section-8.5" class="section-number selfRef">8.5. </a><a href="#name-privacy-considerations" class="section-name selfRef">Privacy Considerations</a>
</h3>
<p id="section-8.5-1">As discussed in <a href="#l4sps_why-not" class="auto internal xref">Section 5.2</a>, the L4S
architecture does not preclude approaches that inspect end-to-end
transport layer identifiers. For instance, L4S support has been added
to FQ-CoDel, which classifies by application flow identifier in the network.
However, the main innovation of L4S is the DualQ AQM framework that
does not need to inspect any deeper than the outermost IP header,
because the L4S identifier is in the IP-ECN field.<a href="#section-8.5-1" class="pilcrow">¶</a></p>
<p id="section-8.5-2">Thus, the L4S architecture enables very low queuing delay without
<em>requiring</em> inspection of information above
the IP layer. This means that users who want to encrypt application
flow identifiers, e.g., in IPsec or other encrypted VPN tunnels,
don't have to sacrifice low delay <span>[<a href="#RFC8404" class="cite xref">RFC8404</a>]</span>.<a href="#section-8.5-2" class="pilcrow">¶</a></p>
<p id="section-8.5-3">Because L4S can provide low delay for a broad set of applications
that choose to use it, there is no need for individual applications or
classes within that broad set to be distinguishable in any way while
traversing networks. This removes much of the ability to correlate
between the delay requirements of traffic and other identifying
features <span>[<a href="#RFC6973" class="cite xref">RFC6973</a>]</span>. There may be some types of
traffic that prefer not to use L4S, but the coarse binary
categorization of traffic reveals very little that could be exploited
to compromise privacy.<a href="#section-8.5-3" class="pilcrow">¶</a></p>
</section>
</section>
</div>
<section id="section-9">
<h2 id="name-informative-references">
<a href="#section-9" class="section-number selfRef">9. </a><a href="#name-informative-references" class="section-name selfRef">Informative References</a>
</h2>
<dl class="references">
<dt id="I-D.ietf-tcpm-accurate-ecn">[ACCECN]</dt>
<dd>
<span class="refAuthor">Briscoe, B.</span>, <span class="refAuthor">Kühlewind, M.</span>, and <span class="refAuthor">R. Scheffenegger</span>, <span class="refTitle">"More Accurate ECN Feedback in TCP"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-ietf-tcpm-accurate-ecn-22</span>, <time datetime="2022-11-09" class="refDate">9 November 2022</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-22">https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-22</a>></span>. </dd>
<dd class="break"></dd>
<dt id="AFCD">[AFCD]</dt>
<dd>
<span class="refAuthor">Xue, L.</span>, <span class="refAuthor">Kumar, S.</span>, <span class="refAuthor">Cui, C.</span>, <span class="refAuthor">Kondikoppa, P.</span>, <span class="refAuthor">Chiu, C-H.</span>, and <span class="refAuthor">S-J. Park</span>, <span class="refTitle">"Towards fair and low latency next generation high speed networks: AFCD queuing"</span>, <span class="refContent">Journal of Network and Computer Applications, Volume 70, pp. 183-193</span>, <span class="seriesInfo">DOI 10.1016/j.jnca.2016.03.021</span>, <time datetime="2016-07" class="refDate">July 2016</time>, <span><<a href="https://doi.org/10.1016/j.jnca.2016.03.021">https://doi.org/10.1016/j.jnca.2016.03.021</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.cardwell-iccrg-bbr-congestion-control">[BBR-CC]</dt>
<dd>
<span class="refAuthor">Cardwell, N.</span>, <span class="refAuthor">Cheng, Y.</span>, <span class="refAuthor">Hassas Yeganeh, S.</span>, <span class="refAuthor">Swett, I.</span>, and <span class="refAuthor">V. Jacobson</span>, <span class="refTitle">"BBR Congestion Control"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-cardwell-iccrg-bbr-congestion-control-02</span>, <time datetime="2022-03-07" class="refDate">7 March 2022</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-cardwell-iccrg-bbr-congestion-control-02">https://datatracker.ietf.org/doc/html/draft-cardwell-iccrg-bbr-congestion-control-02</a>></span>. </dd>
<dd class="break"></dd>
<dt id="BBRv2">[BBRv2]</dt>
<dd>
<span class="refTitle">"TCP BBR v2 Alpha/Preview Release"</span>, <span class="refContent">commit 17700ca</span>, <time datetime="2022-06" class="refDate">June 2022</time>, <span><<a href="https://github.com/google/bbr">https://github.com/google/bbr</a>></span>. </dd>
<dd class="break"></dd>
<dt id="BDPdata">[BDPdata]</dt>
<dd>
<span class="refAuthor">Briscoe, B.</span>, <span class="refTitle">"PI2 Parameters"</span>, <span class="refContent">TR-BB-2021-001, arXiv:2107.01003 [cs.NI]</span>, <span class="seriesInfo">DOI 10.48550/arXiv.2107.01003</span>, <time datetime="2021-10" class="refDate">October 2021</time>, <span><<a href="https://arxiv.org/abs/2107.01003">https://arxiv.org/abs/2107.01003</a>></span>. </dd>
<dd class="break"></dd>
<dt id="BufferSize">[BufferSize]</dt>
<dd>
<span class="refAuthor">Appenzeller, G.</span>, <span class="refAuthor">Keslassy, I.</span>, and <span class="refAuthor">N. McKeown</span>, <span class="refTitle">"Sizing Router Buffers"</span>, <span class="refContent">SIGCOMM '04: Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, pp. 281-292</span>, <span class="seriesInfo">DOI 10.1145/1015467.1015499</span>, <time datetime="2004-10" class="refDate">October 2004</time>, <span><<a href="https://doi.org/10.1145/1015467.1015499">https://doi.org/10.1145/1015467.1015499</a>></span>. </dd>
<dd class="break"></dd>
<dt id="COBALT">[COBALT]</dt>
<dd>
<span class="refAuthor">Palmei, J.</span>, <span class="refAuthor">Gupta, S.</span>, <span class="refAuthor">Imputato, P.</span>, <span class="refAuthor">Morton, J.</span>, <span class="refAuthor">Tahiliani, M. P.</span>, <span class="refAuthor">Avallone, S.</span>, and <span class="refAuthor">D. Täht</span>, <span class="refTitle">"Design and Evaluation of COBALT Queue Discipline"</span>, <span class="refContent">IEEE International Symposium on Local and Metropolitan Area Networks (LANMAN)</span>, <span class="seriesInfo">DOI 10.1109/LANMAN.2019.8847054</span>, <time datetime="2019-07" class="refDate">July 2019</time>, <span><<a href="https://ieeexplore.ieee.org/abstract/document/8847054">https://ieeexplore.ieee.org/abstract/document/8847054</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.morton-tsvwg-codel-approx-fair">[CODEL-APPROX-FAIR]</dt>
<dd>
<span class="refAuthor">Morton, J.</span> and <span class="refAuthor">P. Heist</span>, <span class="refTitle">"Controlled Delay Approximate Fairness AQM"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-morton-tsvwg-codel-approx-fair-01</span>, <time datetime="2020-03-09" class="refDate">9 March 2020</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-morton-tsvwg-codel-approx-fair-01">https://datatracker.ietf.org/doc/html/draft-morton-tsvwg-codel-approx-fair-01</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.briscoe-conex-policing">[CONG-POLICING]</dt>
<dd>
<span class="refAuthor">Briscoe, B.</span>, <span class="refTitle">"Network Performance Isolation using Congestion Policing"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-briscoe-conex-policing-01</span>, <time datetime="2014-02-14" class="refDate">14 February 2014</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-briscoe-conex-policing-01">https://datatracker.ietf.org/doc/html/draft-briscoe-conex-policing-01</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.sridharan-tcpm-ctcp">[CTCP]</dt>
<dd>
<span class="refAuthor">Sridharan, M.</span>, <span class="refAuthor">Tan, K.</span>, <span class="refAuthor">Bansal, D.</span>, and <span class="refAuthor">D. Thaler</span>, <span class="refTitle">"Compound TCP: A New TCP Congestion Control for High-Speed and Long Distance Networks"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-sridharan-tcpm-ctcp-02</span>, <time datetime="2008-11-11" class="refDate">11 November 2008</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-sridharan-tcpm-ctcp-02">https://datatracker.ietf.org/doc/html/draft-sridharan-tcpm-ctcp-02</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.briscoe-docsis-q-protection">[DOCSIS-Q-PROT]</dt>
<dd>
<span class="refAuthor">Briscoe, B., Ed.</span> and <span class="refAuthor">G. White</span>, <span class="refTitle">"The DOCSIS® Queue Protection Algorithm to Preserve Low Latency"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-briscoe-docsis-q-protection-06</span>, <time datetime="2022-05-13" class="refDate">13 May 2022</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-briscoe-docsis-q-protection-06">https://datatracker.ietf.org/doc/html/draft-briscoe-docsis-q-protection-06</a>></span>. </dd>
<dd class="break"></dd>
<dt id="DOCSIS3.1">[DOCSIS3.1]</dt>
<dd>
<span class="refAuthor">CableLabs</span>, <span class="refTitle">"MAC and Upper Layer Protocols Interface (MULPI) Specification, CM-SP-MULPIv3.1"</span>, <span class="seriesInfo">Data-Over-Cable Service Interface Specifications DOCSIS 3.1 Version i17 or later</span>, <time datetime="2019-01-21" class="refDate">21 January 2019</time>, <span><<a href="https://specification-search.cablelabs.com/CM-SP-MULPIv3.1">https://specification-search.cablelabs.com/CM-SP-MULPIv3.1</a>></span>. </dd>
<dd class="break"></dd>
<dt id="DOCSIS3AQM">[DOCSIS3AQM]</dt>
<dd>
<span class="refAuthor">White, G.</span>, <span class="refTitle">"Active Queue Management Algorithms for DOCSIS 3.0: A Simulation Study of CoDel, SFQ-CoDel and PIE in DOCSIS 3.0 Networks"</span>, <span class="refContent">CableLabs Technical Report</span>, <time datetime="2013-04" class="refDate">April 2013</time>, <span><<a href="https://www.cablelabs.com/wp-content/uploads/2013/11/Active_Queue_Management_Algorithms_DOCSIS_3_0.pdf">https://www.cablelabs.com/wp-content/uploads/2013/11/Active_Queue_Management_Algorithms_DOCSIS_3_0.pdf</a>></span>. </dd>
<dd class="break"></dd>
<dt id="DualPI2Linux">[DualPI2Linux]</dt>
<dd>
<span class="refAuthor">Albisser, O.</span>, <span class="refAuthor">De Schepper, K.</span>, <span class="refAuthor">Briscoe, B.</span>, <span class="refAuthor">Tilmans, O.</span>, and <span class="refAuthor">H. Steen</span>, <span class="refTitle">"DUALPI2 - Low Latency, Low Loss and Scalable (L4S) AQM"</span>, <span class="refContent">Proceedings of Linux Netdev 0x13</span>, <time datetime="2019-03" class="refDate">March 2019</time>, <span><<a href="https://www.netdevconf.org/0x13/session.html?talk-DUALPI2-AQM">https://www.netdevconf.org/0x13/session.html?talk-DUALPI2-AQM</a>></span>. </dd>
<dd class="break"></dd>
<dt id="Dukkipati06">[Dukkipati06]</dt>
<dd>
<span class="refAuthor">Dukkipati, N.</span> and <span class="refAuthor">N. McKeown</span>, <span class="refTitle">"Why Flow-Completion Time is the Right Metric for Congestion Control"</span>, <span class="refContent">ACM SIGCOMM Computer Communication Review, Volume 36, Issue 1, pp. 59-62</span>, <span class="seriesInfo">DOI 10.1145/1111322.1111336</span>, <time datetime="2006-01" class="refDate">January 2006</time>, <span><<a href="https://dl.acm.org/doi/10.1145/1111322.1111336">https://dl.acm.org/doi/10.1145/1111322.1111336</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.ietf-tsvwg-ecn-encap-guidelines">[ECN-ENCAP]</dt>
<dd>
<span class="refAuthor">Briscoe, B.</span> and <span class="refAuthor">J. Kaippallimalil</span>, <span class="refTitle">"Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-ietf-tsvwg-ecn-encap-guidelines-17</span>, <time datetime="2022-07-11" class="refDate">11 July 2022</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-encap-guidelines-17">https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-encap-guidelines-17</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.stewart-tsvwg-sctpecn">[ECN-SCTP]</dt>
<dd>
<span class="refAuthor">Stewart, R.</span>, <span class="refAuthor">Tuexen, M.</span>, and <span class="refAuthor">X. Dong</span>, <span class="refTitle">"ECN for Stream Control Transmission Protocol (SCTP)"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-stewart-tsvwg-sctpecn-05</span>, <time datetime="2014-01-15" class="refDate">15 January 2014</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-stewart-tsvwg-sctpecn-05">https://datatracker.ietf.org/doc/html/draft-stewart-tsvwg-sctpecn-05</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.ietf-tsvwg-rfc6040update-shim">[ECN-SHIM]</dt>
<dd>
<span class="refAuthor">Briscoe, B.</span>, <span class="refTitle">"Propagating Explicit Congestion Notification Across IP Tunnel Headers Separated by a Shim"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-ietf-tsvwg-rfc6040update-shim-15</span>, <time datetime="2022-07-11" class="refDate">11 July 2022</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-rfc6040update-shim-15">https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-rfc6040update-shim-15</a>></span>. </dd>
<dd class="break"></dd>
<dt id="FQ_CoDel_Thresh">[FQ_CoDel_Thresh]</dt>
<dd>
<span class="refTitle">"fq_codel: generalise ce_threshold marking for subset of traffic"</span>, <span class="refContent">commit dfcb63ce1de6b10b</span>, <time datetime="2021-10" class="refDate">October 2021</time>, <span><<a href="https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=dfcb63ce1de6b10b">https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=dfcb63ce1de6b10b</a>></span>. </dd>
<dd class="break"></dd>
<dt id="Hohlfeld14">[Hohlfeld14]</dt>
<dd>
<span class="refAuthor">Hohlfeld, O.</span>, <span class="refAuthor">Pujol, E.</span>, <span class="refAuthor">Ciucu, F.</span>, <span class="refAuthor">Feldmann, A.</span>, and <span class="refAuthor">P. Barford</span>, <span class="refTitle">"A QoE Perspective on Sizing Network Buffers"</span>, <span class="refContent">IMC '14: Proceedings of the 2014 Conference on Internet Measurement, pp. 333-346</span>, <span class="seriesInfo">DOI 10.1145/2663716.2663730</span>, <time datetime="2014-11" class="refDate">November 2014</time>, <span><<a href="https://doi.acm.org/10.1145/2663716.2663730">https://doi.acm.org/10.1145/2663716.2663730</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.briscoe-tsvwg-l4s-diffserv">[L4S-DIFFSERV]</dt>
<dd>
<span class="refAuthor">Briscoe, B.</span>, <span class="refTitle">"Interactions between Low Latency, Low Loss, Scalable Throughput (L4S) and Differentiated Services"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-briscoe-tsvwg-l4s-diffserv-02</span>, <time datetime="2018-11-04" class="refDate">4 November 2018</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-briscoe-tsvwg-l4s-diffserv-02">https://datatracker.ietf.org/doc/html/draft-briscoe-tsvwg-l4s-diffserv-02</a>></span>. </dd>
<dd class="break"></dd>
<dt id="L4Sdemo16">[L4Sdemo16]</dt>
<dd>
<span class="refAuthor">Bondarenko, O.</span>, <span class="refAuthor">De Schepper, K.</span>, <span class="refAuthor">Tsang, I.</span>, <span class="refAuthor">Briscoe, B.</span>, <span class="refAuthor">Petlund, A.</span>, and <span class="refAuthor">C. Griwodz</span>, <span class="refTitle">"Ultra-Low Delay for All: Live Experience, Live Analysis"</span>, <span class="refContent">Proceedings of the 7th International Conference on Multimedia Systems, Article No. 33, pp. 1-4</span>, <span class="seriesInfo">DOI 10.1145/2910017.2910633</span>, <time datetime="2016-05" class="refDate">May 2016</time>, <span><<a href="https://dl.acm.org/citation.cfm?doid=2910017.2910633">https://dl.acm.org/citation.cfm?doid=2910017.2910633</a>></span>. </dd>
<dd class="break"></dd>
<dt id="L4Sdemo16-Video">[L4Sdemo16-Video]</dt>
<dd>
<span class="refTitle">"Videos used in IETF dispatch WG 'Ultra-Low Queuing Delay for All Apps' slot"</span>, <span><<a href="https://riteproject.eu/dctth/#1511dispatchwg">https://riteproject.eu/dctth/#1511dispatchwg</a>></span>. </dd>
<dd class="break"></dd>
<dt id="L4Seval22">[L4Seval22]</dt>
<dd>
<span class="refAuthor">De Schepper, K.</span>, <span class="refAuthor">Albisser, O.</span>, <span class="refAuthor">Tilmans, O.</span>, and <span class="refAuthor">B. Briscoe</span>, <span class="refTitle">"Dual Queue Coupled AQM: Deployable Very Low Queuing Delay for All"</span>, <span class="refContent">TR-BB-2022-001, arXiv:2209.01078 [cs.NI]</span>, <span class="seriesInfo">DOI 10.48550/arXiv.2209.01078</span>, <time datetime="2022-09" class="refDate">September 2022</time>, <span><<a href="https://arxiv.org/abs/2209.01078">https://arxiv.org/abs/2209.01078</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.ietf-tsvwg-l4sops">[L4SOPS]</dt>
<dd>
<span class="refAuthor">White, G., Ed.</span>, <span class="refTitle">"Operational Guidance for Deployment of L4S in the Internet"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-ietf-tsvwg-l4sops-03</span>, <time datetime="2022-04-28" class="refDate">28 April 2022</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-l4sops-03">https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-l4sops-03</a>></span>. </dd>
<dd class="break"></dd>
<dt id="LEDBAT_AQM">[LEDBAT_AQM]</dt>
<dd>
<span class="refAuthor">Al-Saadi, R.</span>, <span class="refAuthor">Armitage, G.</span>, and <span class="refAuthor">J. But</span>, <span class="refTitle">"Characterising LEDBAT Performance Through Bottlenecks Using PIE, FQ-CoDel and FQ-PIE Active Queue Management"</span>, <span class="refContent">IEEE 42nd Conference on Local Computer Networks (LCN)</span>, <span class="seriesInfo">DOI 10.1109/LCN.2017.22</span>, <time datetime="2017-10" class="refDate">October 2017</time>, <span><<a href="https://ieeexplore.ieee.org/document/8109367">https://ieeexplore.ieee.org/document/8109367</a>></span>. </dd>
<dd class="break"></dd>
<dt id="lowat">[lowat]</dt>
<dd>
<span class="refAuthor">Meenan, P.</span>, <span class="refTitle">"Optimizing HTTP/2 prioritization with BBR and tcp_notsent_lowat"</span>, <span class="refContent">Cloudflare Blog</span>, <time datetime="2018-10" class="refDate">October 2018</time>, <span><<a href="https://blog.cloudflare.com/http-2-prioritization-with-nginx/">https://blog.cloudflare.com/http-2-prioritization-with-nginx/</a>></span>. </dd>
<dd class="break"></dd>
<dt id="McIlroy78">[McIlroy78]</dt>
<dd>
<span class="refAuthor">McIlroy, M.D.</span>, <span class="refAuthor">Pinson, E. N.</span>, and <span class="refAuthor">B. A. Tague</span>, <span class="refTitle">"UNIX Time-Sharing System: Foreword"</span>, <span class="refContent">The Bell System Technical Journal 57: 6, pp. 1899-1904</span>, <span class="seriesInfo">DOI 10.1002/j.1538-7305.1978.tb02135.x</span>, <time datetime="1978-07" class="refDate">July 1978</time>, <span><<a href="https://archive.org/details/bstj57-6-1899">https://archive.org/details/bstj57-6-1899</a>></span>. </dd>
<dd class="break"></dd>
<dt id="Nadas20">[Nadas20]</dt>
<dd>
<span class="refAuthor">Nádas, S.</span>, <span class="refAuthor">Gombos, G.</span>, <span class="refAuthor">Fejes, F.</span>, and <span class="refAuthor">S. Laki</span>, <span class="refTitle">"A Congestion Control Independent L4S Scheduler"</span>, <span class="refContent">ANRW '20: Proceedings of the Applied Networking Research Workshop, pp. 45-51</span>, <span class="seriesInfo">DOI 10.1145/3404868.3406669</span>, <time datetime="2020-07" class="refDate">July 2020</time>, <span><<a href="https://doi.org/10.1145/3404868.3406669">https://doi.org/10.1145/3404868.3406669</a>></span>. </dd>
<dd class="break"></dd>
<dt id="NASA04">[NASA04]</dt>
<dd>
<span class="refAuthor">Bailey, R.</span>, <span class="refAuthor">Trey Arthur III, J.</span>, and <span class="refAuthor">S. Williams</span>, <span class="refTitle">"Latency Requirements for Head-Worn Display S/EVS Applications"</span>, <span class="refContent">Proceedings of SPIE 5424</span>, <span class="seriesInfo">DOI 10.1117/12.554462</span>, <time datetime="2004-04" class="refDate">April 2004</time>, <span><<a href="https://ntrs.nasa.gov/api/citations/20120009198/downloads/20120009198.pdf?attachment=true">https://ntrs.nasa.gov/api/citations/20120009198/downloads/20120009198.pdf?attachment=true</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.ietf-tsvwg-nqb">[NQB-PHB]</dt>
<dd>
<span class="refAuthor">White, G.</span> and <span class="refAuthor">T. Fossati</span>, <span class="refTitle">"A Non-Queue-Building Per-Hop Behavior (NQB PHB) for Differentiated Services"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-ietf-tsvwg-nqb-15</span>, <time datetime="2023-01-11" class="refDate">11 January 2023</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-nqb-15">https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-nqb-15</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.briscoe-iccrg-prague-congestion-control">[PRAGUE-CC]</dt>
<dd>
<span class="refAuthor">De Schepper, K.</span>, <span class="refAuthor">Tilmans, O.</span>, and <span class="refAuthor">B. Briscoe, Ed.</span>, <span class="refTitle">"Prague Congestion Control"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-briscoe-iccrg-prague-congestion-control-01</span>, <time datetime="2022-07-11" class="refDate">11 July 2022</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-briscoe-iccrg-prague-congestion-control-01">https://datatracker.ietf.org/doc/html/draft-briscoe-iccrg-prague-congestion-control-01</a>></span>. </dd>
<dd class="break"></dd>
<dt id="PragueLinux">[PragueLinux]</dt>
<dd>
<span class="refAuthor">Briscoe, B.</span>, <span class="refAuthor">De Schepper, K.</span>, <span class="refAuthor">Albisser, O.</span>, <span class="refAuthor">Misund, J.</span>, <span class="refAuthor">Tilmans, O.</span>, <span class="refAuthor">Kühlewind, M.</span>, and <span class="refAuthor">A.S. Ahmed</span>, <span class="refTitle">"Implementing the 'TCP Prague' Requirements for Low Latency Low Loss Scalable Throughput (L4S)"</span>, <span class="refContent">Proceedings Linux Netdev 0x13</span>, <time datetime="2019-03" class="refDate">March 2019</time>, <span><<a href="https://www.netdevconf.org/0x13/session.html?talk-tcp-prague-l4s">https://www.netdevconf.org/0x13/session.html?talk-tcp-prague-l4s</a>></span>. </dd>
<dd class="break"></dd>
<dt id="QDyn">[QDyn]</dt>
<dd>
<span class="refAuthor">Briscoe, B.</span>, <span class="refTitle">"Rapid Signalling of Queue Dynamics"</span>, <span class="refContent">TR-BB-2017-001, arXiv:1904.07044 [cs.NI]</span>, <span class="seriesInfo">DOI 10.48550/arXiv.1904.07044</span>, <time datetime="2019-04" class="refDate">April 2019</time>, <span><<a href="https://arxiv.org/abs/1904.07044">https://arxiv.org/abs/1904.07044</a>></span>. </dd>
<dd class="break"></dd>
<dt id="Raaen14">[Raaen14]</dt>
<dd>
<span class="refAuthor">Raaen, K.</span> and <span class="refAuthor">T-M. Grønli</span>, <span class="refTitle">"Latency Thresholds for Usability in Games: A Survey"</span>, <span class="refContent">Norsk IKT-konferanse for forskning og utdanning (Norwegian
ICT conference for research and education)</span>, <time datetime="2014" class="refDate">2014</time>, <span><<a href="http://ojs.bibsys.no/index.php/NIK/article/view/9/6">http://ojs.bibsys.no/index.php/NIK/article/view/9/6</a>></span>. </dd>
<dd class="break"></dd>
<dt id="Rajiullah15">[Rajiullah15]</dt>
<dd>
<span class="refAuthor">Rajiullah, M.</span>, <span class="refTitle">"Towards a Low Latency Internet: Understanding and Solutions"</span>, <span class="refContent">Dissertation, Karlstad University</span>, <time datetime="2015" class="refDate">2015</time>, <span><<a href="https://www.diva-portal.org/smash/get/diva2:846109/FULLTEXT01.pdf">https://www.diva-portal.org/smash/get/diva2:846109/FULLTEXT01.pdf</a>></span>. </dd>
<dd class="break"></dd>
<dt id="I-D.mathis-iccrg-relentless-tcp">[RELENTLESS]</dt>
<dd>
<span class="refAuthor">Mathis, M.</span>, <span class="refTitle">"Relentless Congestion Control"</span>, <span class="refContent">Work in Progress</span>, <span class="seriesInfo">Internet-Draft, draft-mathis-iccrg-relentless-tcp-00</span>, <time datetime="2009-03-04" class="refDate">4 March 2009</time>, <span><<a href="https://datatracker.ietf.org/doc/html/draft-mathis-iccrg-relentless-tcp-00">https://datatracker.ietf.org/doc/html/draft-mathis-iccrg-relentless-tcp-00</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC0970">[RFC0970]</dt>
<dd>
<span class="refAuthor">Nagle, J.</span>, <span class="refTitle">"On Packet Switches With Infinite Storage"</span>, <span class="seriesInfo">RFC 970</span>, <span class="seriesInfo">DOI 10.17487/RFC0970</span>, <time datetime="1985-12" class="refDate">December 1985</time>, <span><<a href="https://www.rfc-editor.org/info/rfc970">https://www.rfc-editor.org/info/rfc970</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC2475">[RFC2475]</dt>
<dd>
<span class="refAuthor">Blake, S.</span>, <span class="refAuthor">Black, D.</span>, <span class="refAuthor">Carlson, M.</span>, <span class="refAuthor">Davies, E.</span>, <span class="refAuthor">Wang, Z.</span>, and <span class="refAuthor">W. Weiss</span>, <span class="refTitle">"An Architecture for Differentiated Services"</span>, <span class="seriesInfo">RFC 2475</span>, <span class="seriesInfo">DOI 10.17487/RFC2475</span>, <time datetime="1998-12" class="refDate">December 1998</time>, <span><<a href="https://www.rfc-editor.org/info/rfc2475">https://www.rfc-editor.org/info/rfc2475</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC2698">[RFC2698]</dt>
<dd>
<span class="refAuthor">Heinanen, J.</span> and <span class="refAuthor">R. Guerin</span>, <span class="refTitle">"A Two Rate Three Color Marker"</span>, <span class="seriesInfo">RFC 2698</span>, <span class="seriesInfo">DOI 10.17487/RFC2698</span>, <time datetime="1999-09" class="refDate">September 1999</time>, <span><<a href="https://www.rfc-editor.org/info/rfc2698">https://www.rfc-editor.org/info/rfc2698</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC2884">[RFC2884]</dt>
<dd>
<span class="refAuthor">Hadi Salim, J.</span> and <span class="refAuthor">U. Ahmed</span>, <span class="refTitle">"Performance Evaluation of Explicit Congestion Notification (ECN) in IP Networks"</span>, <span class="seriesInfo">RFC 2884</span>, <span class="seriesInfo">DOI 10.17487/RFC2884</span>, <time datetime="2000-07" class="refDate">July 2000</time>, <span><<a href="https://www.rfc-editor.org/info/rfc2884">https://www.rfc-editor.org/info/rfc2884</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC3168">[RFC3168]</dt>
<dd>
<span class="refAuthor">Ramakrishnan, K.</span>, <span class="refAuthor">Floyd, S.</span>, and <span class="refAuthor">D. Black</span>, <span class="refTitle">"The Addition of Explicit Congestion Notification (ECN) to IP"</span>, <span class="seriesInfo">RFC 3168</span>, <span class="seriesInfo">DOI 10.17487/RFC3168</span>, <time datetime="2001-09" class="refDate">September 2001</time>, <span><<a href="https://www.rfc-editor.org/info/rfc3168">https://www.rfc-editor.org/info/rfc3168</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC3246">[RFC3246]</dt>
<dd>
<span class="refAuthor">Davie, B.</span>, <span class="refAuthor">Charny, A.</span>, <span class="refAuthor">Bennet, J.C.R.</span>, <span class="refAuthor">Benson, K.</span>, <span class="refAuthor">Le Boudec, J.Y.</span>, <span class="refAuthor">Courtney, W.</span>, <span class="refAuthor">Davari, S.</span>, <span class="refAuthor">Firoiu, V.</span>, and <span class="refAuthor">D. Stiliadis</span>, <span class="refTitle">"An Expedited Forwarding PHB (Per-Hop Behavior)"</span>, <span class="seriesInfo">RFC 3246</span>, <span class="seriesInfo">DOI 10.17487/RFC3246</span>, <time datetime="2002-03" class="refDate">March 2002</time>, <span><<a href="https://www.rfc-editor.org/info/rfc3246">https://www.rfc-editor.org/info/rfc3246</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC3540">[RFC3540]</dt>
<dd>
<span class="refAuthor">Spring, N.</span>, <span class="refAuthor">Wetherall, D.</span>, and <span class="refAuthor">D. Ely</span>, <span class="refTitle">"Robust Explicit Congestion Notification (ECN) Signaling with Nonces"</span>, <span class="seriesInfo">RFC 3540</span>, <span class="seriesInfo">DOI 10.17487/RFC3540</span>, <time datetime="2003-06" class="refDate">June 2003</time>, <span><<a href="https://www.rfc-editor.org/info/rfc3540">https://www.rfc-editor.org/info/rfc3540</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC3649">[RFC3649]</dt>
<dd>
<span class="refAuthor">Floyd, S.</span>, <span class="refTitle">"HighSpeed TCP for Large Congestion Windows"</span>, <span class="seriesInfo">RFC 3649</span>, <span class="seriesInfo">DOI 10.17487/RFC3649</span>, <time datetime="2003-12" class="refDate">December 2003</time>, <span><<a href="https://www.rfc-editor.org/info/rfc3649">https://www.rfc-editor.org/info/rfc3649</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC4340">[RFC4340]</dt>
<dd>
<span class="refAuthor">Kohler, E.</span>, <span class="refAuthor">Handley, M.</span>, and <span class="refAuthor">S. Floyd</span>, <span class="refTitle">"Datagram Congestion Control Protocol (DCCP)"</span>, <span class="seriesInfo">RFC 4340</span>, <span class="seriesInfo">DOI 10.17487/RFC4340</span>, <time datetime="2006-03" class="refDate">March 2006</time>, <span><<a href="https://www.rfc-editor.org/info/rfc4340">https://www.rfc-editor.org/info/rfc4340</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC4774">[RFC4774]</dt>
<dd>
<span class="refAuthor">Floyd, S.</span>, <span class="refTitle">"Specifying Alternate Semantics for the Explicit Congestion Notification (ECN) Field"</span>, <span class="seriesInfo">BCP 124</span>, <span class="seriesInfo">RFC 4774</span>, <span class="seriesInfo">DOI 10.17487/RFC4774</span>, <time datetime="2006-11" class="refDate">November 2006</time>, <span><<a href="https://www.rfc-editor.org/info/rfc4774">https://www.rfc-editor.org/info/rfc4774</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC4960">[RFC4960]</dt>
<dd>
<span class="refAuthor">Stewart, R., Ed.</span>, <span class="refTitle">"Stream Control Transmission Protocol"</span>, <span class="seriesInfo">RFC 4960</span>, <span class="seriesInfo">DOI 10.17487/RFC4960</span>, <time datetime="2007-09" class="refDate">September 2007</time>, <span><<a href="https://www.rfc-editor.org/info/rfc4960">https://www.rfc-editor.org/info/rfc4960</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC5033">[RFC5033]</dt>
<dd>
<span class="refAuthor">Floyd, S.</span> and <span class="refAuthor">M. Allman</span>, <span class="refTitle">"Specifying New Congestion Control Algorithms"</span>, <span class="seriesInfo">BCP 133</span>, <span class="seriesInfo">RFC 5033</span>, <span class="seriesInfo">DOI 10.17487/RFC5033</span>, <time datetime="2007-08" class="refDate">August 2007</time>, <span><<a href="https://www.rfc-editor.org/info/rfc5033">https://www.rfc-editor.org/info/rfc5033</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC5348">[RFC5348]</dt>
<dd>
<span class="refAuthor">Floyd, S.</span>, <span class="refAuthor">Handley, M.</span>, <span class="refAuthor">Padhye, J.</span>, and <span class="refAuthor">J. Widmer</span>, <span class="refTitle">"TCP Friendly Rate Control (TFRC): Protocol Specification"</span>, <span class="seriesInfo">RFC 5348</span>, <span class="seriesInfo">DOI 10.17487/RFC5348</span>, <time datetime="2008-09" class="refDate">September 2008</time>, <span><<a href="https://www.rfc-editor.org/info/rfc5348">https://www.rfc-editor.org/info/rfc5348</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC5670">[RFC5670]</dt>
<dd>
<span class="refAuthor">Eardley, P., Ed.</span>, <span class="refTitle">"Metering and Marking Behaviour of PCN-Nodes"</span>, <span class="seriesInfo">RFC 5670</span>, <span class="seriesInfo">DOI 10.17487/RFC5670</span>, <time datetime="2009-11" class="refDate">November 2009</time>, <span><<a href="https://www.rfc-editor.org/info/rfc5670">https://www.rfc-editor.org/info/rfc5670</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC5681">[RFC5681]</dt>
<dd>
<span class="refAuthor">Allman, M.</span>, <span class="refAuthor">Paxson, V.</span>, and <span class="refAuthor">E. Blanton</span>, <span class="refTitle">"TCP Congestion Control"</span>, <span class="seriesInfo">RFC 5681</span>, <span class="seriesInfo">DOI 10.17487/RFC5681</span>, <time datetime="2009-09" class="refDate">September 2009</time>, <span><<a href="https://www.rfc-editor.org/info/rfc5681">https://www.rfc-editor.org/info/rfc5681</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC6040">[RFC6040]</dt>
<dd>
<span class="refAuthor">Briscoe, B.</span>, <span class="refTitle">"Tunnelling of Explicit Congestion Notification"</span>, <span class="seriesInfo">RFC 6040</span>, <span class="seriesInfo">DOI 10.17487/RFC6040</span>, <time datetime="2010-11" class="refDate">November 2010</time>, <span><<a href="https://www.rfc-editor.org/info/rfc6040">https://www.rfc-editor.org/info/rfc6040</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC6679">[RFC6679]</dt>
<dd>
<span class="refAuthor">Westerlund, M.</span>, <span class="refAuthor">Johansson, I.</span>, <span class="refAuthor">Perkins, C.</span>, <span class="refAuthor">O'Hanlon, P.</span>, and <span class="refAuthor">K. Carlberg</span>, <span class="refTitle">"Explicit Congestion Notification (ECN) for RTP over UDP"</span>, <span class="seriesInfo">RFC 6679</span>, <span class="seriesInfo">DOI 10.17487/RFC6679</span>, <time datetime="2012-08" class="refDate">August 2012</time>, <span><<a href="https://www.rfc-editor.org/info/rfc6679">https://www.rfc-editor.org/info/rfc6679</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC6817">[RFC6817]</dt>
<dd>
<span class="refAuthor">Shalunov, S.</span>, <span class="refAuthor">Hazel, G.</span>, <span class="refAuthor">Iyengar, J.</span>, and <span class="refAuthor">M. Kuehlewind</span>, <span class="refTitle">"Low Extra Delay Background Transport (LEDBAT)"</span>, <span class="seriesInfo">RFC 6817</span>, <span class="seriesInfo">DOI 10.17487/RFC6817</span>, <time datetime="2012-12" class="refDate">December 2012</time>, <span><<a href="https://www.rfc-editor.org/info/rfc6817">https://www.rfc-editor.org/info/rfc6817</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC6973">[RFC6973]</dt>
<dd>
<span class="refAuthor">Cooper, A.</span>, <span class="refAuthor">Tschofenig, H.</span>, <span class="refAuthor">Aboba, B.</span>, <span class="refAuthor">Peterson, J.</span>, <span class="refAuthor">Morris, J.</span>, <span class="refAuthor">Hansen, M.</span>, and <span class="refAuthor">R. Smith</span>, <span class="refTitle">"Privacy Considerations for Internet Protocols"</span>, <span class="seriesInfo">RFC 6973</span>, <span class="seriesInfo">DOI 10.17487/RFC6973</span>, <time datetime="2013-07" class="refDate">July 2013</time>, <span><<a href="https://www.rfc-editor.org/info/rfc6973">https://www.rfc-editor.org/info/rfc6973</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC7560">[RFC7560]</dt>
<dd>
<span class="refAuthor">Kuehlewind, M., Ed.</span>, <span class="refAuthor">Scheffenegger, R.</span>, and <span class="refAuthor">B. Briscoe</span>, <span class="refTitle">"Problem Statement and Requirements for Increased Accuracy in Explicit Congestion Notification (ECN) Feedback"</span>, <span class="seriesInfo">RFC 7560</span>, <span class="seriesInfo">DOI 10.17487/RFC7560</span>, <time datetime="2015-08" class="refDate">August 2015</time>, <span><<a href="https://www.rfc-editor.org/info/rfc7560">https://www.rfc-editor.org/info/rfc7560</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC7567">[RFC7567]</dt>
<dd>
<span class="refAuthor">Baker, F., Ed.</span> and <span class="refAuthor">G. Fairhurst, Ed.</span>, <span class="refTitle">"IETF Recommendations Regarding Active Queue Management"</span>, <span class="seriesInfo">BCP 197</span>, <span class="seriesInfo">RFC 7567</span>, <span class="seriesInfo">DOI 10.17487/RFC7567</span>, <time datetime="2015-07" class="refDate">July 2015</time>, <span><<a href="https://www.rfc-editor.org/info/rfc7567">https://www.rfc-editor.org/info/rfc7567</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC7665">[RFC7665]</dt>
<dd>
<span class="refAuthor">Halpern, J., Ed.</span> and <span class="refAuthor">C. Pignataro, Ed.</span>, <span class="refTitle">"Service Function Chaining (SFC) Architecture"</span>, <span class="seriesInfo">RFC 7665</span>, <span class="seriesInfo">DOI 10.17487/RFC7665</span>, <time datetime="2015-10" class="refDate">October 2015</time>, <span><<a href="https://www.rfc-editor.org/info/rfc7665">https://www.rfc-editor.org/info/rfc7665</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC7713">[RFC7713]</dt>
<dd>
<span class="refAuthor">Mathis, M.</span> and <span class="refAuthor">B. Briscoe</span>, <span class="refTitle">"Congestion Exposure (ConEx) Concepts, Abstract Mechanism, and Requirements"</span>, <span class="seriesInfo">RFC 7713</span>, <span class="seriesInfo">DOI 10.17487/RFC7713</span>, <time datetime="2015-12" class="refDate">December 2015</time>, <span><<a href="https://www.rfc-editor.org/info/rfc7713">https://www.rfc-editor.org/info/rfc7713</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8033">[RFC8033]</dt>
<dd>
<span class="refAuthor">Pan, R.</span>, <span class="refAuthor">Natarajan, P.</span>, <span class="refAuthor">Baker, F.</span>, and <span class="refAuthor">G. White</span>, <span class="refTitle">"Proportional Integral Controller Enhanced (PIE): A Lightweight Control Scheme to Address the Bufferbloat Problem"</span>, <span class="seriesInfo">RFC 8033</span>, <span class="seriesInfo">DOI 10.17487/RFC8033</span>, <time datetime="2017-02" class="refDate">February 2017</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8033">https://www.rfc-editor.org/info/rfc8033</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8034">[RFC8034]</dt>
<dd>
<span class="refAuthor">White, G.</span> and <span class="refAuthor">R. Pan</span>, <span class="refTitle">"Active Queue Management (AQM) Based on Proportional Integral Controller Enhanced (PIE) for Data-Over-Cable Service Interface Specifications (DOCSIS) Cable Modems"</span>, <span class="seriesInfo">RFC 8034</span>, <span class="seriesInfo">DOI 10.17487/RFC8034</span>, <time datetime="2017-02" class="refDate">February 2017</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8034">https://www.rfc-editor.org/info/rfc8034</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8170">[RFC8170]</dt>
<dd>
<span class="refAuthor">Thaler, D., Ed.</span>, <span class="refTitle">"Planning for Protocol Adoption and Subsequent Transitions"</span>, <span class="seriesInfo">RFC 8170</span>, <span class="seriesInfo">DOI 10.17487/RFC8170</span>, <time datetime="2017-05" class="refDate">May 2017</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8170">https://www.rfc-editor.org/info/rfc8170</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8257">[RFC8257]</dt>
<dd>
<span class="refAuthor">Bensley, S.</span>, <span class="refAuthor">Thaler, D.</span>, <span class="refAuthor">Balasubramanian, P.</span>, <span class="refAuthor">Eggert, L.</span>, and <span class="refAuthor">G. Judd</span>, <span class="refTitle">"Data Center TCP (DCTCP): TCP Congestion Control for Data Centers"</span>, <span class="seriesInfo">RFC 8257</span>, <span class="seriesInfo">DOI 10.17487/RFC8257</span>, <time datetime="2017-10" class="refDate">October 2017</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8257">https://www.rfc-editor.org/info/rfc8257</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8290">[RFC8290]</dt>
<dd>
<span class="refAuthor">Hoeiland-Joergensen, T.</span>, <span class="refAuthor">McKenney, P.</span>, <span class="refAuthor">Taht, D.</span>, <span class="refAuthor">Gettys, J.</span>, and <span class="refAuthor">E. Dumazet</span>, <span class="refTitle">"The Flow Queue CoDel Packet Scheduler and Active Queue Management Algorithm"</span>, <span class="seriesInfo">RFC 8290</span>, <span class="seriesInfo">DOI 10.17487/RFC8290</span>, <time datetime="2018-01" class="refDate">January 2018</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8290">https://www.rfc-editor.org/info/rfc8290</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8298">[RFC8298]</dt>
<dd>
<span class="refAuthor">Johansson, I.</span> and <span class="refAuthor">Z. Sarker</span>, <span class="refTitle">"Self-Clocked Rate Adaptation for Multimedia"</span>, <span class="seriesInfo">RFC 8298</span>, <span class="seriesInfo">DOI 10.17487/RFC8298</span>, <time datetime="2017-12" class="refDate">December 2017</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8298">https://www.rfc-editor.org/info/rfc8298</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8311">[RFC8311]</dt>
<dd>
<span class="refAuthor">Black, D.</span>, <span class="refTitle">"Relaxing Restrictions on Explicit Congestion Notification (ECN) Experimentation"</span>, <span class="seriesInfo">RFC 8311</span>, <span class="seriesInfo">DOI 10.17487/RFC8311</span>, <time datetime="2018-01" class="refDate">January 2018</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8311">https://www.rfc-editor.org/info/rfc8311</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8312">[RFC8312]</dt>
<dd>
<span class="refAuthor">Rhee, I.</span>, <span class="refAuthor">Xu, L.</span>, <span class="refAuthor">Ha, S.</span>, <span class="refAuthor">Zimmermann, A.</span>, <span class="refAuthor">Eggert, L.</span>, and <span class="refAuthor">R. Scheffenegger</span>, <span class="refTitle">"CUBIC for Fast Long-Distance Networks"</span>, <span class="seriesInfo">RFC 8312</span>, <span class="seriesInfo">DOI 10.17487/RFC8312</span>, <time datetime="2018-02" class="refDate">February 2018</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8312">https://www.rfc-editor.org/info/rfc8312</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8404">[RFC8404]</dt>
<dd>
<span class="refAuthor">Moriarty, K., Ed.</span> and <span class="refAuthor">A. Morton, Ed.</span>, <span class="refTitle">"Effects of Pervasive Encryption on Operators"</span>, <span class="seriesInfo">RFC 8404</span>, <span class="seriesInfo">DOI 10.17487/RFC8404</span>, <time datetime="2018-07" class="refDate">July 2018</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8404">https://www.rfc-editor.org/info/rfc8404</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8511">[RFC8511]</dt>
<dd>
<span class="refAuthor">Khademi, N.</span>, <span class="refAuthor">Welzl, M.</span>, <span class="refAuthor">Armitage, G.</span>, and <span class="refAuthor">G. Fairhurst</span>, <span class="refTitle">"TCP Alternative Backoff with ECN (ABE)"</span>, <span class="seriesInfo">RFC 8511</span>, <span class="seriesInfo">DOI 10.17487/RFC8511</span>, <time datetime="2018-12" class="refDate">December 2018</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8511">https://www.rfc-editor.org/info/rfc8511</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8888">[RFC8888]</dt>
<dd>
<span class="refAuthor">Sarker, Z.</span>, <span class="refAuthor">Perkins, C.</span>, <span class="refAuthor">Singh, V.</span>, and <span class="refAuthor">M. Ramalho</span>, <span class="refTitle">"RTP Control Protocol (RTCP) Feedback for Congestion Control"</span>, <span class="seriesInfo">RFC 8888</span>, <span class="seriesInfo">DOI 10.17487/RFC8888</span>, <time datetime="2021-01" class="refDate">January 2021</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8888">https://www.rfc-editor.org/info/rfc8888</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC8985">[RFC8985]</dt>
<dd>
<span class="refAuthor">Cheng, Y.</span>, <span class="refAuthor">Cardwell, N.</span>, <span class="refAuthor">Dukkipati, N.</span>, and <span class="refAuthor">P. Jha</span>, <span class="refTitle">"The RACK-TLP Loss Detection Algorithm for TCP"</span>, <span class="seriesInfo">RFC 8985</span>, <span class="seriesInfo">DOI 10.17487/RFC8985</span>, <time datetime="2021-02" class="refDate">February 2021</time>, <span><<a href="https://www.rfc-editor.org/info/rfc8985">https://www.rfc-editor.org/info/rfc8985</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC9000">[RFC9000]</dt>
<dd>
<span class="refAuthor">Iyengar, J., Ed.</span> and <span class="refAuthor">M. Thomson, Ed.</span>, <span class="refTitle">"QUIC: A UDP-Based Multiplexed and Secure Transport"</span>, <span class="seriesInfo">RFC 9000</span>, <span class="seriesInfo">DOI 10.17487/RFC9000</span>, <time datetime="2021-05" class="refDate">May 2021</time>, <span><<a href="https://www.rfc-editor.org/info/rfc9000">https://www.rfc-editor.org/info/rfc9000</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC9113">[RFC9113]</dt>
<dd>
<span class="refAuthor">Thomson, M., Ed.</span> and <span class="refAuthor">C. Benfield, Ed.</span>, <span class="refTitle">"HTTP/2"</span>, <span class="seriesInfo">RFC 9113</span>, <span class="seriesInfo">DOI 10.17487/RFC9113</span>, <time datetime="2022-06" class="refDate">June 2022</time>, <span><<a href="https://www.rfc-editor.org/info/rfc9113">https://www.rfc-editor.org/info/rfc9113</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC9331">[RFC9331]</dt>
<dd>
<span class="refAuthor">De Schepper, K.</span> and <span class="refAuthor">B. Briscoe, Ed.</span>, <span class="refTitle">"The Explicit Congestion Notification (ECN) Protocol for Low Latency, Low Loss, and Scalable Throughput (L4S)"</span>, <span class="seriesInfo">RFC 9331</span>, <span class="seriesInfo">DOI 10.17487/RFC9331</span>, <time datetime="2023-01" class="refDate">January 2023</time>, <span><<a href="https://www.rfc-editor.org/info/rfc9331">https://www.rfc-editor.org/info/rfc9331</a>></span>. </dd>
<dd class="break"></dd>
<dt id="RFC9332">[RFC9332]</dt>
<dd>
<span class="refAuthor">De Schepper, K.</span>, <span class="refAuthor">Briscoe, B., Ed.</span>, and <span class="refAuthor">G. White</span>, <span class="refTitle">"Dual-Queue Coupled Active Queue Management (AQM) for Low Latency, Low Loss, and Scalable Throughput (L4S)"</span>, <span class="seriesInfo">RFC 9332</span>, <span class="seriesInfo">DOI 10.17487/RFC9332</span>, <time datetime="2023-01" class="refDate">January 2023</time>, <span><<a href="https://www.rfc-editor.org/info/rfc9332">https://www.rfc-editor.org/info/rfc9332</a>></span>. </dd>
<dd class="break"></dd>
<dt id="SCReAM-L4S">[SCReAM-L4S]</dt>
<dd>
<span class="refTitle">"SCReAM"</span>, <span class="refContent">commit fda6c53</span>, <time datetime="2022-06" class="refDate">June 2022</time>, <span><<a href="https://github.com/EricssonResearch/scream">https://github.com/EricssonResearch/scream</a>></span>. </dd>
<dd class="break"></dd>
<dt id="TCP-CA">[TCP-CA]</dt>
<dd>
<span class="refAuthor">Jacobson, V.</span> and <span class="refAuthor">M. Karels</span>, <span class="refTitle">"Congestion Avoidance and Control"</span>, <span class="seriesInfo">Laurence Berkeley Labs Technical Report </span>, <time datetime="1988-11" class="refDate">November 1988</time>, <span><<a href="https://ee.lbl.gov/papers/congavoid.pdf">https://ee.lbl.gov/papers/congavoid.pdf</a>></span>. </dd>
<dd class="break"></dd>
<dt id="UnorderedLTE">[UnorderedLTE]</dt>
<dd>
<span class="refAuthor">Austrheim, M.</span>, <span class="refTitle">"Implementing immediate forwarding for 4G in a network simulator"</span>, <span class="refContent">Master's Thesis, University of Oslo</span>, <time datetime="2018" class="refDate">2018</time>. </dd>
<dd class="break"></dd>
</dl>
</section>
<section id="appendix-A">
<h2 id="name-acknowledgements">
<a href="#name-acknowledgements" class="section-name selfRef">Acknowledgements</a>
</h2>
<p id="appendix-A-1">Thanks to <span class="contact-name">Richard Scheffenegger</span>, <span class="contact-name">Wes Eddy</span>, <span class="contact-name">Karen Nielsen</span>, <span class="contact-name">David Black</span>, <span class="contact-name">Jake Holland</span>, <span class="contact-name">Vidhi Goel</span>, <span class="contact-name">Ermin Sakic</span>, <span class="contact-name">Praveen Balasubramanian</span>, <span class="contact-name">Gorry Fairhurst</span>, <span class="contact-name">Mirja Kuehlewind</span>, <span class="contact-name">Philip Eardley</span>, <span class="contact-name">Neal Cardwell</span>,
<span class="contact-name">Pete Heist</span>, and <span class="contact-name">Martin Duke</span>
for their useful review comments. Thanks also to the area reviewers:
<span class="contact-name">Marco Tiloca</span>, <span class="contact-name">Lars Eggert</span>,
<span class="contact-name">Roman Danyliw</span>, and <span class="contact-name">Éric Vyncke</span>.<a href="#appendix-A-1" class="pilcrow">¶</a></p>
<p id="appendix-A-2"><span class="contact-name">Bob Briscoe</span> and <span class="contact-name">Koen De Schepper</span> were partly funded by the European Community under its Seventh
Framework Programme through the Reducing Internet Transport Latency
(RITE) project (ICT-317700). The contribution of <span class="contact-name">Koen De Schepper</span> was also partly funded by the 5Growth and DAEMON EU H2020
projects. <span class="contact-name">Bob Briscoe</span> was also partly funded by the
Research Council of Norway through the TimeIn project, partly by
CableLabs, and partly by the Comcast Innovation Fund. The views expressed
here are solely those of the authors.<a href="#appendix-A-2" class="pilcrow">¶</a></p>
</section>
<div id="authors-addresses">
<section id="appendix-B">
<h2 id="name-authors-addresses">
<a href="#name-authors-addresses" class="section-name selfRef">Authors' Addresses</a>
</h2>
<address class="vcard">
<div dir="auto" class="left"><span class="fn nameRole">Bob Briscoe (<span class="role">editor</span>)</span></div>
<div dir="auto" class="left"><span class="org">Independent</span></div>
<div dir="auto" class="left"><span class="country-name">United Kingdom</span></div>
<div class="email">
<span>Email:</span>
<a href="mailto:ietf@bobbriscoe.net" class="email">ietf@bobbriscoe.net</a>
</div>
<div class="url">
<span>URI:</span>
<a href="https://bobbriscoe.net/" class="url">https://bobbriscoe.net/</a>
</div>
</address>
<address class="vcard">
<div dir="auto" class="left"><span class="fn nameRole">Koen De Schepper</span></div>
<div dir="auto" class="left"><span class="org">Nokia Bell Labs</span></div>
<div dir="auto" class="left"><span class="locality">Antwerp</span></div>
<div dir="auto" class="left"><span class="country-name">Belgium</span></div>
<div class="email">
<span>Email:</span>
<a href="mailto:koen.de_schepper@nokia.com" class="email">koen.de_schepper@nokia.com</a>
</div>
<div class="url">
<span>URI:</span>
<a href="https://www.bell-labs.com/about/researcher-profiles/koende_schepper/" class="url">https://www.bell-labs.com/about/researcher-profiles/koende_schepper/</a>
</div>
</address>
<address class="vcard">
<div dir="auto" class="left"><span class="fn nameRole">Marcelo Bagnulo</span></div>
<div dir="auto" class="left"><span class="org">Universidad Carlos III de Madrid</span></div>
<div dir="auto" class="left"><span class="street-address">Av. Universidad 30</span></div>
<div dir="auto" class="left">
<span class="postal-code">28911</span> <span class="locality">Madrid</span> </div>
<div dir="auto" class="left"><span class="country-name">Spain</span></div>
<div class="tel">
<span>Phone:</span>
<a href="tel:34%2091%206249500" class="tel">34 91 6249500</a>
</div>
<div class="email">
<span>Email:</span>
<a href="mailto:marcelo@it.uc3m.es" class="email">marcelo@it.uc3m.es</a>
</div>
<div class="url">
<span>URI:</span>
<a href="https://www.it.uc3m.es" class="url">https://www.it.uc3m.es</a>
</div>
</address>
<address class="vcard">
<div dir="auto" class="left"><span class="fn nameRole">Greg White</span></div>
<div dir="auto" class="left"><span class="org">CableLabs</span></div>
<div dir="auto" class="left"><span class="country-name">United States of America</span></div>
<div class="email">
<span>Email:</span>
<a href="mailto:G.White@CableLabs.com" class="email">G.White@CableLabs.com</a>
</div>
</address>
</section>
</div>
<script>const toc = document.getElementById("toc");
toc.querySelector("h2").addEventListener("click", e => {
toc.classList.toggle("active");
});
toc.querySelector("nav").addEventListener("click", e => {
toc.classList.remove("active");
});
</script>
</body>
</html>
|