1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 2886 2887 2888 2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 2929 2930 2931 2932 2933 2934 2935 2936 2937 2938 2939 2940 2941 2942 2943 2944 2945 2946 2947 2948 2949 2950 2951 2952 2953 2954 2955 2956 2957 2958 2959 2960 2961 2962 2963 2964 2965 2966 2967 2968 2969 2970 2971 2972 2973 2974 2975 2976 2977 2978 2979 2980 2981 2982 2983 2984 2985 2986 2987 2988 2989 2990 2991 2992 2993 2994 2995 2996 2997 2998 2999 3000 3001 3002 3003 3004 3005 3006 3007 3008 3009 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 3020 3021 3022 3023 3024 3025 3026 3027 3028 3029 3030 3031 3032 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 3052 3053 3054 3055 3056 3057 3058 3059 3060 3061 3062 3063 3064 3065 3066 3067 3068 3069 3070 3071 3072 3073 3074 3075 3076 3077 3078 3079 3080 3081 3082 3083 3084 3085 3086 3087 3088 3089 3090 3091 3092 3093 3094 3095 3096 3097 3098 3099 3100 3101 3102 3103 3104 3105 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 3117 3118 3119 3120 3121 3122 3123 3124 3125 3126 3127 3128 3129 3130 3131 3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 3143 3144 3145 3146 3147 3148 3149 3150 3151 3152 3153 3154 3155 3156 3157 3158 3159 3160 3161 3162 3163 3164 3165 3166 3167 3168 3169 3170 3171 3172 3173 3174 3175 3176 3177 3178 3179 3180 3181 3182 3183 3184 3185 3186 3187 3188 3189 3190 3191 3192 3193 3194 3195 3196 3197 3198 3199 3200 3201 3202 3203 3204 3205 3206 3207 3208 3209 3210 3211 3212 3213 3214 3215 3216 3217 3218 3219 3220 3221 3222 3223 3224 3225 3226 3227 3228 3229 3230 3231 3232 3233 3234 3235 3236 3237 3238 3239 3240 3241 3242 3243 3244 3245 3246 3247 3248 3249 3250 3251 3252 3253 3254 3255 3256 3257 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 3273 3274 3275 3276 3277 3278 3279 3280 3281 3282 3283 3284 3285 3286 3287 3288 3289 3290 3291 3292 3293 3294 3295 3296 3297 3298 3299 3300 3301 3302 3303 3304 3305 3306 3307 3308 3309 3310 3311 3312 3313 3314 3315 3316 3317 3318 3319 3320 3321 3322 3323 3324 3325 3326 3327 3328 3329 3330 3331 3332 3333 3334 3335 3336 3337 3338 3339 3340 3341 3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 3355 3356 3357 3358 3359 3360 3361 3362 3363 3364 3365 3366 3367 3368 3369 3370 3371 3372 3373 3374 3375 3376 3377 3378 3379 3380 3381 3382 3383 3384 3385 3386 3387 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 3407 3408 3409 3410 3411 3412 3413 3414 3415 3416 3417 3418 3419 3420 3421 3422 3423 3424 3425 3426 3427 3428 3429 3430 3431 3432 3433 3434 3435 3436 3437 3438 3439 3440 3441 3442 3443 3444 3445 3446 3447 3448 3449 3450 3451 3452 3453 3454 3455 3456 3457 3458 3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 3470 3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 3489 3490 3491 3492 3493 3494 3495 3496 3497 3498 3499 3500 3501 3502 3503 3504 3505 3506 3507 3508 3509 3510 3511 3512 3513 3514 3515 3516 3517 3518 3519 3520 3521 3522 3523 3524 3525 3526 3527 3528 3529 3530 3531 3532 3533 3534 3535 3536 3537 3538 3539 3540 3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 3552 3553 3554 3555 3556 3557 3558 3559 3560 3561 3562 3563 3564 3565 3566 3567 3568 3569 3570 3571 3572 3573 3574 3575 3576 3577 3578 3579 3580 3581 3582 3583 3584 3585 3586 3587 3588 3589 3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 3637 3638 3639 3640 3641 3642 3643 3644 3645 3646 3647 3648 3649 3650 3651 3652 3653 3654 3655 3656 3657 3658 3659 3660 3661 3662 3663 3664 3665 3666 3667 3668 3669 3670 3671 3672 3673 3674 3675 3676 3677 3678 3679 3680 3681 3682 3683 3684 3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 3696 3697 3698 3699 3700 3701 3702 3703 3704 3705 3706 3707 3708 3709 3710 3711 3712 3713 3714 3715 3716 3717 3718 3719 3720 3721 3722 3723 3724 3725 3726 3727 3728 3729 3730 3731 3732 3733 3734 3735 3736 3737 3738 3739 3740 3741 3742 3743 3744 3745 3746 3747 3748 3749 3750 3751 3752 3753 3754 3755 3756 3757 3758 3759 3760 3761 3762 3763 3764 3765 3766 3767 3768 3769 3770 3771 3772 3773 3774 3775 3776 3777 3778 3779 3780 3781 3782 3783 3784 3785 3786 3787 3788 3789 3790 3791 3792 3793 3794 3795 3796 3797 3798 3799 3800 3801 3802 3803 3804 3805 3806 3807 3808 3809 3810 3811 3812 3813 3814 3815 3816 3817 3818 3819 3820 3821 3822 3823 3824 3825 3826 3827 3828 3829 3830 3831 3832 3833 3834 3835 3836 3837 3838 3839 3840 3841 3842 3843 3844 3845 3846 3847 3848 3849 3850 3851 3852 3853 3854 3855 3856 3857 3858 3859 3860 3861 3862 3863 3864 3865 3866 3867 3868 3869 3870 3871 3872 3873 3874 3875 3876 3877 3878 3879 3880 3881 3882 3883 3884 3885 3886 3887 3888 3889 3890 3891 3892 3893 3894 3895 3896 3897 3898 3899 3900 3901 3902 3903 3904 3905 3906 3907 3908 3909 3910 3911 3912 3913 3914 3915 3916 3917 3918 3919 3920 3921 3922 3923 3924 3925 3926 3927 3928 3929 3930 3931 3932 3933 3934 3935 3936 3937 3938 3939 3940 3941 3942 3943 3944 3945 3946 3947 3948 3949 3950 3951 3952 3953 3954 3955 3956 3957 3958 3959 3960 3961 3962 3963 3964 3965 3966 3967 3968 3969 3970 3971 3972 3973 3974 3975 3976 3977 3978 3979 3980 3981 3982 3983 3984 3985 3986 3987 3988 3989 3990 3991 3992 3993 3994 3995 3996 3997 3998 3999 4000 4001 4002 4003 4004 4005 4006 4007 4008 4009 4010 4011 4012 4013 4014 4015 4016 4017 4018 4019 4020 4021 4022 4023 4024 4025 4026 4027 4028 4029 4030 4031 4032 4033 4034 4035 4036 4037 4038 4039 4040 4041 4042 4043 4044 4045 4046 4047 4048 4049 4050 4051 4052 4053 4054 4055 4056 4057 4058 4059 4060 4061 4062 4063 4064 4065 4066 4067 4068 4069 4070 4071 4072 4073 4074 4075 4076 4077 4078 4079 4080 4081 4082 4083 4084 4085 4086 4087 4088 4089 4090 4091 4092 4093 4094 4095 4096 4097 4098 4099 4100 4101 4102 4103 4104 4105 4106 4107 4108 4109 4110 4111 4112 4113 4114 4115 4116 4117 4118 4119 4120 4121 4122 4123 4124 4125 4126 4127 4128 4129 4130 4131 4132 4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 4157 4158 4159 4160 4161 4162 4163 4164 4165 4166 4167 4168 4169 4170 4171 4172 4173 4174 4175 4176 4177 4178 4179 4180 4181 4182 4183 4184 4185 4186 4187 4188 4189 4190 4191 4192 4193 4194 4195 4196 4197 4198 4199 4200 4201 4202 4203 4204 4205 4206 4207 4208 4209 4210 4211 4212 4213 4214 4215 4216 4217 4218 4219 4220 4221 4222 4223 4224 4225 4226 4227 4228 4229 4230 4231 4232 4233 4234 4235 4236 4237 4238 4239 4240 4241 4242 4243 4244 4245 4246 4247 4248 4249 4250 4251 4252 4253 4254 4255 4256 4257 4258 4259 4260 4261 4262 4263 4264 4265 4266 4267 4268 4269 4270 4271 4272 4273 4274 4275 4276 4277 4278 4279 4280 4281 4282 4283 4284 4285 4286 4287 4288 4289 4290 4291 4292 4293 4294 4295 4296 4297 4298 4299 4300 4301 4302 4303 4304 4305 4306 4307 4308 4309 4310 4311 4312 4313 4314 4315 4316 4317 4318 4319 4320 4321 4322 4323 4324 4325 4326 4327 4328 4329 4330 4331 4332 4333 4334 4335 4336 4337 4338 4339 4340 4341 4342 4343 4344 4345 4346 4347 4348 4349 4350 4351 4352 4353 4354 4355 4356 4357 4358 4359 4360 4361 4362 4363 4364 4365 4366 4367 4368 4369 4370 4371 4372 4373 4374 4375 4376 4377 4378 4379 4380 4381 4382 4383 4384 4385 4386 4387 4388 4389 4390 4391 4392 4393 4394 4395 4396 4397 4398 4399 4400 4401 4402 4403 4404 4405 4406 4407 4408 4409 4410 4411 4412 4413 4414 4415 4416 4417 4418 4419 4420 4421 4422 4423 4424 4425 4426 4427 4428 4429 4430 4431 4432 4433 4434 4435 4436 4437 4438 4439 4440 4441 4442 4443 4444 4445
|
2010-01-18 Brian Warner <warner@lothar.com>
* foolscap/_version.py: release Foolscap-0.5.0
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
2010-01-18 Brian Warner <warner@lothar.com>
* NEWS: update for upcoming release
2010-01-11 Brian Warner <warner@lothar.com>
* foolscap/logging/web.py (WebViewer.start): add --open, which
uses stdlib's webbrowser.open() function to automatically open a
browser window with the "flogtool web-viewer" contents.
* foolscap/pb.py (Tub.getReferenceForName): for unknown names,
only include the first two letters of the swissnum in the KeyError
we return, to avoid revealing the whole thing in the caller's
error logs. Closes #133.
* foolscap/test/test_pb.py (TestCallable.testWrongSwiss): test it
* foolscap/test/test_gifts.py (Bad.test_swissnum): same
(Bad.testReturn_swissnum): same
2009-12-28 Brian Warner <warner@lothar.com>
* doc/flappserver.xhtml: general improvements
* foolscap/appserver/cli.py (Create.run): chmod the basedir go-rwx
to keep the private key hidden from other users
* foolscap/test/test_appserver.py (CLI.test_create): test it
* doc/flappserver.xhtml: mention the chmod
* foolscap/appserver/cli.py (Start.run): use the handy run()
function from twisted.scripts.twistd instead of os.execvp to give
control to twistd. This removes a lot of grossness and should
remove the need to modify $PATH when running "bin/flappserver
start" from an uninstalled source tree.
* foolscap/logging/dumper.py (LogDumper.run): don't eat IOErrors
when you point it at a missing file. Only eat the EPIPE that
occurs when you run "flogtool dump FILE |less" and quit the pager
early.
* foolscap/logging/dumper.py: add --timestamps, which can be
"short-local", "long-local" (ISO-8601 with timezone), or
"utc" (ISO-8601 with "Z" suffix). Closes #100.
* foolscap/logging/web.py: add --timestamps, "local" or "utc".
This controls the default format. Added tooltips over the time
strings to show the event time in local, iso-8601, and
iso-8601-utc. Added links to the all-events page to reload it with
various timestamp formats and sort options.
* foolscap/test/test_logging.py (Dumper): update tests to match
(Web): same
2009-10-04 Brian Warner <warner@lothar.com>
* foolscap/slicers/set.py: suppress the DeprecationWarning that
occurs when you import 'sets' on py2.6: seems like the best way to
preserve compatibility with py2.3-compatible application code
while still avoiding the noise when running with 2.6. Closes #124.
* foolscap/appserver/client.py (Uploader.run): expanduser() the
filename, so you can use 'flappclient upload-file ~/foo.txt' even
without the shell expanding the filename for you
(ClientOptions.read_furlfile): same, closes #134.
2009-09-17 Brian Warner <warner@lothar.com>
* setup.py: use entry_points= on windows, to create executable
.bat files for flogtool/flappserver/flappclient . Stick to
scripts= on non-windows (and remember to install with
--single-version-externally-managed) to get non-opaque flogtool
scripts that don't use magic pkg_resources functions to import
foolscap. This should close #109.
2009-07-30 Brian Warner <warner@lothar.com>
* foolscap/pb.py (Tub.setLocationAutomatically): fix comment
* foolscap/test/test_serialize.py: oops, fix pyflakes warnings
* foolscap/test/test_reference.py: same
* foolscap/test/test_gifts.py: same
2009-07-28 Brian Warner <warner@lothar.com>
* foolscap/test/*: stop using old-style shouldFail (which would
mask errors)
2009-06-24 Brian Warner <warner@lothar.com>
* doc/listings/xfer-client.py, xfer-server.py, command-client.py,
command-server.py: remove, obsoleted by flappserver
* foolscap/pb.py (getRemoteURL_TCP): remove this function, I never
liked it, and it wasn't very good anyway.
* foolscap/api.py, foolscap/__init__/py, foolscap/deprecated.py:
same
* foolscap/test/test_pb.py: remove dead code
* foolscap/deprecated.py: define wrappers that emit
DeprecationWarning upon use, then wrap them around everything that
used to be in foolscap/__init__.py .
* foolscap/__init__.py: apply the wrappers. Please import from
foolscap.api instead. See #122 for details.
* foolscap/*: stop importing functions/classes directly from the
top-level 'foolscap' module, instead import them from the
'foolscap.api' module (i.e. 'from foolscap.api import Tub' instead
of 'from foolscap import Tub'). I'm planning to add deprecation
wrappers around the things you can get from 'foolscap', and this
is to make sure our own code doesn't run afoul of those warnings.
2009-06-22 Brian Warner <warner@lothar.com>
* foolscap/ipb.py (DeadReferenceError): improve the string form by
adding remote tubid, and information from the PendingRequest.
Since DeadReferenceError frequently happens in an eventually()
context, there's not usually a useful stack trace, so any piece of
information will help.
* foolscap/referenceable.py (RemoteReference._callRemote): give
interfacename+methodname to PendingRequest, so it can give that
information to DeadReferenceError.__str__ . Some day it'd be cool
to include a stack trace here.
* foolscap/call.py (PendingRequest.__init__): store it
(PendingRequest.getMethodNameInfo): return it
* foolscap/broker.py (Broker.abandonAllRequests): pass
PendingRequest to DeadReferenceError
* foolscap/test/test_call.py (TestCall.test_connection_lost_is_deadref):
test it
* foolscap/test/test_pb.py: update to match, remove dead code
* doc/examples/git-remote-add-furl: fix shbang line
* doc/examples/git-clone-furl: same
2009-06-20 Brian Warner <warner@lothar.com>
* doc/examples/git-clone-furl: easier tool to set up FURL-based
git repo
* doc/examples/git-remote-add-furl: same
* doc/examples/git-publish-with-furl: better name for
git-setup-flappserver-repo
* doc/examples/git-proxy-flappclient: mention those scripts
* foolscap/appserver/client.py: improve --help strings
* foolscap/appserver/cli.py (CreateOptions.opt_umask): capture the
current umask at "flappserver create" time for use by the eventual
server. This should reduce surprises.
* doc/flappserver.xhtml: explain the new behavior
* foolscap/appserver/server.py (AppServer): require the
umask file to contain an octal string, don't use eval()
* foolscap/test/test_appserver.py: update tests
2009-06-19 Brian Warner <warner@lothar.com>
* doc/flappserver.xhtml: add --umask= to "flappserver create",
since twisted forces it to 077 on startup, and the
git-proxy-flappclient system was winding up with overly
restrictive filemodes
* foolscap/appserver/cli.py (CreateOptions): same
* foolscap/appserver/server.py (AppServer): same
* foolscap/test/test_appserver.py (CLI.test_create2): test it
* doc/examples/git-setup-flappserver-repo: suggest --umask
2009-06-18 Brian Warner <warner@lothar.com>
* doc/examples/git-proxy-flappclient:
* doc/examples/git-setup-flappserver-repo: new pair of tools to
run the Git remote-access protocol over a flappserver-based
connection
* MANIFEST.in: include them in the source tarball
* foolscap/_version.py: bump version between releases
* misc/*/debian/changelog: same
2009-06-16 Brian Warner <warner@lothar.com>
* foolscap/_version.py: release Foolscap-0.4.2
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
2009-06-16 Brian Warner <warner@lothar.com>
* foolscap/test/test_gifts.py: remove all timeouts.. I think one
buildslave took slightly too long, which flunked the test, and
then screwed up every test after that point.
* setup.py (packages): add foolscap/appserver
* NEWS: update for upcoming release
* setup.py (setup_args): include flappclient as a script
* foolscap/negotiate.py (TubConnector.connect): if we have no
location hints, set self.failureReason so we can error out
cleanly. Closes #129.
* foolscap/tokens.py (NoLocationHintsError): new exception for
this purpose
* foolscap/negotiate.py (TubConnector.failed): call
tub.connectorFinished() too, so any pending Tub.stopService can be
retired.
* foolscap/pb.py (Tub.getReference): wrap in a maybeDeferred, so
that unparseable FURLs give us errbacks instead of synchronous
exceptions. This is the other half of #129.
(Tub.connectorFinished): stop complaining (at log.WEIRD) if the
connector wasn't in our table. I don't know what was causing this,
and I don't really care. Closes #81.
* foolscap/test/test_tub.py (BadLocationFURL): test #129 stuff
* foolscap/pb.py (Tub.getReference): always return a Deferred,
even if given an unparseable FURL. Addresses #129.
* foolscap/test/test_tub.py (NoLocationFURL): test it
* foolscap/appserver/client.py (UploadFile): accept multiple files
* foolscap/test/test_appserver.py (Upload.test_run): test it
* doc/flappserver.xhtml: update example to match
* doc/flappserver.xhtml: rename tools: use "upload-file" and
"run-command" instead of file-uploader and exe
* doc/foolscap/appserver/client.py: same
* doc/foolscap/appserver/services.py: same
* foolscap/test/test_appserver.py: same
* foolscap/test/test_appserver.py (RunCommand): improve coverage
* foolscap/appserver/cli.py: move flappclient code out of cli.py ..
* foolscap/appserver/client.py (run_flappclient): .. into client.py
(Exec.dataReceived): don't spin up StandardIO until after the
server tells us they want stdin. This also means we can stop
buffering stdin.
* foolscap/test/test_appserver.py: match client.run_flappclient
change
(RunCommand.test_run): first test of run-command/exec code
* foolscap/broker.py (Broker.doNextCall): don't run any calls
after we've been shut down. This fixes a racy bug in which two
calls arrive and get queued in the same input hunk, the first one
provokes a Tub.stopService, and the second one gets run after the
Tub has been shutdown, leaving an "Unhandled Error in Deferred"
lying around. This was occasionally flunking an appserver test.
* foolscap/test/test_tub.py (CancelPendingDeliveries): test it
* foolscap/appserver/cli.py (Restart): add 'flappserver restart'
* foolscap/appserver/services.py (Exec): make run-command work,
including optional stdin/stdout/stderr, logging of each, and
return of exitcode. Still needs unit tests and a better name.
* foolscap/appserver/client.py (Exec): same
(parse_options): pass a 'stdio' object in separately, to let
unit tests control stdin
2009-06-15 Brian Warner <warner@lothar.com>
* foolscap/appserver/services.py (FileUploader.remote_putfile):
use os.chmod, since Twisted-2.5 doesn't have FilePath.chmod
* bin/flappclient: generic client for flappserver
* foolscap/appserver/client.py: same
* foolscap/test/test_appserver.py (Upload.test_run): test file
uploading
(Client.test_help): test --help and --version
* foolscap/appserver/*: fixes to make file-upload work
* foolscap/appserver/services.py (FileUploader): implement it for
real, using twisted's FilePath. Not sure it actually works yet.
* foolscap/test/test_appserver.py: begin unit tests for appserver
* foolscap/appserver/*: clean up Create, add
AppServer.when_ready() to handle setLocationAutomatically()
better, display simple UsageError/BadServiceArguments exceptions
better (no traceback)
2009-06-14 Brian Warner <warner@lothar.com>
* foolscap/appserver/server.py (AppServer.__init__): improve
startup message
* foolscap/appserver/services.py (FileUploader.__init__): note
that tub= might be None (such as in 'add' when we're merely
validating the service arguments)
* foolscap/appserver/cli.py: run Tub for 'create', but not for
'add' or 'list'. Stash the furl-prefix to support this. The idea
will be to rewrite furl-prefix each time the server is started.
This also allows 'add' and 'list' to be used while the server is
already running.
* doc/flappserver.xhtml: add example transcript
* foolscap/pb.py (generateSwissnumber): pull this out to a
seperate function, to be called externally
(Tub.generateSwissnumber): same
* doc/flappserver.xhtml: start to create an "application server":
an easy tool to deploy pre-written foolscap-based services in a
shared server process. This first step creates the bin/flappserver
tool, and implements the "create", "add", "list", "start", and
"stop" commands. The next step will be to write some basic tests,
then implement some built-in services (file-upload and
exec-command), then write some clients to drive those services.
Eventually new services will be loaded with a plugin architecture.
Tracked in ticket #82.
* bin/flappserver: this tool is used to manipulate app servers
* foolscap/appserver/*.py: the actual code
* setup.py: bin/flappserver is a script too
2009-06-05 Brian Warner <warner@lothar.com>
* foolscap/referenceable.py (RemoteReferenceOnly.getLocationHints):
new method
(RemoteReferenceOnly.getSturdyRef): remove "this is not secure"
comment: getSturdyRef now *does* check the tubid against the
connection
(SturdyRef): factor out encode_furl/decode_furl into new functions
* foolscap/test/test_pb.py (TestReferenceable): test both
getLocationHints() and isConnected()
(TestCallable.testGetSturdyRef): same
* foolscap/test/test_gifts.py: update to use encode_furl
2009-06-03 Brian Warner <warner@lothar.com>
* foolscap/referenceable.py (RemoteReferenceOnly.isConnected): new
method, tests the same thing as notifyOnDisconnect but this one is
immediate instead of callback-based
* foolscap/test/test_call.py (TestCall.testNotifyOnDisconnect):
test it
* foolscap/test/test_call.py
(ReferenceCounting.test_reference_counting):
oops, turn off debugging noise which was accidentally left in back
at 0.4.0
* foolscap/referenceable.py (TheirReferenceUnslicer): remove the
200-byte limit on FURLs which are passed as third-party "gifts".
(ReferenceUnslicer): same, for inbound my-references
* foolscap/_version.py: bump version between releases
* misc/*/debian/changelog: same
2009-05-22 Brian Warner <warner@lothar.com>
* foolscap/_version.py: release Foolscap-0.4.1
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
2009-05-22 Brian Warner <warner@lothar.com>
* NEWS: udpate for upcoming release
* foolscap/tokens.py (RemoteException): override __str__, since we
don't put arguments in .args . Otherwise, attempting to stringify
a RemoteException will fail under python2.4 (but not 2.5).
* foolscap/test/test_logging.py (Basic.testFailure): test it
* foolscap/test/test_call.py (ExamineFailuresMixin._examine_raise):
more thorough test
* foolscap/_version.py: bump version between releases
2009-05-19 Brian Warner <warner@lothar.com>
* foolscap/_version.py: release Foolscap-0.4.0
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
2009-05-19 Brian Warner <warner@lothar.com>
* NEWS: update for the next release
* README: mention python2.6 compatibility
* foolscap/test/test__versions.py (Versions.test_record): record
versions of platform/python/twisted/foolscap in the test logs
* foolscap/api.py: add some more symbols: ChoiceOf, IntegerConstraint
* foolscap/banana.py (Banana.handleData): increment the inbound
objectCounter as soon as we see the OPEN token, rather than doing
it in handleOpen(), since the latter is skipped if we're dropping
tokens due to a Violation. This avoids the loss-of-sync that
occurred when the dropped tokens included more OPEN
sequences (e.g. when a remote method call used a bad method name,
and its arguments contained lists or tuples or other sequences).
This fixes the bug in which subsequent method calls that used
shared arguments saw their "reference" sequences point to the
wrong value. Closes #104.
* foolscap/test/test_call.py (ReferenceCounting): test it
* foolscap/referenceable.py (RemoteReferenceTracker.__init__):
require the proposed per-my-reference FURL to match the TubID that
we're using for the parent Broker. This prevents the far end from
sending a bogus FURL and thus breaking the security properties of
rref.getSturdyRef(). (they couldn't confuse Tub.getReference,
because the spoofed FURL would only be added to the per-Broker
table, but they could confuse someone who relied upon the
correctness of getSturdyRef). This closes #84.
(RemoteReferenceTracker.__init__): oops, don't compare tubids when
using an UnauthenticatedTub, since that fails the non-crypto tests
* foolscap/test/test_reference.py (TubID): test it
* foolscap/test/test_schema.py (Interfaces.test_remotereference):
update to match
* foolscap/hashutil.py: new file to smoothly use hashlib when
available and fall back to sha1/md5 when not (i.e. python2.4).
When we eventually drop py2.4 support, we'll get rid of this file.
Thanks to Stephan Peijnik for the patch. Closes #118.
* foolscap/sslverify.py: use hashutil instead of the md5 module
* foolscap/vocab.py (hashVocabTable): use hashutil instead of the
sha module
* foolscap/schema.py (PolyConstraint.checkToken): remove unused
code, thanks to Stephan Peijnik for the patch.
* foolscap/test/test_banana.py (VocabTest1.test_table_hashes):
make sure we don't change the pre-agreed vocab tables, or the hash
algorithm that we use to confirm their contents. This is in
preparation for the conditional switch to 'hashlib', from
Stephan's patch in ticket #118
2009-05-18 Stephan Peijnik <stephan@peijnik.at>
* foolscap/call.py, foolscap/constraint.py, foolscap/eventual.py,
foolscap/observer.py, foolscap/promise.py,
foolscap/reconnector.py, foolscap/remoteinterface.py,
foolscap/slicer.py: Convert all old-style classes to new-style
classes. Should close #96.
* foolscap/reconnector.py, foolscap/referenceable.py,
foolscap/remoteinterface.py: Import each module with a separate
import statement.
2009-05-18 Brian Warner <warner@lothar.com>
* foolscap/test/test_tub.py (TestCertFile.test_tubid): confirm
that we get the TubID right when using a pre-recorded cert. If we
were to accidentally change the TubID-computing hash function in
the future, this test should catch it.
(CERT_DATA): stash the contents of a pre-recorded cert
* bin/flogtool: let 'base' default to abs("."), to help windows and
systems where somebody has managed to copy it to /tmp/flogtool or
/sbin/flogtool . Closes #108.
2009-05-13 Brian Warner <warner@lothar.com>
* foolscap/test/test_call.py (ExamineFailuresMixin): rearrange
code to try and fix a weird py2.4 failure
* foolscap/api.py (RemoteException): hush pyflakes
2009-05-12 Brian Warner <warner@lothar.com>
* foolscap/logging/dumper.py: emit PID and embedded versions,
except when using --just-numbers or --verbose. Closes #97.
* foolscap/logging/web.py (Welcome.render): same
* foolscap/logging/log.py (LogFileObserver): store app_versions in
the log file created by $FLOGFILE
* foolscap/test/test_logging.py: test it all
* foolscap/tokens.py (RemoteException): add a Tub option that
causes remote exceptions to be reported with a special exception
type, named foolscap.api.RemoteException, so that they won't
be confused with locally-generated exceptions. Closes #105.
* foolscap/api.py: make RemoteException available through api.py
* doc/failures.xhtml: explain the new option
* foolscap/pb.py (Tub.setOption): handle the new
expose-remote-exception-types option, default remains True to
retain the old behavior
* foolscap/broker.py (Broker.setTub): copy the flag from the Tub
into the Broker
* foolscap/call.py (ErrorUnslicer.receiveClose): wrap exceptions
if the mode says we should
* foolscap/test/test_call.py (Failures): test RemoteException
wrapping
(TubFailures): test setOption() too
* foolscap/test/common.py (ShouldFailMixin.shouldFail.done):
return the Failure for further testing, but wrapped in an array to
avoid triggering the errback
2009-05-11 Brian Warner <warner@lothar.com>
* Makefile (LORE): parameterize "lore" binary
* doc/stylesheet.css: update to new version from Lore
* doc/schema.xhtml: fix mis-nested H3 header, to appease new Lore
2009-05-04 Brian Warner <warner@lothar.com>
* foolscap/api.py: new preferred entry point. Application code
should import symbols from "foolscap.api" instead of directly from
foolscap/__init__.py . Importing from the __init__.py will be
deprecated in the next major release, and removed in the
subsequent one. Addresses #122 (but it won't be closed until we
remove everything from __init__.py).
* foolscap/__init__.py: add comment about preferring foolscap.api
* doc/using-foolscap.xhtml: update examples to use e.g.
"from foolscap.api import Tub"
* doc/listings/*.py: update examples
* foolscap/pb.py (Tub.setLogGathererFURL): allow users to call
both Tub.setOption("log-gatherer-furl") and
Tub.setOption("log-gatherer-furlfile") on the same Tub. Previously
this was disallowed. Also avoid making multiple connections to the
same gatherer. Closes #114.
* foolscap/test/test_logging.py
(Gatherer.test_log_gatherer_furlfile_multiple): test it
* foolscap/test/test_logging.py (Publish.test_logpublisher_catchup):
don't use an absolute delay, since it causes spurious test
failures on slow systems.
* foolscap/logging/web.py (WebViewer.start): put the welcome page
at the root URL, in addition to /welcome . This simplifies the
starting URL. Closes #120.
(Reload.render_POST): make the "Reload Logfile" button point at
the new URL too.
* foolscap/logging/web.py (Welcome.render): display PID of
logfile, if available. This will be present in incident records
that were written out by the original process or gathered by a log
gatherer, in the files written by "flogtool tail --save-to=", and
in the file written by creating an explicit FileLogObserver (such
as done by setting the FLOGFILE environment variable). This should
close #80.
* foolscap/logging/log.py (LogFileObserver): record PID in header
* foolscap/test/test_logging.py (Web.test_basic): test PID
* foolscap/test/test_banana.py (ThereAndBackAgain.test_decimal):
use Decimal("Inf") instead of decimal.Inf, to unbreak python2.6.2
which privatized both Inf and Nan. Thanks to ivank for the patch.
Closes #121.
2008-10-15 Brian Warner <warner@allmydata.com>
* misc/*/debian/rules (install/python-foolscap): include
misc/classify_foolscap.py in the debian package, uncompressed, so
it goes into /usr/share/doc/python-foolscap/classify_foolscap.py ,
so you can make a symlink to it from an incident-gatherer.
* misc/classify_foolscap.py (TUBCON_RE): update to match
foolscap-tubconnector messages for both old and new versions of
Foolscap, and for python2.4 and 2.5 .
* foolscap/logging/incident.py (IncidentClassifier): add --verbose
option to 'flogtool classify-incident' to show the trigger
dictionary for any unclassifiable incidents. This is useful when
developing classification functions.
* foolscap/test/test_logging.py (Incidents.test_classify): test it
* Makefile (test-figleaf): oops, fix the test-figleaf-poll target,
by making the test-figleaf target use $(TRIAL), not hardcoded
'trial'.
* foolscap/_version.py: bump revision to 0.3.2+ while between releases
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
2008-10-14 Brian Warner <warner@lothar.com>
* foolscap/_version.py: release Foolscap-0.3.2
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
* MANIFEST.in: add misc/classify_foolscap.py to source distribution
* NEWS: update for the next release
* Makefile (test-poll): add a convenience target, for running
tests on a system with the broken combination of pyopenssl and
twisted that requires the pollreactor
(test-figleaf-poll): same
2008-10-14 Brian Warner <warner@allmydata.com>
* foolscap/logging/tail.py (LogTail): include the retrieved PID in
the saved logfile. Part of #80.
* foolscap/logging/incident.py (IncidentReporter.incident_declared):
also include the PID in the incident report's header.
* doc/specifications/logfiles.xhtml: document the ['versions']
and ['pid'] fields in incident reports and the 'flogtool tail'
savefile.
* foolscap/logging/web.py: mark any incident triggers in the
logfile, and add links to them from the welcome page. Closes #79.
* foolscap/logging/web.py (Reload): add a 'Reload Logfile' button
to the web-viewer's Welcome page. Not automatic, but it means you
don't have to get back to the shell and restart the viewer. Works
well-enough to say Closes #103.
* misc/classify_foolscap.py: plugin to classify some
foolscap-internal incidents
* foolscap/logging/cli.py: new "flogtool classify-incident"
subcommand: given an incident, say what categories it falls into.
Closes #102.
* foolscap/logging/gatherer.py (IncidentGathererService): factor
out the classification pieces into IncidentClassifierBase
* foolscap/logging/incident.py (IncidentClassifierBase): same
(IncidentClassifier.run): support for the new CLI command
* foolscap/test/test_logging.py (Incidents.test_classify): test it
* foolscap/logging/gatherer.py
(IncidentGathererService.classify_incident): change the classifier
function signature: now it just takes a single 'trigger' dict.
* foolscap/test/test_logging.py: update to match
* doc/logging.xhtml (gatherer): same
* foolscap/logging/gatherer.py
(IncidentGathererService.add_classify_files): make the incident
gatherer look in its base directory for "classify_*.py" files, use
them as plugins with classification functions.
(INCIDENT_GATHERER_TACFILE): update example text to match
* foolscap/test/test_logging.py (IncidentGatherer): test it
(Gatherer.test_wrongdir): exercise another error case
* doc/logging.xhtml (gatherer): document it
2008-10-13 Brian Warner <warner@allmydata.com>
* foolscap/logging/publish.py (IncidentSubscription.catch_up):
Don't tell the gatherer about incidents that don't have triggers:
these are malformed logfiles, such as the zero-length files that
result from a process being terminated before it manages to write
anything.
* foolscap/logging/log.py (Count): replace itertools.count with a
version that doesn't overflow at 2**31-1 (thanks to Zooko for the
patch). Closes #99.
(FoolscapLogger.__init__): use it
* foolscap/pb.py (Listener.startFactory): add log facility
identifiers
(Listener.stopFactory): same
(Listener.buildProtocol): same
(Tub.getReference): same
2008-09-20 Brian Warner <warner@lothar.com>
* doc/listings/xfer-client.py (_failure): do sys.exit(1) upon
failure, so the caller can detect it via the exit code
2008-09-10 Brian Warner <warner@allmydata.com>
* foolscap/test/test__versions.py (Versions.test_required): check
for bug #62 (openssl>0.7, twisted<=8.1.0, selectreactor) and print
a warning if it is likely that the tests would fail, to remind the
user to re-run with -r poll.
(Versions.test_required): oops, guard import of OpenSSL on having
crypto available, otherwise this test gets an error when OpenSSL
is not installed.
2008-09-08 Brian Warner <warner@lothar.com>
* doc/listings/xfer-client.py: use os.path.expanduser() on the
filenames passed in by the user, to enable things like
--furlfile ~/.upload.furl
* foolscap/_version.py: bump revision to 0.3.1+ while between releases
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
2008-09-03 Brian Warner <warner@lothar.com>
* foolscap/_version.py: release Foolscap-0.3.1
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
* NEWS: update for next release
2008-09-02 Brian Warner <warner@lothar.com>
* foolscap/negotiate.py (TubConnector.__repr__): add more
information to the repr, to help track down #81. Made TubConnector
a new-style class in the process, to make upcalling easier.
(TubConnector.checkForIdle): improve the log message
* foolscap/pb.py (Tub._log_gatherer_connected): use callRemoteOnly
to pass the logport to the gatherer: we don't need to hear about
any problems it has.
* foolscap/logging/gatherer.py
(IncidentGathererService.classify_stored_incidents): reclassify
everything that isn't already present in one of the classified/*
files. This makes it a lot easier to iterate over the [start
gatherer; see what is unknown; update classifiers; remove
classified/unknown; repeat] loop. Also log classification events
better. Closes #94.
* foolscap/test/test_logging.py (IncidentGatherer.test_emit): test it
* foolscap/test/common.py (PollMixin): replace the use of chained
Deferreds with a task.LoopingCall-based version, from Tahoe. This
avoids the weird and annoying maximum-recursion-depth-exceeded
error that occurs when the check function is called more than
about 600 times. Closes #95.
2008-08-29 Brian Warner <warner@lothar.com>
* foolscap/logging/app_versions.py (versions): move the
application version dict (which gets reported to remote
subscribers, and copied into logfiles) to a separate module. It
was causing circular import problems when it lived in an attribute
of LogPublisher.
(add_version): provide a setter method
* foolscap/logging/publish.py (LogPublisher.versions): same
(LogPublisher.remote_get_versions): same
* foolscap/logging/incident.py (IncidentReporter.incident_declared):
same
* foolscap/test/test_logging.py (IncidentGatherer.test_emit):
improve test shutdown a little bit
2008-08-28 Brian Warner <warner@lothar.com>
* foolscap/logging/incident.py (IncidentReporter.incident_declared):
put versions in the incident header too, also for #80.
* foolscap/test/test_logging.py (Incidents.test_basic): test it
* foolscap/logging/tail.py (LogPrinter.got_versions): record
remote application versions to the --save-to file, in the header.
Part of #80.
(LogPrinter.got_versions): oops, control where stdout goes
* foolscap/logging/dumper.py (LogDumper.start): show those
versions with --verbose
* foolscap/test/test_logging.py (Dumper.test_dump): update to match
(Tail.test_logprinter): same, make sure got_versions is called
* foolscap/logging/gatherer.py (IncidentObserver): only fetch one
incident at a time, to limit the size of the sender's outbound
queue. This should help close #85.
(IncidentGathererService.new_incident): include the classification
results in the per-incident log messages
* foolscap/test/test_logging.py (IncidentGatherer.test_emit): tests
* foolscap/logging/log.py (FoolscapLogger.declare_incident):
combine overlapping incidents, by passing new triggers to an
existing reporter instead of creating a new one. Helps with #85.
* foolscap/logging/incident.py (IncidentReporter.new_trigger): new
method to add subsequent triggers to an existing reporter. This
doesn't do anything yet. It should be improved to record the other
triggers in a trailer (since it's too late to add it to the
header). Every triggering event will make it into an incident
somewhere, but the report-file analysis tools may not know how to
pay attention to the subsequent triggers.
* foolscap/logging/interfaces.py (IIncidentReporter.new_trigger): same
* foolscap/test/test_logging.py (Incidents.test_overlapping): test it
* doc/logging.xhtml: mention incident-coalescing
* foolscap/logging/log.py: add bridge-foolscap-logs-to-twisted
functionality, set up by either calling bridgeLogsToTwisted(), or
by setting the FLOGTOTWISTED environment variable (to anything).
The default filter will exclude events below the OPERATIONAL
severity level, and those generated by foolscap internals (i.e.
facility.startswith("foolscap") ). Closes #93.
* foolscap/pb.py (Tub.setOption): update to match
* foolscap/test/test_logging.py (Bridge): tests for it
(Publish): update to use new APIs
* doc/logging.xhtml: docs
2008-08-25 Brian Warner <warner@lothar.com>
* foolscap/broker.py (Broker.abandonAllRequests): map both
ConnectionLost and ConnectionDone into DeadReferenceError, so that
application code only needs to check for one exception type.
* foolscap/negotiate.py (Negotiation.evaluateNegotiationVersion1):
when an existing connection is dropped in favor of a new one, drop
it with DeadReferenceError instead of ConnectionDone. The mapping
in Broker.abandonAllRequests doesn't seem to quite catch
everything in unit tests.
(Negotiation.acceptDecisionVersion1): same, when we're the slave
* foolscap/test/test_call.py (TestCall.test_connection_lost_is_deadref):
test it
(TestCall.test_connection_done_is_deadref): same
(TestCall.testChoiceOf): switch to use ShouldFailMixin
* foolscap/test/test_gifts.py (Gifts): remove
ignoreConnectionDone, just look for DeadReferenceError
2008-08-21 Brian Warner <warner@lothar.com>
* foolscap/_version.py (verstr): bump revision to 0.3.0+ while
between releases
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
2008-08-04 Brian Warner <warner@lothar.com>
* foolscap/_version.py (verstr): release Foolscap-0.3.0
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
2008-08-04 Brian Warner <warner@lothar.com>
* foolscap/logging/interfaces.py
(RILogPublisher.subscribe_to_incidents): small edit, make it more
clear that since= is used for catch_up=True
* NEWS: update for the upcoming release
* foolscap/referenceable.py (RemoteReferenceOnly.getRemoteTubID):
make this secure, by using the broker's .remote_tubref field,
instead of the remote-side-controlled sturdyref.
(RemoteReferenceOnly.getSturdyRef): add a note about the
insecurity of this method
* foolscap/test/test_pb.py (TestCallable.testGetSturdyRef): add a
test for getRemoteTubID
* doc/logging.xhtml: change filenames in the incident-gatherer to
have fewer files starting with incident*, so tab-completion works
better.
* foolscap/logging/gatherer.py (IncidentGathererService): same
(create_incident_gatherer): same
* foolscap/test/test_logging.py (Filter): test 'flogtool filter'
* foolscap/logging/filter.py: control stdout/stderr better
* foolscap/test/test_logging.py (Dumper): test 'flogtool dump'
* foolscap/logging/cli.py (dispatch): pass stdout/stderr through
attributes on the subcommand's Options instance, rather than as
separate function arguments.. makes testing easier.
* foolscap/logging/gatherer.py (create_log_gatherer): same
(create_incident_gatherer): same
(CreateGatherOptions): same
(CreateIncidentGatherOptions): same
* foolscap/logging/dumper.py (DumpOptions): same
2008-08-01 Brian Warner <warner@lothar.com>
* foolscap/logging/log.py (LogFileObserver): make these more
useful, by not doing the addSystemEventTrigger in __init__
(LogFileObserver.stop_on_shutdown): do it here instead
(FLOGFILE): when using $FLOGFILE, add call to stop_on_shutdown
* foolscap/logging/gatherer.py (GathererService.do_rotate): oops,
implement the precautions claimed by the comment in startService:
test self._savefile before doing anything.
(GathererService.__init__): set self._savefile to None
* foolscap/test/test_logging.py (Gatherer.test_log_gatherer2): add
a timed rotator, which caught the problem in do_rotate
* foolscap/test/test_logging.py (IncidentGatherer.test_emit): add
test coverage for re-classifying existing incidents
(IncidentGatherer.test_emit._update_classifiers_again): and verify
that leaving the classified/ directory in place properly inhibits
reclassification at startup
* foolscap/referenceable.py (RemoteReferenceOnly.getRemoteTubID):
new method to extract the remote tubid from a RemoteReference. I'm
not sure this is a good idea, but it fixes the immediate problem
I'm dealing with.
* foolscap/logging/gatherer.py: use "tubid_s" instead of "nodeid_s"
(IncidentGathererService.remote_logport): use getRemoteTubID()
instead of trying to sanitize the tubid we receive, since we
use it as a directory name
(IncidentGathererService.add_classifier): rename addClassifier to
add_classifier, I'm more fond of the latter form these days. Also
remove a pyflakes warning.
* foolscap/test/test_logging.py (IncidentGatherer.test_emit): add
test of incident generation, publish, recording, and default
classification
* foolscap/logging/gatherer.py (IncidentGathererService): get
control over stdout, so we can exercise more code during tests
(IncidentGathererService.startService): oops, this needs to be
startService instead of start
* foolscap/test/test_logging.py (IncidentGatherer): basic test of
an incident gatherer, just setup and connection so far
* foolscap/test/common.py (StallMixin): factor stall() out into a
separate class
(TargetMixin): same
* foolscap/test/test_logging.py (LogfileReaderMixin): refactor
(Gatherer.test_log_gatherer): turn on bzip=True, for more coverage
* foolscap/test/test_logging.py (Gatherer._check_gatherer): ignore
internal foolscap messages (like connection negotiation), since
they occur at unpredictable times (specifically in
test_log_gatherer_furlfile_multiple, which has establishes
multiple connections)
* foolscap/test/common.py (GoodEnoughTub): factor this and
crypto_available out of all the other unit tests, make
GoodEnoughTub(certFile=) work even if crypto is unavailable and
we must therefore discard the certFile= argument.
* foolscap/test/test_crypto.py: same
* foolscap/test/test_gifts.py: same
* foolscap/test/test_keepalive.py: same
* foolscap/test/test_logging.py: same
* foolscap/test/test_loopback.py: same
* foolscap/test/test_negotiate.py: same
* foolscap/test/test_pb.py: same
* foolscap/test/test_serialize.py: same
* foolscap/test/test_tub.py: same
* foolscap/logging/gatherer.py (GatheringBase): rewrite Gatherers
to make them easier to test: now they are a Service (with a
subordinate Tub), meant to be run by a .tac file or manually. The
intermediate class has been removed. All .tac files are unchanged:
gatherers created by old versions of 'flogtool create-gatherer'
will continue to work.
(GathererService.do_rotate): return the name of the logfile that
was just closed and/or compressed, so tests can know where to
look.
* foolscap/test/test_logging.py (Gatherer): update to match
* foolscap/util.py (get_local_ip_for): update reactor comment
2008-07-30 Brian Warner <warner@lothar.com>
* setup.py: update comment about bug #62 (pyopenssl problems)
* foolscap/test/test_logging.py (CLI.test_create_gatherer):
improve test coverage a little bit, by recording stdout
(CLI.test_create_incident_gatherer): exercise the 'flogtool
create-incident-gatherer' command
* foolscap/logging/cli.py (run_flogtool): capture stdout+stderr
(dispatch): same
* foolscap/logging/gatherer.py (create_log_gatherer): same
(create_incident_gatherer): same
* trees/tahoe/Makefile (figleaf-output): exclude foolscap/test/ from
the HTML results
* doc/logging.xhtml (Running an Incident Gatherer): describe the
Incident Gatherer, like the Log Gatherer but it only gathers
incidents. It also does classification, and will eventually do
reporting. No unit tests yet, but some manual system-level tests
have been run.
* foolscap/logging/gatherer.py (IncidentGatherer): implement it
* foolscap/logging/cli.py (Options.subCommands): add the CLI
command, named 'flogtool create-incident-gatherer'
* foolscap/logging/incident.py
(IncidentReporter.incident_declared): clean up incident naming
* foolscap/logging/interfaces.py (RILogPublisher.list_incidents): same
* foolscap/logging/log.py (FoolscapLogger.incident_recorded): same
* foolscap/logging/publish.py (LogPublisher.list_incident_names): same
* foolscap/test/test_logging.py
(IncidentPublisher.test_list_incident_names): test the
incident-naming cleanup
2008-07-29 Brian Warner <warner@lothar.com>
* foolscap/logging/gatherer.py (GatheringBase): refactor the log
gatherer to share code with the upcoming incident gatherer
(LogGatherer.__init__): add a basedir= argument, rather than
using os.getcwd
* foolscap/test/test_logging.py (Gatherer): add basedir= argument
* doc/logging.xhtml (Setting up the logport): New feature (well,
it didn't work before, and now it does, so make it explicit): the
log-gatherer FURL can be configured (but will not be connected)
until after setLocation. This should resolve a crash I've seen in
Tahoe (which runs a slow /sbin/ifconfig command to figure out the
addresses to pass to setLocation) in which the app connects to the
log gatherer before it figures out its own location, and then gets
an exception during registerReference. Closes #55.
* foolscap/pb.py (Tub._maybeConnectToGatherer): don't initiate the
gatherer connection until locationHints is set
(Tub.setLocation): call _maybeConnectToGatherer after the location
is set. Also, don't let setLocation be called multiple times.
* foolscap/test/test_logging.py
(Publish.test_logport_furlfile2): test it
(Gatherer.test_log_gatherer2): same
(Gatherer.test_log_gatherer_furlfile2): same
* doc/logging.xhtml (Setting up the logport): slight API
restriction: the logport and its FURL are not available until
after Tub.setLocation is called. This results in a better error
message than the usual one inside registerReference.
* foolscap/pb.py (Tub.getLogPort): enforce the API restriction by
throwing an exception when it is violated
(Tub.getLogPortFURL): same
* foolscap/tokens.py (NoLocationError): new exception for it
* foolscap/test/test_tub.py (SetLocation.test_set_location): test it
* doc/logging.xhtml (Configuring a Log Gatherer): allow multiple
log-gatherer furls in the log-gatherer-furlfile
* foolscap/pb.py (Tub._maybeConnectToGatherer): same
* foolscap/logging/gatherer.py (LogGatherer.remote_logport):
return the subscribe_to_all Deferred, for testing
* foolscap/test/test_logging.py (Gatherer.test_log_gatherer):
refactor, to accomodate new test
(Gatherer.test_log_gatherer_furlfile_multiple): test it
2008-07-28 Brian Warner <warner@lothar.com>
* foolscap/logging/interfaces.py (RILogPublisher.list_incidents):
change signature to remove tubid/incarnation from the response.
(RILogPublisher.subscribe_to_incidents): add pubsub interface for
incidents, including catch_up= and since=
(RILogObserver.new_incident): same
(RILogObserver.done_with_incident_catchup): same
* foolscap/logging/publish.py (IncidentSubscription): same
* foolscap/logging/incident.py
(IncidentReporter.finished_recording): tell the logger about the
incident name, so it can publish it.
* foolscap/logging/log.py
(FoolscapLogger.addImmediateIncidentObserver): same
(FoolscapLogger.incident_recorded): same
* foolscap/test/test_logging.py (IncidentPublisher._check_listed):
same
(IncidentPublisher.test_subscribe): test it
* foolscap/logging/interfaces.py (RISubscription.unsubscribe):
move the RILogPublisher.unsubscribe() method to the RISubscription
object, since that's a better place for it.
(RILogPublisher.unsubscribe): deprecate this one
* foolscap/logging/publish.py (Subscription.remote_unsubscribe): same
* foolscap/logging/publish.py (LogPublisher.remote_get_incident):
reject invalid incident names
* doc/logging.xhtml: add details about running a Log Gatherer
(Python 'logging' module): comment out this section, it is wrong
2008-07-09 Brian Warner <warner@lothar.com>
* doc/specifications/logfiles.xhtml: fix typos
* doc/logging.xhtml (Remote log aggregation): same
2008-07-07 Brian Warner <warner@lothar.com>
* foolscap/logging/dumper.py (LogDumper): when dumping an Incident
Report, mark the triggering event with "[INCIDENT-TRIGGER]"
* foolscap/_version.py (verstr): bump revision to 0.2.9+ while
between releases
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
2008-07-02 Brian Warner <warner@lothar.com>
* foolscap/_version.py (verstr): release Foolscap-0.2.9
* misc/{dapper|edgy|feisty|gutsy|hardy|sarge|etch|sid}/debian/changelog:
same
* NEWS: update for the upcoming release
* MANIFEST.in: add gutsy/hardy directories
2008-07-02 Brian Warner <warner@allmydata.com>
* foolscap/test/test_logging.py
(Publish.test_logpublisher_overload): change poller condition to
avoid spurious failures on slow systems
* misc/*/debian/watch: update to point at foolscap.lothar.com
* Makefile (debian-gutsy, debian-hardy): add .deb targets for
gutsy and hardy. The rules are the same as for feisty. Closes #76.
* misc/gutsy/*, misc/hardy/*: same
* foolscap/logging/publish.py (Subscription.send): change the
discard policy to discard-new instead of discard-random: in tests
with a busy Tahoe node, this seems to give good enough
behavior (probably since the busyness is bursty), although I can't
say we fully understand what's really going on. discard-new lets
us use a faster and simpler deque instead of requiring random
access to the list.
* foolscap/test/test_logging.py
(Publish.test_logpublisher_overload): update to match
2008-07-02 Brian Warner <warner@lothar.com>
* foolscap/logging/publish.py (Subscription.send): add a
size-limited queue for sending messages to subscribers, with a
default queue size of 2000 entries. This should probably keep the
memory footprint bounded to perhaps 1MB. When the queue gets full,
we randomly discard old messages, so recent messages are more
likely to survive than earlier ones. We allow 10 messages to be
outstanding on the wire at once, to pipeline them a bit and
improve network utilization. Any errors during sending will cause
the subscription to be dropped. This should close #72.
* foolscap/logging/log.py (FoolscapLogger.addImmediateObserver):
add a new kind of observer, so that the logport publisher can also
avoid unboundedness in the eventual-send queue. If we can't throw
away messages fast enough, callers to log.msg will block, slowing
down the inlet rate.
* foolscap/test/test_logging.py
(Publish.test_logpublisher_overload): test it, by throwing 10k
messages at log.msg and counting how many make it through.
* foolscap/test/common.py (PollMixin): add a comment: this
chained-Deferred pattern will run up against python's recursion
limit if the check function is called more than about 300 times.
2008-07-01 Brian Warner <warner@lothar.com>
* foolscap/logging/publish.py (Subscription): refactor a bit, in
preparation for #72 limit log-publishing queue size
2008-07-01 Brian Warner <warner@allmydata.com>
* doc/logging.xhtml ("That Was Weird" Buttons): provide a simple
example of giving the user a way to trigger Incident logging.
Closes #75.
* foolscap/logging/interfaces.py (RILogPublisher.get_pid): new
interface, to retrieve the process ID through the logport
* foolscap/test/test_logging.py
(Publish.test_logpublisher_catchup._got_logport._check_pid): test it
* foolscap/logging/publish.py (LogPublisher.remote_get_pid):
implement it
* foolscap/logging/tail.py (LogTail._got_logpublisher._announce):
Use it, but tolerate old logports that don't offer it. Closes #71.
* foolscap/slicers/decimal_slicer.py: handle Decimal objects,
serializing them as a string. No constraints yet. Closes #50.
* foolscap/slicers/allslicers.py: import decimal_slicer
* foolscap/test/test_banana.py (ThereAndBackAgain.test_decimal):
test it
* foolscap/logging/tail.py (LogTail._got_logpublisher): exit if we
get an error while connecting or subscribing. Closes #63.
(LogTail._print_versions): print remote versions immediately after
connecting to the logport. Closes #70.
* foolscap/logging/log.py (FoolscapLogger.setLogDir): oops, allow
the incident directory to be re-used: os.makedirs throws an
exception if the directory already exists.
* foolscap/test/test_logging.py (Incidents.test_basic): test it
* foolscap/_version.py (verstr): bump revision to 0.2.8+ while
between releases
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2008-06-04 Brian Warner <warner@lothar.com>
* foolscap/_version.py (verstr): release Foolscap-0.2.8
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
* NEWS: update for the upcoming release
2008-06-04 Brian Warner <warner@allmydata.com>
* foolscap/pb.py (Tub._tubsAreNotRestartable): fix args so that
these methods actually get run (and produce a useful error
message, instead of TypeError). Thanks to Brian Granger for the
patch. Closes #65.
* foolscap/test/test_tub.py (Shutdown.test_doublestop): test it
* foolscap/test/common.py (ShouldFailMixin.shouldFail): add some
support code
* foolscap/referenceable.py (RemoteReferenceOnly.getPeer): new
method to get the IP address and port number of the other end of a
connection. This returns a twisted.internet.interfaces.IAddress
provider. Loopback connections give
foolscap.broker.LoopbackAddress instances. Real remote connections
give twisted.internet.address.IPv4Address instances, so you can use
rref.getPeer().host and rref.getPeer().port on them. Closes #45.
* setup.py: add an "extras_require" clause to the
setuptools-specific setup args, to declare that our
"secure_connections" feature requires pyOpenSSL. This helps other
packages, which can declare a dependency on
"Foolscap[secure_connections]", rather than claiming to require
pyOpenSSL themselves. Addresses #66.
* foolscap/pb.py (Tub.registerReference): fix an exception that
occurs if you call this with both name= and furlFile= and the
furlFile already exists. Also prohibit attempts to change the name
to something other than what is in the furlFile. Thanks to Brian
Granger for the patch. Closes #64.
* foolscap/_version.py (verstr): bump revision to 0.2.7+ while
between releases
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2008-05-13 Brian Warner <warner@lothar.com>
* foolscap/_version.py (verstr): release Foolscap-0.2.7
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
* NEWS: update for new "OMG flogtool is broken" release.
2008-05-12 Brian Warner <warner@allmydata.com>
* foolscap/logging/cli.py (run_flogtool): fix use of sys.argv,
the previous version was completely broken
* foolscap/_version.py (verstr): bump revision to 0.2.6+ while
between releases
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2008-05-06 Brian Warner <warner@lothar.com>
* foolscap/_version.py (verstr): release Foolscap-0.2.6
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
* NEWS: update for the upcoming release
2008-05-05 Brian Warner <warner@allmydata.com>
* foolscap/test/test_logging.py (Web): add tests for 'flogtool
web-viewer'
* foolscap/logging/web.py (WebViewer.start): make this class more
amenable to being tested
* foolscap/logging/interfaces.py: misc cleanups
(RILogPublisher.list_incidents): provide an interface to retrieve
stored incident reports from a logport.
(RILogPublisher.get_incident): same
* foolscap/logging/publish.py (LogPublisher): same
* foolscap/test/test_logging.py (IncidentPublisher): test it
* foolscap/pb.py (Tub.setup): make the Tub's logger a bit easier
to override, for tests
* foolscap/logging/incident.py (IncidentReporter): make the
compressed logfile use a temporary name until we've closed it, so
later observers don't get tricked into using an incomplete
compressed logfile (they should use the more-complete uncompressed
logfile instead).
(IncidentReporter.incident_declared): make it easier to turn off
the gather-trailing-events behavior
(NonTrailingIncidentReporter): convenience subclass that does that
* doc/logging.xhtml: update to match current reality, remove some
TODO warnings
* foolscap/logging/log.py (FoolscapLogger): rename
setIncidentReporterClass to setIncidentReporterFactory
* foolscap/test/test_logging.py (Incidents.test_customize): same
* foolscap/logging/interfaces.py (RISubscription): remove spurious
'pass', figleaf thought it was real code
* bin/flogtool: split out the CLI dispatcher to..
* foolscap/logging/cli.py (run_flogtool): here, to help with #51
(run_flogtool): make it possible to run with a wrapper, closes #51.
* foolscap/test/test_logging.py (CLI.test_wrapper): test a wrapper,
provide an example of how to build one
* foolscap/logging/gatherer.py (CreateGatherOptions.optFlags): add
a --quiet flag to make the unit test less noisy
* misc/testutils/trial_figleaf.py: make this compatible with
twisted-8.0.x: the earlier version didn't write out any coverage
data on newer twisteds
* foolscap/logging/log.py (FoolscapLogger.setLogDir): wire up
Incident handling. We still need incident publishing. This gets
us most of the way to #61.
(setIncidentQualifier): this customizes the qualification function
(setIncidentReporterClass): this customizes the reporter/recorder
* foolscap/logging/levels.py: move log.WEIRD and friends here to
avoid circular import problems elsewhere. They are still available
from log.py .
* foolscap/logging/incident.py (IncidentReporter.incident_declared):
flush the uncompressed logfile just before we switch into
gather-trailing-events mode
(IncidentReporter.stop_recording): oops, fix typo
* foolscap/test/test_logging.py (Incidents): test most
incident-handling functionality
2008-05-02 Brian Warner <warner@allmydata.com>
* foolscap/logging/incident.py: start to implement Incident
handling, for ticket #61. Not complete yet, might be completely
broken, needs tests and to be wired up.
* foolscap/logging/log.py (FoolscapLogger.setIncidentQualifier):
start to add the new interfaces, not complete yetxo
* foolscap/logging/interfaces.py (IIncidentReporter): same
* doc/logging.xhtml (Incidents): document the new features
2008-05-01 Brian Warner <warner@allmydata.com>
* doc/logging.xhtml: fix discussion of format=, since the
instructions and examples were simply wrong
* doc/specifications/logfiles.xhtml (Logfile Headers): define
"headers", a separate dictionary at the start of the logfile that
contains metadata. The specific use for this will be the
"Triggering Event" that gets put into incident reports, once we
implement those. This induces a backwards compatibility break:
logfiles produced after this change will probably not be tolerated
by tools like 'flogtool dump' from before this point. The other
direction is ok: newer tools can handle either format.
* foolscap/logging/gatherer.py (LogGatherer._open_savefile): emit
a header, with type="gatherer", and a starting timestamp in
"start". Refactor a bit to make this easier.
* foolscap/logging/tail.py (LogPrinter): emit a header, with
type="tail"
* foolscap/logging/log.py (LogFileObserver): emit a header, with
type="log-file-observer", and a "threshold" key
* foolscap/logging/dumper.py (LogDumper.start): tolerate headers
* foolscap/logging/filter.py (Filter.run): same
* foolscap/logging/web.py (WebViewer.process_logfiles): same
* foolscap/test/test_logging.py: update tests to match
* doc/specifications/logfiles.xhtml: document the current saved
logfile format (event dictionaries, pickled wrapper dicts)
* foolscap/referenceable.py (SturdyRef): accept multiple
connection hints in unauthenticated FURLs. This fixes a test
failure induced by the #60 changes when pyOpenSSL is unavailable.
* doc/using-foolscap.xhtml: document the new feature.
* foolscap/referenceable.py (SturdyRef): tolerate extensions in
tubid and location-hints fields, change FURL parsing, store
structured hints in the SturdyRef instead of just strings. This
should give us some wiggle room in the future to gracefully
transition applications to using new features while retaining
backwards compatibilty. Many thanks to Zooko for the suggestion.
Closes #60.
* foolscap/negotiate.py (TubConnector): same
* foolscap/test/test_sturdyref.py (URL.testTubIDExtensions): new
tests for it
(URL.testLocationHintExtensions): same
* foolscap/test/test_tub.py (SetLocation.test_set_location):
update to match
* foolscap/test/test_gifts.py (Bad): same
* foolscap/test/test_negotiate.py
(Versus.testVersusHTTPServerAuthenticated): same
* foolscap/base32.py (is_base32): new utility function
* foolscap/test/test_util.py (Base32.test_is_base32): test it
2008-04-22 Brian Warner <warner@lothar.com>
* doc/listings/command-client.py: don't emit extra newlines, use
/usr/bin/env on shbang line
* doc/listings/command-server.py: use /usr/bin/env on shbang line
2008-04-22 Brian Warner <warner@allmydata.com>
* foolscap/logging/filter.py: add "--above UNUSUAL" option (to
discard events below the given level", and "--from [TUBID]" (to
discard events that weren't recorded by the given tubid).
* foolscap/logging/dumper.py (LogDumper.print_event): use
--rx-time to show both event-generation time and event-receive
time in the logs, useful if you suspect the application is getting
bogged down and events are being delivered slowly.
(LogDumper.print_event): Use a different format to display failure
tracebacks.
* foolscap/logging/web.py (WebViewer.get_events): catch
ValueError, mention it as a possible truncated pickle file
* foolscap/logging/log.py (FoolscapLogger.msg): if something goes
wrong, print both str() and repr() in case it helps figure out the
problem
* foolscap/logging/dumper.py (LogDumper.print_event): include
failure tracebacks in 'flogtool dump' output
2008-04-09 Brian Warner <warner@allmydata.com>
* foolscap/constraint.py (Constraint.checkOpentype): always
accept ('reference',) sequences. The per-token constraint checking
system is a defense against resource-exhaustion attacks, and
shared reference don't consume any more memory or stack frames
than any other object. The check-all-args that CallUnslicer does
just before delivering the arguments is responsible for making
sure the final (resolved) types all match the constraint. By
allowing shared references here, we fix a bug in which a schema
violation was raised when python combined two equivalent tuples
into a single object.
* foolscap/test/test_call.py (TestCall.testMega3): test it
* foolscap/test/common.py (MegaSchema3): same
* setup.py: finally remove zip_safe=False
* foolscap/test/test_interfaces.py (TestInterface.testStack):
allow the test to pass even if the source code isn't
available (i.e. it's locked away inside an egg). This ought to
remove the need for zip_safe=False.
* foolscap/test/test_logging.py (Tail.test_logprinter): make the
test pass on python2.5: I think twisted now emits type(exception)
instead of str(exception), or something: the printed form changed.
* foolscap/schema.py (PolyConstraint): improve error messages
* foolscap/test/test_logging.py (Tail.test_logprinter): improve
error messages, to figure out why this fails on python2.5
* foolscap/__init__.py (_unused): add __version__, to hush pyflakes
* foolscap/test/test_logging.py (Publish.test_logpublisher): make
the test work with twisted-8.0.x, which puts more information in
log.err Failures than earlier versions did.
* foolscap/__init__.py (__version__): move version from here..
* foolscap/_version.py (verstr): .. to here
* foolscap/setup.py: merge zooko's patch to get foolscap version
by scanning foolscap/_version.py instead of importing foolscap.
This makes setuptools a lot happier: it doesn't need to have
Twisted installed (so foolscap can be imported) while it's
building foolscap as part of automatic dependency satisfaction.
Also change the shebang line to use /usr/bin/env, and import (but
do not use) setuptools if it is available. We set 'zip_safe=False'
in the extra setuptools arguments because two of the unit tests
assert that their stack traces have source code lines in them,
and that doesn't happen if foolscap is living inside an egg.
We will probably change those tests soon and allow zipped eggs.
2008-03-31 Brian Warner <warner@allmydata.com>
* foolscap/test/test_logging.py (Tail): basic tests for
logging.tail
* foolscap/logging/tail.py (LogPrinter.remote_msg): parameterize
the output filehandle so we can test it
* foolscap/logging/gatherer.py (LogGatherer): remove the "Z"
suffix from the from- and to- timestamps that we put into
filenames, since we're actually using localtime. If and when we
switch to use UTC, we'll bring the Z back. Also remove the hyphens
between the time portions: from-2008-03-31-161721 instead of
from-2008-03-31-16-17-21. I'm still looking for a format that
feels readable, scannable, and clear.
* foolscap/logging/tail.py (LogPrinter.formatted_print): remove
spurious "0" from timestamps: i.e. print "08:09:12.345" instead of
"08:09:120.345"
* foolscap/logging/web.py (LogEvent.to_html): same
* foolscap/__init__.py: bump revision to 0.2.5+ while between releases
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2008-03-25 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.2.5
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2008-03-25 Brian Warner <warner@lothar.com>
* MANIFEST.in: add LICENSE
* NEWS: update for the upcoming release
* bin/flogtool (Options.synopsis): update with all commands.
There's still something broken with --help output, though.
* doc/listings/command-server.py:
* doc/listings/command-client.py: new sample programs, a
client/server pair which lets the client trigger a specific
command to be run on the server. Like xfer-server.py, but for
running commands instead of transferring files.
* foolscap/test/common.py (PollMixin): refactor a bit, to extract
the poll() method for use by other tests
* foolscap/test/test_logging.py (Publish.test_logpublisher):
improve the timing a bit, by waiting until the observer has heard
only silence for a full second, instead of starting the verify
pass one second after subscribing.
(Publish.test_logpublisher_catchup): same. It makes a bigger
difference here, because catch_up=True means that we'll be seeing
several hundred messages, which may take a non-trivial amount of
time to receive. I was seeing intermittent test failures with the
one-second-from-subscribe stall.
2008-03-24 Brian Warner <warner@lothar.com>
* foolscap/logging/interfaces.py (RILogPublisher.subscribe_to_all):
add a catch_up= argument, which causes the publisher to dump all
its stored messages just after adding the subscriber. Closes #49.
* foolscap/logging/log.py (FoolscapLogger.get_buffered_events): same
(LogFileObserver.msg): catch+print exceptions while pickling events
* foolscap/logging/publish.py (LogPublisher): add catch_up=, make
sure that any log events arrive after subscribe_to_all() has
returned, to make event sequencing easier on the subscriber. Also
make sure that any catch-up events arrive before subsequent log
events.
* foolscap/logging/tail.py: add --catch-up option, make sure we
remain compatible with <=0.2.4 publishers (as long as you don't
use --catch-up)
* foolscap/test/test_logging.py (Publish.test_logpublisher_catchup):
test it
(Publish.test_logpublisher_catchup): make test more reliable
* LICENSE: make it clear that Foolscap ships under the MIT license,
the same as Twisted uses. Closes #47.
* README: same
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/copyright: same
2008-03-24 Brian Warner <warner@allmydata.com>
* foolscap/logging/dumper.py (LogDumper.print_event): event dicts
store the printable tubid, not a binary form
* foolscap/logging/gatherer.py (Observer.__init__): same
(LogGatherer.remote_logport): same
* foolscap/logging/interfaces.py (TubID): same
* foolscap/logging/web.py (LogEvent.__init__): same
* foolscap/logging/gatherer.py (LogGatherer.__init__): make bzip=
argument optional, for unit tests
* foolscap/logging/tail.py: fix --save-to, also put exception
information into the twistd.log when we can't pickle the event
* foolscap/logging/gatherer.py: add --rotate and --bzip options to
'flogtool create-gatherer': rotate the logfile every N seconds,
and optionally compress the results. Each logfile gets named like
from-2008-03-24-20-31-46Z--to-2008-03-24-20-31-56Z.flog.bz2 , and
the open one is named from-2008-03-24-20-32-16Z--to-present.flog .
Sending SIGHUP to the gatherer will force a rotation. Closes #48.
* foolscap/logging/tail.py: add --save-to option to 'flogtool
tail', which saves the log events to a file (in addition to
printing them to stdout). 'flogtool dump' can be used on the saved
file later. Also refactor things a bit to let us grab the
tubid from the target furl, since this gets recorded in the
save file format.
* foolscap/logging/gatherer.py (LogSaver): move to tail.py
* foolscap/logging/tail.py (LogPrinter): print formatted event
lines by default. Use the new --verbose option to dump raw event
dictionaries. Closes #43.
* bin/flogtool (dispatch): same
2008-02-17 Brian Warner <warner@lothar.com>
* doc/using-foolscap.xhtml: fix href, thanks to Stephen Waterbury
for the catch. Note that the .xhtml points to .xhtml, and lore
converts the target of the link to .html in the .html output.
* doc/copyable.xhtml: same
* foolscap/logging/gatherer.py: make 'flogtool create-gatherer' to
build a .tac file (which can be launched with twistd) rather than
starting a gatherer right away. This is much more useable in
practice.
* bin/flogtool: same
2008-01-31 Brian Warner <warner@allmydata.com>
* foolscap/test/test_tub.py: hush pyflakes, make it work without
crypto too
* doc/listings/xfer-server.py:
* doc/listings/xfer-client.py: new sample programs, a
client/server pair which allow the client to put files in the
server's directory. Useful as a replacement for restricted-command
passphraseless ssh key arrangements.
* foolscap/pb.py (Tub.setLocationAutomatically): new method that
guesses an externally-visible IP address and uses it to call
setLocation().
* foolscap/test/test_tub.py (SetLocation.test_set_location): test
for it
* foolscap/util.py (get_local_ip_for): make get_local_ip_for()
more generally available, moving out of gatherer.py
* foolscap/logging/gatherer.py: same
* foolscap/negotiate.py (TubConnector.connectToAll): fix log message
* foolscap/logging/web.py (LogEvent.__init__): include the
incarnation number in the anchor index, so that logfiles which
contain events from multiple incarnations of the same Tub will not
suffer from href collisions
* foolscap/broker.py: switch to foolscap.logging
(Broker.connectionLost): log this event
(Broker.freeYourReference): same, at log.UNUSUAL
(Broker._callFinished): same, at log.UNUSUAL
* foolscap/__init__.py: bump revision to 0.2.4+ while between releases
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2008-01-28 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.2.4
* misc/{sid|sarge|dapper|edgy|feisty}/debian/changelog: same
2008-01-28 Brian Warner <warner@lothar.com>
* NEWS: update for the upcoming release
* foolscap/logging/filter.py (FilterOptions.parseArgs): make it
possible to filter the event file in place, by using 'flogtool
filter filename' instead of 'flogtool filter infile outfile'
* foolscap/negotiate.py (Negotiation.evaluateNegotiationVersion1):
include tubid in error message
* foolscap/call.py (CopiedFailure.__getstate__): don't use
reflect.qual on a string. This might improve behavior when we copy
failures around, particularly in logging.
2008-01-18 Brian Warner <warner@lothar.com>
* foolscap/negotiate.py: put all negotiation flog messages in
facility "foolscap.negotiation", to make it easier to strip them
out with 'flogtool filter --strip-facility foolscap'
* foolscap/call.py (PendingRequest.fail): switch remote-exception
logging over to flogging
(InboundDelivery.logFailure): switch local-exception logging to
flogging too
* foolscap/test/test_pb.py (TestCallable.testLogLocalFailure): match it
(TestCallable.testLogRemoteFailure): same
* foolscap/logging/log.py (FoolscapLogger.__init__): remove unused
.buffer attribute
(FoolscapLogger.msg): avoid pickling application-specific
exception classes when using failure=; use CopiedFailure to avoid
it. This allows the emitted log pickle to be loaded on systems
that do not have the original source code around.
* foolscap/logging/filter.py: add --strip-facility option, to
remove events that pertain to a given facility or its children
2008-01-17 Brian Warner <warner@lothar.com>
* foolscap/logging/web.py (EventView.render): add a sort= query
argument. If set to 'nested' (the default), you get the default
nested view, in which the root events (those without a parent) are
shown in chronological order, and the child events of each node
are shown in chronological order underneath it. If sort=time, then
no nesting is used, and all events are shown in chronological
order. If sort=number, then all events are shown in numerical
order, which is nominally better for single-process logfiles and
coarse timestamps, but practically speaking is no better than
sort=time. Using a non-nested mode can make it easier to spot
events that happen in different areas but at about the same time;
the nested display puts these events further apart
(EventView._emit_events): minor formatting change
2008-01-10 Brian Warner <warner@lothar.com>
* bin/flogtool (Options.opt_version): make 'flogtool --version'
show the foolscap version.
* foolscap/logging/web.py (SummaryView): add summaries to the
'flogtool web-viewer' tool: show counts of events by severity
level, show lists of those events, with each event hyperlinked to
the correct line in the full display. The full display also has
anchor tags which let you construct bookmarks to specific lines.
* foolscap/logging/filter.py (Filter): add 'flogtool filter'
subcommand, to take one large eventlog pickle file and produce
a smaller one with just a subset of the events. Currently this only
allows you to filter by timestamp, and requires timestamps be
provided as seconds since epoch.
* bin/flogtool (dispatch): same
* foolscap/logging/log.py (FoolscapLogger.err): add log.err, which
behaves just like twisted's log.err: it accepts an exception or
Failure object, or it can be used inside an except: clause to
log the current exception.
* foolscap/test/test_logging.py (Publish): test it
* foolscap/logging/gatherer.py (LogSaver.remote_msg): if we can't
pickle something, complain to stdout, rather than causing an error
message to be sent back to the sender's Tub.
* foolscap/call.py (CopiedFailure.__getstate__): make
CopiedFailures pickleable, and make sure they come back looking
just like they started. The issue was that we play games with the
.type attribute to make .trap/.check work.
* foolscap/test/test_copyable.py (Copyable._testFailure1_1): test it
(Copyable._testFailure2_1): same
* Makefile (pyflakes): new pyflakes doesn't uniqueify its output
* NEWS: fix misspelling
* foolscap/__init__.py: bump revision to 0.2.3+ while between releases
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2007-12-24 Brian Warner <warner@lothar.com>
* foolscap/__init__.py (__version__): release 0.2.3
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2007-12-24 Brian Warner <warner@lothar.com>
* setup.py: point url= at the trac page, instead of the root,
since I keep forgetting to update the tarball links on the root
* NEWS: update for the upcoming release
* foolscap/logging/log.py (LogFileObserver.__init__): open the
flogfile with mode "wb" instead of "a", so that both compressed
and uncompressed files work the same way. This truncates the file
on each run instead of appending to it.
* foolscap/broker.py (Broker.getRemoteInterfaceByName): remove
duplicate definition of method, detected by pyflakes
* foolscap/test/test_banana.py: change import of names from
foolscap.tokens to appease the new (stricter) pyflakes
* foolscap/logging/gatherer.py (get_local_ip_for): move imports up
to module level, remove duplicates
* foolscap/pb.py (Tub.setLogPortFURLFile): if the application has
done setOption("logport-furlfile"), make sure the logport furlfile
is created as soon as possible (i.e. when setLocation is called).
Allow these two calls to occur in either order. The setOption call
must still occur before doing getLogPortFURL(), or before the tub
connects to the log gatherer; otherwise the furlfile will be
ignored. Closes #38.
* foolscap/test/test_logging.py (Publish.test_logport_furlfile1):
test it
* doc/logging.xhtml: update to match, fix a few typos
* foolscap/referenceable.py (RemoteReferenceOnly.getSturdyRef):
fix this, it had bitrotted. Closes #35.
* foolscap/test/test_pb.py (TestCallable.testGetSturdyRef): test it
* foolscap/logging/log.py (FoolscapLogger.msg): only record
e['args'] if there were any, to avoid spurious formatting attempts
* foolscap/negotiate.py: update log.msg calls to use format=
* foolscap/logging/log.py (format_message): refactor event
formatting into a separate function. Switch to using Twisted's
style of format= argument to indicate that we want to use
keyword-argument formatting. Closes #39.
(FoolscapLogger.msg): same, use format_message() to test whether
the event is stringifiable or not.
(TwistedLogBridge._old_twisted_log_observer): copy dict directly
if there's a format= kwarg, or stringify the message portion if
there isn't. I *think* this ought to match what twisted does.
* foolscap/logging/dumper.py (LogDumper.print_event): same
* foolscap/logging/web.py (LogEvent.to_html): use format_message
* foolscap/test/test_logging.py (Basic.testLog): improve tests
2007-12-23 Brian Warner <warner@lothar.com>
* foolscap/negotiate.py (Negotiation.__init__): log everything
under the 'foolscap.negotiation' facility, and use NOISY by
default.
(Negotiation.log): same
* foolscap/logging/log.py (NOISY): log levels are now defined in
terms of their stdlib 'logging' counterparts, so they're ints from
a larger scale (10 to 40). This removes the need for 'levelmap',
and allows log levels to be compared better.
(FoolscapLogger.msg): all messages get a ['level'] key (which
defaults to OPERATIONAL), so that observers and viewers don't have
to define a default or handle a missing key.
(FoolscapLogger.add_event): observers are now a simple callable,
and they're always invoked with eventually(). log-to-file is now
done with an observer, as are remote subscribers.
(FoolscapLogger.logTo): remove this, log-to-file is now done with
an observer
(LogFileObserver): new class to implement log-to-file. This
accepts a minimum level to pay attention to. Pickling is done at
level '2'. It will compress the output if the filename ends in
.bz2, and will use twisted's reactor.addSystemEventTrigger to try
and close the file at shutdown. This is good enough for trial test
cases, since trial helpfully fires this trigger for us after all
test cases have been run.
(FLOGFILE): if the $FLOGFILE environment variable is set, write
all log events of $FLOGLEVEL or higher to the named file (opened
at import time). Set FLOGLEVEL=1 to include NOISY debug messages.
Set FLOGTWISTED=1 to get twisted.log events in the same file. Use
all three when running trial tests to see what foolscap is saying
as the tests run, but note that you can't do
FLOGFILE=_trial_temp/flog.out.bz2 because foolscap is usually
imported before trial creates (or clears) that directory, thus
deleting the newly-created flogfile. Applications are expected to
use a better API to control log-to-file, but it doesn't really
exist yet.
* foolscap/logging/dumper.py (LogDumper): accept .bz2 files
* foolscap/logging/web.py (WebViewer.get_events): same
* foolscap/logging/publish.py (LogPublisher): observers are now
simple callables, so build a wrapper to do the callRemote for us.
Track both the wrapper and the subscriber, since we need to use
them both at unsubscribe time.
* foolscap/test/test_logging.py (Basic.testLog): make sure we can
tolerate '%' in log messages. (this needs to be changed, see #39).
Also test the generation threshold, and using num=
(Advanced.testObserver): test the new observer-as-callable scheme
(Advanced.testPriorities): facility names are now dot-separated
2007-12-21 Brian Warner <warner@lothar.com>
* doc/logging.xhtml: define facilities as dot-separated instead of
slash-separated, to match stdlib logging package
* foolscap/__init__.py: bump revision to 0.2.2+ while between releases
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2007-12-12 Brian Warner <warner@lothar.com>
* foolscap/__init__.py (__version__): release 0.2.2
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2007-12-12 Brian Warner <warner@lothar.com>
* NEWS: update for new release
* foolscap/negotiate.py (Negotiation): fix duplicate-connection
handling logic. Simplify by removing the attempt-id information,
and thus reject any connections with old seqnums (to prevent
connection flap for parallel (connection-hint) offers). Reject
connections from master_IR=None. Send the 'my-incarnation' field
with all offers (both client and server), so that it will be
available to whomever the master happens to be.
* foolscap/broker.py (Broker.__init__): remove attempt-id
* foolscap/test/test_negotiate.py (Replacement): update to match
* foolscap/negotiate.py: covert logging to flogging (i.e. call
foolscap.logging.log.msg instead of twisted.log.msg). This gives
us parent/child structure, more useful severity levels, improves
display of Failure instances, and will give us more data to work
with in the future as the foolscap.logging tools mature.
* foolscap/pb.py: same
* foolscap/test/test_negotiate.py: same
* bin/flogtool: move all usage.Options into the file that contains
the implementations: TailOptions are moved to
foolscap.logging.tail, etc.
* foolscap/logging/gatherer.py: same
* foolscap/logging/tail.py (TailOptions): same
* foolscap/logging/dumper.py: same
(DumpOptions.optFlags): add --verbose (show all event keys),
--just-numbers.
(LogDumper.print_event): change event printing to handle funny
messages
* foolscap/logging/web.py (WebViewerOptions.optParameters): same
(FLOG_CSS): colorize the background of unusual messages
(LogEvent): improve stringification, add better timestamps,
event numbers, escape HTML better, display Failure instances better,
(WebViewer.process_logfiles): event numbers use 'num', not 'number'
* foolscap/logging/log.py (FoolscapLogger.set_generation_threshold):
implement generation thresholds: don't even record messages if
they fall below this threshold. Still needs a lot of usability work.
The default is level>=NOISY, which records everything.
(FoolscapLogger.logTo): give this a filename, and all log messages
will be pickled and written to the given file. Messages are
written with the same dict wrapper as 'flogtool gather' uses, so
they are displayable by 'flogtool dump' or 'flogtool web-view'.
Still needs work, ideally this would be a file to which we dump
buffered messages once a problem is detected. To enable this from,
say, trial, set os.environ['FLOGFILE'] to a filename. Also, set
FLOGTWISTED to enable a twisted.log-to-flog bridge.
(FoolscapLogger.msg): implement generation_threshold, also change
stringification to survive things like "100%" in the log message.
This needs work too, the 'except ValueError' clause is icky.
* foolscap/pb.py (Tub.brokerAttached): use an eventual-send when
informing everyone in waitingForBrokers, to match the
asynchronicity of disconnection notifications delivered in
Broker.finish()
2007-12-11 Brian Warner <warner@allmydata.com>
* foolscap/call.py (PendingRequest.fail): tolerate pass-then-fail,
which is just as weird of an error case but shouldn't cause an
exception. The fact that this is necessary indicates significant
problems in the new connection-management code. Sigh.
* foolscap/banana.py (Banana.handleError): loseConnection takes a
Failure, not an exception. Addresses #36.
* foolscap/negotiate.py (Negotiation.acceptDecisionVersion1): same
* foolscap/test/test_call.py: same
* foolscap/test/test_reconnector.py (Reconnector._got_ref): same
* foolscap/broker.py (Broker.shutdown): assert that we get a Failure
* foolscap/test/common.py (Loopback.loseConnection): same
* foolscap/__init__.py: bump revision to 0.2.1+ while between releases
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2007-12-10 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release 0.2.1
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
* foolscap/broker.py (Broker.connectionTimedOut): sigh, brown
paper bag bug: broker.shutdown() requires a Failure, not an
exception. Unfortunately the unit tests didn't catch this.
* foolscap/pb.py (Tub.stopService): same
* foolscap/negotiate.py (Negotiation.evaluateNegotiationVersion1):
same
* foolscap/logging/interfaces.py: give fully-qualified
__remote_name__ strings to all RemoteInterfaces, to avoid
collision with other code that might use these names.
2007-12-10 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release 0.2.0
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2007-12-10 Brian Warner <warner@allmydata.com>
* setup.py: add the foolscap/logging package
* NEWS: update for upcoming release
* foolscap/negotiate.py (Negotiation.handle_old): finally
implement this: if enabled, "old" brokers (older than 60 seconds)
will be replaced by new offers, but "new" ones are not. The
default timeout is 60 seconds, but you can set it to something
else by calling tub.setOption("handle-old-duplicate-connections",
120) instead of using 'True'. Closes #34.
* foolscap/pb.py (Tub.setOption): same
* foolscap/broker.py (Broker.__init__): record creation timestamp
* foolscap/test/test_negotiate.py
(Replacement.testAncientClientWorkaround): test it
2007-12-10 Brian Warner <warner@lothar.com>
* doc/logging.xhtml: updates
* foolscap/reconnector.py (Reconnector.reset): add a utility
method to manually force a Reconnector to reconnect, most useful
from a manhole or other in-process eval loop. Closes #30.
(Reconnector.getDelayUntilNextAttempt): another to query the delay
until the next connection attempt will be made. Returns None if no
attempt is currently scheduled.
(Reconnector.getLastFailure): and another to provide the last
failure, useful if you want to find out why it keeps reconnecting.
* foolscap/negotiate.py (Negotiation.compareOfferAndExisting): if
the seqnum is too old, accept the offer anyways: this handles the
lost-a-decision-message case better at the expense of worse
handling of offers-delivered-out-of-order case (which is far less
likely to occur than a lost decision message)
* foolscap/test/test_negotiate.py (Replacement): more tests
(Replacement.testBouncedClient_Reverse): same
* foolscap/logging/log.py (FoolscapLogger.msg): handle two kinds
of argument formatting: log.msg("%d>%d", a, b), and
log.msg("%(foo)s!", foo=something). Also handle the twisted_log
equivalent of the dict case: log.msg(format="%(foo)s", foo=blah).
* foolscap/test/test_logging.py (Basic.testLog): test it
(Publish.test_logpublisher._got_logport._check_observer): same
2007-12-07 Brian Warner <warner@allmydata.com>
* foolscap/negotiate.py (Negotiation.compareOfferAndExisting): new
duplicate-connection-handling, for ticket #28. Details in the
ticket, and need to be put in the docs. Each end stores and sends
more information to give the master a better chance of detecting
race conditions correctly. The end result is that silently-lost
TCP connections (due to NAT timeouts or laptops being yanked from
one network to another) should not cause 35-minute reconnector
delays any longer. Many many thanks to Rob Kinninmont and Zooko
for their invaluable help in finding a good solution to this
problem. Hopefully this closes #28.
(Negotiation.handle_old): placeholder method for the code to
handle <=0.1.7 clients, not yet written.
(Negotiation.acceptDecisionVersion1): record information from the
decision in the Broker, so we can compare against it later
* foolscap/broker.py (Broker.__init__): same
* foolscap/pb.py (Tub.setup): record an "Incarnation Record", just
a unique (random) string, so that remote servers can tell if we
ought to remember them or not.
(Tub.setOption): add new 'handle-old-duplicate-connections' option,
for the <=0.1.7-client-handler code (not yet written). This option
name might change.
* foolscap/test/test_negotiate.py (Replacement): test much of the
new code, probably covers 60% of the code paths. Still need to
test the opposite direction (client==master) to make sure I didn't
break anything.
* Makefile (.figleaf.el): copy more tools from Tahoe: convert figleaf
coverage data into an emacs-lisp -parseable format
* misc/testutils/figleaf.el: elisp code to highlight uncovered lines
* misc/testutils/figleaf_htmlizer.py: emit lines *not* covered in a
separate column, and sort by that: this makes it a lot easier to
pay attention to the places that need work. Code copied over
from Tahoe.
2007-12-06 Brian Warner <warner@allmydata.com>
* foolscap/broker.py (Broker.shutdown): refactor shutdown code, to
make it safer to replace one connection with another.
(Broker.abandonAllRequests): use an eventual-send for each
flunking, to make sure the connection is completely gone by the
time application code gets to run in the errbacks.
* foolscap/pb.py (Tub.stopService): give a reason for the
shutdown. Keep it a ConnectionDone (so we don't need to update
other code that wouldn't expect a new exception type), but mention
Tub.stopService in the arguments.
(Tub.brokerAttached): add assert self.running.. this should never
be called on a Tub that's already been shutdown.
(Tub.brokerDetached): fix last-broker-detached detection, since
the sequencing changed a bit. Make sure we don't fire the
observerlist twice, by only firing it when we actually remove a
broker and it was the last one (i.e. we started with some brokers,
and finished with none).
(Tub.__repr__): add the TubID to the repr
* foolscap/call.py (FailureSlicer.getStateToCopy): truncate
.value, .type, and .parents to the lengths defined in
FailureConstraint. Unfortunately we don't seem to impose these
constraints during serialization, so if some exception happens to
have a really long name or arguments, it's the recipient of the
CopiedFailure who will complain, making it hard to figure out
where the real problem lies. The fact that the Violation doesn't
seem to specify which attribute was in violation only adds
confusion: it said <RootUnslicer>.<error-2>.??? for a 'token too
large: 1819>1000', in RemoteCopyUnslicer.checkToken, which was
probably f.value since that's the only 1000byte constraint.
2007-11-29 Brian Warner <warner@lothar.com>
* foolscap/pb.py (Tub._log_gatherer_connected): for
UnauthenticatedTubs, send tubid="<unauth>" instead of tubid=None,
since RILogGatherer.logport is expecting a string. Ideally I'd
like this to be ChoiceOf(str,None), but ChoiceOf doesn't work. In
the long run UnauthenticatedTubs will acquire normal (but
distinct) tubids, so this issue will go away.
* foolscap/logging/interfaces.py: minor cleanup
* foolscap/test/test_logging.py (Publish._test_gatherer._check):
update to match
* foolscap/constraint.py: improve various Violation messages
* foolscap/logging/log.py: get logport/gatherers working the way
they're described in the docs
* foolscap/logging/publish.py: merge into log.py
* foolscap/logging/interfaces.py: improve imports
* foolscap/pb.py: fix API for access to this stuff
* foolscap/test/test_logging.py: fix up tests to match
* doc/logging.xhtml: more cleanup
2007-11-28 Brian Warner <warner@lothar.com>
* Makefile: misc new targets, parameterize $(TRIAL)
* doc/logging.xhtml: document new logging scheme, some of this is
still speculative, unimplemented, or just plain wrong
* foolscap/logging/log.py: frontend for logging scheme
* foolscap/pb.py: partial API support for logging
* foolscap/test/test_logging.py: tests for new code, partially adapted
from Petmail log tests
2007-11-27 Brian Warner <warner@lothar.com>
* foolscap/logging/*: new logging support, including hierarchical
logging, remote publishing, gathering tools
* foolscap/test/test_logging.py: minimal tests for it
* bin/flogtool: command-line tool to view/gather log events
* setup.py (scripts=): include flogtool
2007-10-16 Brian Warner <warner@lothar.com>
* foolscap/test/test_gifts.py (Bad.test_location._introduce): use
127.0.0.1:2 as the bogus port, instead of 127.0.0.47:1 .
Connecting to .47 causes a long delay on OS-X (probably because of
some sort of personal firewall), whereas .1 fails right away. Port
2 is not listed in my copy of /etc/services and seems unlikely to
have a real service running on it.
2007-09-24 Brian Warner <warner@lothar.com>
* doc/serializing.xhtml: oops, fix closing tags
* foolscap/__init__.py: bump revision to 0.1.7+ while between releases
* misc/*/debian/changelog: same
2007-09-24 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release 0.1.7
* misc/{dapper|edgy|etch|fesity|sarge|sid}/debian/changelog: same
2007-09-24 Brian Warner <warner@lothar.com>
* setup.py: remove download_url, I think url= is sufficient.
* NEWS: update for the upcoming release
* all: remove default size limits on Constraints. Users who wish
to enforce size limits should provide a maxLength= argument to
their Constraint constructors. These limits turned out to be more
surprising than helpful. Closes #26.
* foolscap/constraint.py (everythingTaster): remove the default
SIZE_LIMIT on STRING tokens.
(ByteStringConstraint.__init__): and on StringConstraint
* foolscap/slicers/unicode.py (UnicodeConstraint.__init__): and here
* foolscap/slicers/list.py (ListConstraint.__init__): and here
* foolscap/slicers/dict.py (DictConstraint.__init__): and here
* foolscap/slicers/set.py (SetConstraint.__init__): and here
* foolscap/test/test_schema.py (CreateTest.testMakeConstraint):
update tests to match
* foolscap/test/test_copyable.py (MyRemoteCopy4.stateSchema): same
2007-09-16 Brian Warner <warner@lothar.com>
* foolscap/pb.py (Tub.debug_listBrokers): add a debugging method
that tells you about all the connected brokers, and what methods
are outstanding (both inbound and outbound) for each. If you see
any method is sitting in one of these lists for a long time, there
might be a problem in the execution of that method.
* foolscap/broker.py (Broker.doNextCall): don't allow slow
remote_foo methods to stall subsequent calls. This fixes a major
message-delivery bug. Closes #25.
* foolscap/test/test_call.py (TestCall.testStallOrdering): test it
* all: merge in serialization-refactoring branch, adding
Tub.(un)serialize (which can handle Referenceables) and moving us
slightly closer to Sealers/Unsealers. The regular
storage.serialize can now handle Copyables and objects for which
you've registered an ISlicer adapter. Eventually the default
unserialization interface will be 'safe' (meaning it won't create
instances of arbitrary classes), and you'll have to give it
additional arguments to enable 'unsafe' behavior.
* doc/serializing.xhtml: document it
* foolscap/banana.py: create slicer/unslicer at connectionMade,
not __init__. Also change the way that errors are handled, and
remove the use of 'types'.
* foolscap/broker.py (PBRootUnslicer.open): refactor, move code
to slicers.root.RootUnslicer
(Broker.use_remote_broker): new attribute, True for connected
brokers, False for the non-connected one that Tub.serialize uses.
This is used by ReferenceableSlicer to decide whether to emit a
my-reference sequence or a their-reference sequence.
(StorageBroker): new subclass of Broker, uses new
StorageBrokerRoot(un)Slicer. This broker is specialized to accept
exactly one object, and hand it off to a waiting Deferred. There's
still some useful refactoring to do, to mix in the behavior of
ScopedRootUnslicer better.
* foolscap/debug.py: delete this, having it around made the
refactoring too difficult
* foolscap/ipb.py (IBroker): use this to distinguish between a
regular Banana instance and a Broker. The PB-specific slicers use it
to assert that their .protocol is really a Broker.
* foolscap/referenceable.py (ReferenceableSlicer): if the
Referenceable is being sliced for storage, emit a their-reference
sequence instead of a (useless) my-reference. Use giftID=0 to
indicate that we don't want to do reference counting.
(TheirReferenceUnslicer.ackGift): giftID=0 means don't ack
* foolscap/slicers/root.py: RootSlicer refactoring
(ScopedRootSlicer): new class for refactoring
(ScopedRootUnslicer): same
* foolscap/storage.py: same. Rewrite serialize/unserialize, they
now return Deferreds and let you override the banana and
root (un)slicer class to use, as well as the IO stream.
* foolscap/test/test_banana.py: massive hacking to make it work.
The storage classes changed a lot, requiring this cleanup. Also
all banana/broker objects must have a transport and their
connectionMade() method needs to be called before you can use them.
* foolscap/test/test_pb.py: same
* foolscap/test/test_serialize.py: test all the new functionality,
including how serialized data cannot keep a Referenceable alive,
and how you you can only serialize Referenceables with
Tub.serialize(), not with foolscap.serialize().
2007-09-11 Brian Warner <warner@lothar.com>
* foolscap/call.py (PendingRequest.fail): improve
logRemoteFailures by adding a source+dest TubID to the log
message. Closes #23.
(InboundDelivery.logFailure): same for logLocalFailures
(CallUnslicer.receiveClose): tell the InboundDelivery about its
broker
* foolscap/pb.py (Tub.getTubID, .getShortTubID): utility methods
(UnauthenticatedTub.getTubID, .getShortTubID): same
* foolscap/referenceable.py (TubRef.getShortTubID): same
(NoAuthTubRef.getShortTubID): same
(RemoteReference._callRemote): tell the PendingRequest about the
interface name it is using, so it can log a fully-qualified
remote method name
* foolscap/broker.py (Broker): tell the Broker about the remote
TubRef it is attached to
(Broker.doNextCall): make sure to log.err any problems that occur
during callFailed, rather than discarding them. This catches
errors in InboundDelivery.logFailure
* foolscap/negotiate.py (Negotiation.switchToBanana): pass the
remote TubRef to the newly-created Broker
* foolscap/test/test_pb.py (TestCallable.testLogLocalFailure): test it
(TestCallable.testLogLocalFailure): reduce timeout
(TestCallable.testLogRemoteFailure._check): test it
(TestCallable.testLogRemoteFailure): reduce timeout
* foolscap/test/test_call.py (TestCall.testFailStringException): same
(TestCall.testCopiedFailure): same
* foolscap/test/test_gifts.py (Gifts.testGift): same
* foolscap/test/common.py (TargetMixin.setupBrokers): provide tubref
2007-09-08 Brian Warner <warner@lothar.com>
* foolscap/pb.py (Tub.registerReference): add furlFile= argument,
to make it easy to persist unguessable FURLs in a file on disk.
* foolscap/test/test_tub.py (FurlFile): test it
* foolscap/tokens.py (WrongTubIdError): new exception for it
* doc/using-foolscap.xhtml (Using a Persistent FURL): document it.
Also replace all use of 'url' with 'furl'.
* foolscap/__init__.py: bump revision to 0.1.6+ while between releases
* misc/*/debian/changelog: same
2007-09-02 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release 0.1.6
* misc/{dapper|edgy|feisty|sarge|sid}/debian/changelog: same
* misc/etch: copy sid packaging for etch
* Makefile (debian-etch): new target for etch .debs
* setup.py: update download_url for 0.1.6
* MANIFEST.in: include etch files in the source release
* NEWS: update for the upcoming release
* Makefile (api-docs): new target to run 'epydoc' and generate API
documentation. Addresses #16.
(docs): fix the URLs of API doc targets to match
* foolscap/negotiate.py: fix some docstrings
* foolscap/pb.py: same
* foolscap/promise.py: same
2007-08-31 Brian Warner <warner@lothar.com>
* foolscap/schema.py (PolyConstraint.checkToken): we need to
override this method as well as checkObject, otherwise inbound
tokens may get rejected by the token-checking phase. robk noticed
this with a ChoiceOf(StringConstraint(maxLength=3000, None)),
which refused to accept a 2000 byte string (since the default
Constraint.checkToken uses the everythingTaster which only accepts
1000-byte strings). Closes #13.
* foolscap/test/test_call.py (TestCall.testChoiceOf): test it
2007-08-21 Brian Warner <warner@lothar.com>
* foolscap/call.py (FailureSlicer.getStateToCopy): if we can't fit
the whole traceback, elide the middle rather than truncate the
end, since the end is usually the most interesting part.
* foolscap/pb.py (Tub.setupEncryptionFile): don't use
os.path.exists to decide if the certFile exists or not, just try
to open it and deal with the exception if it happens. This avoids
a race condition, not a big deal here but a good pattern to get in
the habit of using everywhere.
* foolscap/slicers/unicode.py (UnicodeConstraint.checkObject):
improve error message
2007-08-17 Brian Warner <warner@lothar.com>
* doc/using-foolscap.xhtml: fix more typos, as reported by
arch_o_median. Closes #15.
* .hgignore: ignore generated .html files in doc/
2007-08-11 Brian Warner <warner@lothar.com>
* doc/using-foolscap.xhtml: replace the phrase "private-key
certificate" with the more accurate and more widely-used
"public-key certificate". Thanks to Zooko for the patch.
2007-08-09 Brian Warner <warner@lothar.com>
* doc/using-foolscap.xhtml: fix typos, thanks to David Ripton for
the catch.
* setup.py: add classifiers= and platforms=, update metadata, so
that I can use 'setup.py register' for the next release. Closes #7.
* README: update references to the home page. Closes #9.
2007-08-08 Brian Warner <warner@lothar.com>
* misc/{sid|sarge|dapper|edgy|feisty}/debian/rules: fix references
to renamed docs/ files. Closes #8.
* foolscap/__init__.py: bump revision to 0.1.5+ while between releases
* misc/{sid|sarge|dapper|edgy|feisty}/debian/changelog: same
2007-08-07 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.1.5
* misc/{sid|sarge|dapper|edgy|feisty}/debian/changelog: same
2007-08-07 Brian Warner <warner@lothar.com>
* NEWS: update for the upcoming release
* foolscap/pb.py (Tub.registerNameLookupHandler): new function to
augment Tub.registerReference(). This allows names to be looked up
at request time, rather than requiring all Referenceables be
pre-registered with registerReference(). The chief use of this
would be for FURLs which point at objects that live on disk in
some persistent state until they are needed. Closes #6.
(Tub.unregisterNameLookupHandler): allow handlers to be removed
(Tub.getReferenceForName): use the handler during lookup
* foolscap/test/test_tub.py (NameLookup): test it
2007-07-27 Brian Warner <warner@lothar.com>
* foolscap/referenceable.py (LocalReferenceable): implement an
adapter that allows code to do IRemoteReference(t).callRemote(...)
and have it work for both RemoteReferences and local
Referenceables. You might want to do this if you're getting back
introductions to a variety of remote Referenceables, some of which
might actually be on your local system, and you want to treat all
of the, the same way. Local Referenceables will be wrapped with a
class that implements callRemote() and makes it behave like an
actual remote callRemote() would. Closes ticket #1.
* foolscap/test/test_reference.py (LocalReference): test it
2007-07-26 Brian Warner <warner@lothar.com>
* foolscap/call.py (AnswerUnslicer.receiveChild): accept a
ready_deferred, to accomodate Gifts in return values. Closes #5.
(AnswerUnslicer.receiveClose): .. and don't fire the response
until any such Gifts resolve
* foolscap/test/test_gifts.py (Gifts.testReturn): test it
(Gifts.testReturnInContainer): same
(Bad.testReturn_swissnum): and test the failure case too
* foolscap/test/test_pb.py (TestAnswer.testAccept1): fix a test
which wasn't calling start() properly and was broken by that change
(TestAnswer.testAccept2): same
* foolscap/test/test_gifts.py (Bad.setUp): disable these tests when
we don't have crypto, since TubIDs are not mangleable in the same
way without crypto.
* foolscap/slicer.py (BaseUnslicer.receiveChild): new convention:
Unslicers should accumulate their children's ready_deferreds into
an AsyncAND, and pass it to the parent. If something goes wrong,
the ready_deferred should errback, which will abandon the method
call that contains it.
* foolscap/slicers/dict.py (DictUnslicer.receiveClose): same
* foolscap/slicers/tuple.py (TupleUnslicer.receiveClose): same
(TupleUnslicer.complete): same
* foolscap/slicers/set.py (SetUnslicer.receiveClose): same
* foolscap/slicers/list.py (ListUnslicer.receiveClose): same
* foolscap/call.py (CallUnslicer.receiveClose): same
* foolscap/referenceable.py (TheirReferenceUnslicer.receiveClose):
use our ready_deferred to signal whether the gift resolves
correctly or not. If it fails, errback ready_deferred (to prevent
the message from being delivered without the resolved gift), but
callback obj_deferred with a placeholder to avoid causing too much
distress to the container.
* foolscap/broker.py (PBRootUnslicer.receiveChild): accept
ready_deferred in the InboundDelivery, stash both of them in the
broker.
(Broker.scheduleCall): rewrite inbound delivery handling: use a
self._call_is_running flag to prevent concurrent deliveries, and
wait for the ready_deferred before delivering the top-most
message. If the ready_deferred errbacks, that gets routed to
self.callFailed so the caller hears about the problem. This closes
ticket #2.
* foolscap/call.py (InboundDelivery): remove whenRunnable, relying
upon the ready_deferred to let the Broker know when the message
can be delivered.
(ArgumentUnslicer): significant cleanup, using ready_deferred.
Remove isReady and whenReady.
* foolscap/test/test_gifts.py (Base): factor setup code out
(Base.createCharacters): registerReference(tubname), for debugging
(Bad): add a bunch of tests to make sure that gifts which fail to
resolve (for various reasons) will inform the caller about the
problem, via an errback on the original callRemote()'s Deferred.
2007-07-25 Brian Warner <warner@lothar.com>
* foolscap/util.py (AsyncAND): new utility class, which is like
DeferredList but is specifically for control flow rather than data
flow.
* foolscap/test/test_util.py: test it
* foolscap/call.py (CopiedFailure.setCopyableState): set .type to
a class that behaves (as least as far as reflect.qual() is
concerned) just like the original exception class. This improves
the behavior of derived Failure objects, as well as trial's
handling of CopiedFailures that get handed to log.err().
CopiedFailures are now a bit more like actual Failures. See ticket
#4 (http://foolscap.lothar.com/trac/ticket/4) for more details.
(CopiedFailureSlicer): make sure that CopiedFailures can be
serialized, so that A-calls-B-calls-C can return a failure all
the way back.
* foolscap/test/test_call.py (TestCall.testCopiedFailure): test it
* foolscap/test/test_copyable.py: update to match, now we must
compare reflect.qual(f.type) against some extension classname,
rather than just f.type.
* foolscap/test/test_pb.py: same
* foolscap/test/common.py: same
2007-07-15 Brian Warner <warner@lothar.com>
* foolscap/test/test_interfaces.py (TestInterface.testStack):
don't look for a '/' in the stacktrace, since it won't be there
under windows. Thanks to 'strank'. Closes Twisted#2731.
2007-06-29 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: bump revision to 0.1.4+ while between releases
* misc/{sid|sarge|dapper|edgy|feisty}/debian/changelog: same
2007-05-14 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.1.4
* misc/{sid|sarge|dapper|edgy|feisty}/debian/changelog: same, also
remove a bunch of old between-release version numbers
2007-05-14 Brian Warner <warner@lothar.com>
* NEWS: update for the upcoming release
* doc/using-foolscap.xhtml: rename from doc/using-pb.xhtml
* doc/using-pb.xhtml: replace all uses of 'PB URL' with 'FURL'
* foolscap/pb.py (Tub.getReference): if getReference() is called
before Tub.startService(), queue the request until startup.
(Tub.connectTo): same for connectTo().
(Tub.startService): launch pending getReference() and connectTo()
requests. There are all fired with eventual-sends.
* foolscap/reconnector.py (Reconnector): don't automatically start
the Reconnector in __init__, rather wait for the Tub to start it.
* foolscap/test/test_tub.py (QueuedStartup): test it
* doc/using-pb.xhtml: update docs to match
* foolscap/test/test_call.py (TestCall.testCall1): replace an
arbitrary delay with a polling loop, to make the test more
reliable under load
* foolscap/referenceable.py (SturdyRef.asLiveRef): remove a method
that was never used, didn't work, and is of dubious utility
anyways.
(_AsLiveRef): remove this too
* misc/testutils/figleaf.py (CodeTracer.start): remove leftover
debug logging
* foolscap/remoteinterface.py (RemoteInterfaceConstraint): accept
gifts too: allow sending of RemoteReferences on the outbound side,
and accept their-reference sequences on the inbound side.
* foolscap/test/test_gifts.py (Gifts.test_constraint): test it
* foolscap/test/test_schema.py (Interfaces.test_remotereference):
update test, since now we allow RemoteReferences to be sent on the
outbound side
* foolscap/remoteinterface.py (getRemoteInterface): improve the
error message reported when a Referenceable class implements
multiple RemoteInterfaces
* foolscap/remoteinterface.py (RemoteMethodSchema.initFromMethod):
properly handle methods like 'def foo(nodefault)' that are missing
*all* default values. Previously this resulted in an unhelpful
exception (since typeList==None), now it gives a sensible
InvalidRemoteInterface exception.
* foolscap/test/test_schema.py (Arguments.test_bad_arguments):
test it
2007-05-11 Brian Warner <warner@lothar.com>
* foolscap/slicers/set.py (FrozenSetSlicer): finally acknowledge
our dependence on python2.4 or newer, by using the built-in 'set'
and 'frozenset' types by default. We'll serialize the old sets.Set
and sets.ImmutableSet too, but they'll emerge as a set/frozenset.
This will cause problems for code that was written to be
compatible with python2.3 (by using sets.Set) and wasn't changed
when moved to 2.4, if it tries to mingle sets.Set with the data
coming out of Foolscap. Unfortunate, but doing it this way
preserves both sanity and good behavior for modern 2.4-or-later
apps.
(SetUnslicer): fix handling of children that were unreferenceable
during construction, fix handling of children that are not ready
for use (i.e. gifts).
(FrozenSetUnslicer): base this off of TupleUnslicer, since
previously the cycle-handling logic was completely broken. I'm not
entirely sure this is necessary, since I think the contents of
sets must be transitively immutable (or at least transitively
hashable), but it good to review and clean it up anyways.
* foolscap/slicers/allslicers.py: match name change
* foolscap/slicers/tuple.py (TupleUnslicer.receiveClose): fix
handling of unready children (i.e. gifts), previously gifts inside
containers were completely broken.
* foolscap/slicers/list.py (ListUnslicer.receiveClose): same
* foolscap/slicers/dict.py (DictUnslicer.receiveClose): same
* foolscap/call.py: add debug log messages (disabled)
* foolscap/referenceable.py (TheirReferenceUnslicer.receiveClose):
gifts must declare themselves 'unready' until the RemoteReference
resolves, since we might be inside a container of some sort.
Without this fix, methods would be invoked too early, before the
RemoteReference was really available.
* foolscap/test/test_banana.py (ThereAndBackAgain.test_set): match
new set/sets.Set behavior
(ThereAndBackAgain.test_cycles_1): test some of the cycles
(ThereAndBackAgain.test_cycles_3): add (disabled) test for
checking cycles that involve sets. I think these tests are
non-sensical, since sets can't really participate in the sorts of
cycles we worry about, but I left the (disabled) test code in
place in case it becomes useful again.
* foolscap/test/test_gifts.py (Gifts.testContainers): validate
that gifts can appear in all sorts of containers successfully.
2007-05-11 Brian Warner <warner@lothar.com.com>
* foolscap/__init__.py: bump revision to 0.1.3+ while between releases
* misc/{sid|sarge|dapper|edgy|feisty}/debian/changelog: same
2007-05-02 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.1.3
* misc/{sid|sarge|dapper|edgy|feisty}/debian/changelog: same
2007-05-02 Brian Warner <warner@lothar.com>
* MANIFEST.in: include some recently-added files to the source
tarball
* NEWS: update for the upcoming release
* foolscap/reconnector.py (Reconnector._failed): simplify
log/no-log logic
* foolscap/slicers/unicode.py (UnicodeConstraint): add a new
constraint that only accepts unicode objects. It isn't complete:
I've forgotten how the innards of Constraints work, and as a
result this one is too permissive: it will probably accept too
many tokens over the wire before raising a Violation (although the
post-receive just-before-the-method-is-called check should still
be enforced, so application code shouldn't notice the issue).
* foolscap/test/test_schema.py (ConformTest.testUnicode): test it
(CreateTest.testMakeConstraint): check the typemap too
* foolscap/test/test_call.py (TestCall.testMegaSchema): test in a call
* foolscap/test/common.py: same
* foolscap/constraint.py (ByteStringConstraint): rename
StringConstraint to ByteStringConstraint, to more accurately
describe its function. This constraint will *not* accept unicode
objects.
* foolscap/call.py, foolscap/copyable.py, foolscap/referenceable.py:
* foolscap/slicers/vocab.py: same
* foolscap/schema.py (AnyStringConstraint): add a new constraint
to accept either bytestrings or unicode objects. I don't think it
actually works yet, particularly when used inside containers.
(constraintMap): map 'str' to ByteStringConstraint for now. Maybe
someday it should be mapped to AnyStringConstraint, but not today.
Map 'unicode' to UnicodeConstraint.
* foolscap/pb.py (Tub.getReference): assert that the Tub is
already running, either because someone called Tub.startService(),
or because we've been attached (with tub.setServiceParent) to a
running service. This requirement appeared with the
connector-tracking code, and I hope to relax it at some
point (such that any pre-startService getReferences will be queued
and serviced when the Tub is finally started), but for this
release it is a requirement to start the service before trying to
use it.
(Tub.connectTo): same
* doc/using-pb.xhtml: document it
* doc/listings/pb1client.py: update example to match
* doc/listings/pb2client.py: update example to match
* doc/listings/pb3client.py: update example to match
* foolscap/pb.py (Tub.connectorFinished): if, for some reason,
we're removing the same connector twice, log and ignore rather
than explode. I can't find a code path that would allow this, but
I *have* seen it occur in practice, and the results aren't pretty.
Since the whole connection-tracking thing is really for the
benefit of unit tests anyways (who want to know when
Tub.stopService is done), I think it's more important to keep
application code running.
* foolscap/negotiate.py (TubConnector.shutdown): clear out
self.remainingLocations too, in case it helps to shut things down
faster. Add some comments.
* foolscap/negotiate.py (Negotiation): improve error-message
delivery, by keeping track of what state the receiver is in (i.e.
whether we should send them an HTTP error block, an rfc822-style
error-block, or a banana ERROR token).
(Negotiation.switchToBanana): empty self.buffer, to make sure that
any extra data is passed entirely to the new Banana protocol and
none of it gets passed back to ourselves
(Negotiation.dataReceived): same, only recurse if there's something
still in self.buffer. In other situtations we recurse here because
we might have somehow received data for two separate phases in a
single packet.
* foolscap/banana.py (Banana.sendError): rather than explode when
trying to send an overly-long error message, just truncate it.
2007-04-30 Brian Warner <warner@lothar.com>
* foolscap/broker.py (Broker.notifyOnDisconnect): if the
RemoteReference is already dead, notify the callback right away.
Previously we would never notify them, which was a problem.
(Broker.dontNotifyOnDisconnect): be tolerant of attempts to
unregister callbacks that have already fired. I think this makes it
easier to write correct code, but on the other hand it loses the
assertion feedback if somebody tries to unregister something that
was never registered in the first place.
* foolscap/test/test_call.py (TestCall.testNotifyOnDisconnect):
test this new tolerance
(TestCall.testNotifyOnDisconnect_unregister): same
(TestCall.testNotifyOnDisconnect_already): test that a handler
fires when the reference was already broken
* foolscap/call.py (InboundDelivery.logFailure): don't use
f.getTraceback() on string exceptions: twisted explodes
(FailureSlicer.getStateToCopy): same
* foolscap/test/test_call.py (TestCall.testFailStringException):
skip the test on python2.5, since string exceptions are deprecated
anyways and I don't want the warning message to clutter the test
logs
* doc/using-pb.xhtml (RemoteInterfaces): document the fact that
the default name is *not* fully-qualified, necessitating the use
of __remote_name__ to distinguish between foo.RIBar and baz.RIBar
* foolscap/remoteinterface.py: same
* foolscap/call.py (FailureSlicer.getStateToCopy): handle string
exceptions without exploding, annoying as they are.
* foolscap/test/test_call.py (TestCall.testFail4): test them
2007-04-27 Brian Warner <warner@lothar.com>
* foolscap/broker.py (Broker.freeYourReference._ignore_loss):
change the way we ignore DeadReferenceError and friends, since
f.trap is not suitable for direct use as an errback
* foolscap/referenceable.py (SturdyRef.__init__): log the repr of
the unparseable FURL, rather than just the str, in case there are
weird control characters in it
* foolscap/banana.py (Banana.handleData): rewrite the typebyte
scanning loop, to remove the redundant pos<64 check. Also, if we
get an overlong prefix, log it so we can figure out what's going
wrong.
* foolscap/test/test_banana.py: update to match
* foolscap/negotiate.py (Negotiation.dataReceived): if a
non-NegotiationError exception occurs, log it, since it indicates
a foolscap coding failure rather than some disagreement with the
remote end. Log it with 'log.msg' for now, since some of the unit
tests seem to trigger startTLS errors that flunk tests which
should normally pass. I suspect some problems with error handling
in twisted's TLS implementation, but I'll have to investigate it
later. Eventually this will turn into a log.err.
* foolscap/pb.py (Tub.keepaliveTimeout): set the default keepalive
timer to 4 minutes. This means that at most 8 minutes will go by
without any traffic at all, which should be a reasonable value to
keep NAT table entries alive. PINGs are only sent if no other
traffic was received, and they are only one byte long, so the
traffic overhead should be minimal. Note that we are not turning
on disconnectTimeout by default: if you want quietly broken
connections to be disconnected before TCP notices a problem you'll
need to do tub.setOption("disconnectTimeout", 10*60) or something.
* foolscap/pb.py: remove an unused import
* foolscap/pb.py (Tub.generateSwissnumber): always use os.urandom
to generate the unguessable identifier. Previously we used either
PyCrypto or fell back to the stdlib 'random' module (which of
course isn't very random at all). I did it this way originally to
provide compatibility with python2.3 (which lacked os.urandom):
now that we require python2.4 or newer, os.urandom is a far better
source (it uses /dev/random or equivalent).
* doc/using-pb.xhtml: don't mention PyCrypto now that we aren't
using it at all.
* foolscap/negotiate.py (Negotiation.minVersion): bump both min
and max version to '3', since we've added PING and PONG tokens
that weren't present before. It would be feasible to accomodate v2
peers (by adding a Banana flag that refrains from ever sending
PINGs), but there aren't enough 0.1.2 installations present to
make this seem like a good idea just now.
(Negotiation.maxVersion): same
(Negotiation.evaluateNegotiationVersion3): same
(Negotiation.acceptDecisionVersion3): same
* foolscap/test/test_negotiate.py (Future): same
* foolscap/banana.py (Banana): add keepalives and idle-disconnect.
The first timeout value says that if we haven't received any data
for this long, poke the other side by sending them a PING message.
The other end is obligated to respond with a PONG (both PING and
PONG are otherwise ignored). If we still haven't heard anything
from them by the time the second timeout expires, we drop the
connection.
(Banana.dataReceived): if we're using keepalives, update the
dataLastReceivedAt timestamp on every inbound message.
(Banana.sendPING, sendPONG): new messages and handlers. Both are
ignored, and serve only to update dataLastReceivedAt.
* foolscap/tokens.py: add PING and PONG tokens
* doc/specifications/banana.xhtml: document PING and PONG
* foolscap/broker.py (Broker.__init__): add keepaliveTimeout and
disconnectTimeout arguments. Both default to 'None' to disable
keepalives and disconnects.
* foolscap/negotiate.py (Negotiation.switchToBanana): copy
timeouts from the Tub into the new Banana/Broker instance
* foolscap/pb.py (Tub.setOption): accept 'keepaliveTimeout' and
'disconnectTimeout' options to enable this stuff.
* foolscap/test/test_keepalive.py: test it
* foolscap/pb.py (Tub.brokerClass): parameterize the kind of
Broker that this Tub will create, to make certain unit tests
easier to write (allowing them to substitute a custom Broker
subclass).
* foolscap/negotiate.py (Negotiation.brokerClass): same
(Negotiation.initClient): capture the brokerClass here for clients
(Negotiation.handlePLAINTEXTServer): and here for listeners
(Negotiation.switchToBanana): use it
2007-04-26 Brian Warner <warner@lothar.com>
* README (DEPENDENCIES, INSTALLATION): add docs
2007-04-16 Brian Warner <warner@lothar.com>
* foolscap/remoteinterface.py
(RemoteInterfaceConstraint.checkObject): string-format the object
inside a tuple, to avoid an annoying logging failure when the
object in question is actually a tuple
* foolscap/test/test_gifts.py (ignoreConnectionDone): trap both
ConnectionDone and ConnectionLost, since it appears that windows
signals ConnectionLost. Hopefully this will make the unit tests
pass under windows.
* foolscap/banana.py (Banana.handleData): when the token prefix is
too long, log and emit the repr of the prefix string, so somebody
can figure out where it came from.
* foolscap/test/test_banana.py (InboundByteStream.testString):
update to match
2007-04-13 Brian Warner <warner@lothar.com>
* foolscap/copyable.py (CopyableSlicer.slice): set self.streamable
before yielding any tokens, otherwise contained elements that use
streaming will trigger an exception. Many thanks to
Ricky (iacovou-AT-gmail.com) for trying out advanced features of
Foolscap and discovering this problem, I would never have stumbled
over this one on my own. TODO: we still need unit tests to
exercise this sort of thing on a regular basis.
(Copyable2): same thing
* foolscap/schema.py (_tupleConstraintMaker): redefine what tuples
mean in constraint specifications. They used to indicate an
alternative: (int,str) meant accept either an int *or* a string.
Now tuples indicate actual tuples, so (int,str) means a 2-element
tuple in which the first element is an int, and the second is a
string. I don't know what I was thinking back then. If you really
want to use alternatives, use schema.ChoiceOf instead.
* foolscap/test/test_schema.py (CreateTest.testMakeConstraint):
test that tuples mean tuples
* foolscap/reconnector.py (Reconnector._failed): the old f.trap()
could sometimes cause the reconnector to stop trying forever.
Remove that. Thanks to Rob Kinninmont for finding the problem. Add
new code to log the failure if f.check() indicates that it is a
NegotiationError, since that's the sort of weird thing that users
will probably want to see.
* foolscap/test/test_reconnector.py: add lots of new tests
* misc/testutils: add tools to do figleaf-based code-coverage
checks while running unit tests. We have 89.2% coverage! Use
'make test-figleaf figleaf-output' to see the results.
* Makefile: new targets for figleaf
(test): enable 'make test TEST=foolscap.test.test_call' to work
(test-figleaf): same
* foolscap/__init__.py: bump revision to 0.1.2+ while between releases
* misc/{sid|sarge|dapper|edgy|feisty}/debian/changelog: same
2007-04-04 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.1.2
* misc/{sid|sarge|dapper|edgy|feisty}/debian/changelog: same
* NEWS: update for new release
2007-04-04 Brian Warner <warner@lothar.com>
* misc/feisty/debian/*: add debian packaging support for the
Ubuntu 'feisty' distribution
* Makefile: and a way to invoke it
* misc/edgy/debian/*: same for the 'edgy' distribution
* MANIFEST.in: include the edgy/feisty files in the source tarball
* foolscap/test/test_call.py (TestCall.testMegaSchema): add a new
test to exercise lots of constraint code
* foolscap/test/common.py: support code for it
* foolscap/slicers/set.py (SetUnslicer.setConstraint): fix bugs
discovered as a result
(SetConstraint.__init__): same
* foolscap/__init__.py: bump revision to 0.1.1+ while between releases
* misc/{sid|sarge|dapper}/debian/changelog: same
2007-04-03 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.1.1
* misc/{sid|sarge|dapper}/debian/changelog: same
* NEWS: update for new release
2007-04-03 Brian Warner <warner@lothar.com>
* foolscap/negotiate.py (Negotiation): bump both minVersion and
maxVersion to 2, indicating that this release is not compatible
with 0.1.0, since the reqID=0 change will cause the first method
call in either direction (probably a getYourReferenceByName) to
never receive a response. The handler functions were rearranged a
bit too.
* foolscap/test/test_negotiate.py (Future): update to match
* NEWS: get ready for release
* foolscap/test/test_pb.py (TestCallable.testLogLocalFailure):
validate that Tub.setOption("logLocalFailures") actually works
(TestCallable.testLogRemoteFailure): same
* foolscap/remoteinterface.py (UnconstrainedMethod): add a
"constraint" that can be used to mark a method as accepting
anything and returning anything. This might be useful if you have
RemoteInterface for most of your application, but there are still
one or two methods which should not enforce a schema of any sort.
This mostly defeats the purpose of schemas in the first place, but
offering UnconstrainedMethod means developers can make the
schema-or-not decision differently for individual methods, rather
than for a whole class at a time.
* foolscap/constraint.py (IRemoteMethodConstraint): document the
requirements on IRemoteMethodConstraint-providing classes, now that
there are two of them.
* foolscap/test/test_call.py (TestCall.testUnconstrainedMethod):
test it
* foolscap/test/common.py: add some support code for the test
* foolscap/referenceable.py
(RemoteReferenceTracker._handleRefLost): refrain from sending
decref messages with count=0
* foolscap/negotiate.py (TubConnectorClientFactory.__repr__):
include both the origin and the target of the factory
* foolscap/test/*.py (tearDown): insure that all tests use the
now-standard stopService+flushEventualQueue teardown procedure, to
avoid trial complaints about leftover timers and selectables.
* foolscap/test/test_pb.py (GoodEnoughTub): when crypto is not
available, skip some tests that really require it. Modify others
to not really require it.
* foolscap/test/test_crypto.py: same
* foolscap/test/test_gifts.py: same
* foolscap/test/test_loopback.py: same
* foolscap/test/test_negotiate.py: same
* foolscap/test/test_*.py (localhost): replace all use of
"localhost" with "127.0.0.1" to avoid invoking the address
resolver, which sometimes leaves a cleanup timer running. I think
the root problem is there's no clean way to interrupt a connection
attempt which still in the address resolution phase. You can stop
it, but there's no way to wait for the resolver's cleanup timer to
finish, which is what we'd need to make Trial happy.
tcp.BaseClient.resolveAddress does not keep a handle to the
resolver, so failIfNotConnected cannot halt its timer.
* foolscap/test/test_zz_resolve.py: removed this test
* foolscap/crypto.py (_ssl): import SSL goo in a different way to
appease pyflakes
* all: fix some pyflakes warnings by checking for the
importability of foolscap.crypto in a different way
* foolscap/pb.py (Tub.stopService): when shutting down the Tub,
make sure all outstanding connections are shut down as well. By
the time stopService's deferred fires, all of our TCP transports
should have had their 'connectionLost' methods fired. This is
specifically to help unit tests that use Trial, which insists upon
having a clean reactor between tests. With this change, test
suites should use a tearDown() method that looks like: 'd =
tub.stopService(); d.addCallback(flushEventualQueue); return d',
and trial shouldn't complain about foolscap selectables or timers
being left over.
(Tub.stopService): also, since Tubs are not currently restartable,
modify some entry points at shutdown to make sure nobody gets
confused about why their getReference() doesn't work anymore. Be
aware that at some point soon, we'll start enforcing the rule that
the Tub must be started before you can get any connections out of
it, at which point getReference() will queue requests until
startService() is called. The idea is that the Tub will not use
the network at all unless it is running.
* foolscap/broker.py: drop the connection when shutdown() is called
* foolscap/negotiate.py (Negotiate): rearrange error reporting and
connection shutdown. Now errors are stashed and loseConnection()
is called, but the errors are not reported to negotiationFailed()
until connectionLost() is fired (which will be after any remaining
data gets sent out over the wire).
(TubConnector): the TubConnector reports success once the first
connection has passed negotiation, but now lives until all of the
connections are finally closed. It then informs the Tub that it is
done, so the Tub can forget about it (and possibly notify
stopService that it can finally complete).
* foolscap/observer.py (OneShotObserverList): eventual-send -using
event distributor, following the pattern espoused by Mark Miller's
"Concurrency Among Strangers" paper. Many thanks to AllMyData.com
for contributing this class.
* foolscap/test/test_observer.py: tests for it
2007-03-22 Brian Warner <warner@lothar.com>
* foolscap/constraint.py (StringConstraint): add a regexp= argument
* foolscap/test/test_schema.py (ConformTest.testString): test it
* foolscap/test/test_banana.py (TestBananaMixin.shouldDropConnection):
fix a pyflakes warning
* foolscap/call.py: same, don't fall back to plain StringIO if
cStringIO is unavailable
* foolscap/debug.py: same
* foolscap/storage.py: same
* foolscap/slicers/list.py (ListConstraint): add a minLength=
argument, fix maxLength=None
* foolscap/test/test_schema.py (ConformTest.testList): test it
* foolscap/constraint.py (StringConstraint): add a minLength=
argument
* foolscap/test/test_schema.py (ConformTest.testString): test it
* foolscap/slicers/set.py (BuiltinFrozenSetSlicer): add slicer for
the builtin 'frozenset' type that appeared in python2.4
(SetConstraint): provide a constraint for sets
* foolscap/schema.py (SetOf): add an alias
* foolscap/test/test_schema.py (ConformTest.testSet): test it
2007-03-20 Brian Warner <warner@lothar.com>
* foolscap/banana.py (Banana.outgoingVocabTableWasReplaced): remove
verbose debug message, not really needed anymore
* foolscap/ipb.py (IRemoteReference.callRemoteOnly): new method to
invoke a remote method without waiting for a response. Useful for
certain messages where we really don't care whether the far end
receives them or not.
* foolscap/referenceable.py (RemoteReference.callRemoteOnly):
implement it
(TheirReferenceUnslicer.ackGift): use it
* foolscap/broker.py (Broker.initBroker): use reqID=0 to mean "we
don't want a response". Note that this is a compatibility barrier:
older endpoints which use reqID=0 for the first message will not
get a response. All subsequent messages will be ok, though.
(Broker._callFinished): don't send a response if reqID=0
(Broker.callFailed): don't send an error if reqID=0
* foolscap/call.py (InboundDelivery.logFailure): fix arg logging
(CallUnslicer.receiveChild): don't create an activeLocalCalls
entry if reqID=0
* foolscap/test/test_call.py (TestCallOnly.testCallOnly): test it
(TestCall._testFailWrongReturnLocal_1): update expectations,
now that reqIDs start at 1 instead of 0
* foolscap/test/common.py (TargetMixin.poll): new support code
* foolscap/referenceable.py: add Referenceable to
schema.constraintMap, so that RemoteInterfaces can use 'return
Referenceable' to indicate that they return a Referenceable of any
sort. This is like using 'return RIFoo' to indicate that the
method returns a Referenceable that implements RIFoo, but without
the specific interface requirement.
* foolscap/remoteinterface.py (RemoteInterfaceConstraint): support
this by skipping the interface check if self.interface=None
* foolscap/test/test_schema.py (CreateTest): test it
* foolscap/test/test_interfaces.py (Types): update test to match,
since the error messages changed
* foolscap/test/common.py: more test support changes
2007-03-19 Brian Warner <warner@lothar.com>
* foolscap/ipb.py (IRemoteReference): new interface ..
* foolscap/referenceable.py (RemoteReferenceOnly): .. implemented here
* foolscap/remoteinterface.py
(RemoteInterfaceConstraint.checkObject): remove a circular import
by using IRemoteReference to detect RemoteReference instances,
rather than using isinstance().
* foolscap/test/test_schema.py (Interfaces): test it
* everything: massive Constraint refactoring. Primitive
constraints (StringConstraint, IntegerConstraint, etc) are now in
foolscap/constraint.py, while opentype-specific constraints like
ListConstraint and BooleanConstraint are in the same module that
defines the associated Slicer. Remote method constraints are in
remoteinterface.py and copyable.py, FailureConstraint is in
call.py . A new foolscap/constraint.py module contains the base
classes but is careful not to import much else. foolscap/schema.py
contains a reference to all constraints, so that user code can get
at them conveniently. Tests were updated to import from the new
places. Some circular imports were resolved. zope.interface
adaptation has been used to assist with the conversion from the
"shorthand" forms of constraint specification into the full form
(i.e. converting x=str into x=StringConstraint()), specifically
IConstraint(shorthand) will return a Constraint instance.
* foolscap/__init__.py: bump revision to 0.1.0+ while between releases
* misc/{sid|sarge|dapper}/debian/changelog: same
2007-03-15 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.1.0
* misc/{sid|sarge|dapper}/debian/changelog: same
* README: update for new release
* NEWS: update for new release
2007-02-16 Brian Warner <warner@lothar.com>
* foolscap/eventual.py (_SimpleCallQueue._turn): retire all
pending eventual-send messages before returning control to the
reactor, rather than doing exactly one event per reactor turn.
This seems likely to help avoid starvation, as we now finish as
much work as possible before accepting IO (which might cause more
work to be added to our queue), and probably makes the interaction
between eventual-send and DelayedCalls a bit more consistent.
Thanks to Rob Kinninmont for the suggestion.
2007-02-08 Brian Warner <warner@lothar.com>
* foolscap/test/test_pb.py: move TestCall out to..
* foolscap/test/test_call.py: .. a new test file
* foolscap/test/test_negotiate.py (BaseMixin.tearDown): add a
100ms stall between shutting down all the Tubs and actually
finishing the test. This seems to be enough to stop the occasional
test failures that probably occur because TCP connections that
we've dropped haven't finished signalling the other end (also in
our process) that they've been closed.
2007-01-30 Brian Warner <warner@lothar.com>
* foolscap/pb.py (Tub): add certFile= argument, to allow the Tub
to manage its own certificates. This argument provides a filename
where the Tub should read or write its certificate. If the file
exists, the Tub will read the certificate data from there. If not,
the Tub will generate a new certificate and write it to the file.
* foolscap/test/test_tub.py: test it
* doc/using-pb.xhtml: document certFile=
* doc/listings/pb2server.py: use certFile= in the example
2007-01-24 Brian Warner <warner@lothar.com>
* foolscap/crypto.py (MyOptions._makeContext.alwaysValidate): add
code to ignore two additional OpenSSL certificate validation
errors: X509_V_ERR_CERT_NOT_YET_VALID (9) and
X509_V_ERR_CERT_HAS_EXPIRED (10). Foolscap uses certificates very
differently than web sites, and it is exceedingly common to start
using a cert mere seconds after creating it. If there is any
significant clock skew between the two systems, then insisting
that the cert's "valid after X" time is actually in the past will
cause a lot of false errors.
2007-01-22 Brian Warner <warner@lothar.com>
* .darcs-boringfile: ignore files that are generated by distutils
when we make a source release (dist/*) and when making a debian
package (build/* and the debian install directory).
* foolscap/__init__.py: bump revision to 0.0.7+ while between releases
* misc/{sid|sarge|dapper}/debian/changelog: same
2007-01-16 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.0.7
* misc/{sid|sarge|dapper}/debian/changelog: same
* NEWS: update for 0.0.7
2007-01-16 Brian Warner <warner@lothar.com>
* foolscap/pb.py (Tub.getBrokerForTubRef): special-case an attempt
to connect to a tub with an ID equal to our own, by attaching a
Broker to a special LoopbackTransport that delivers serialized
data directly to a peer without going through a socket.
* foolscap/broker.py (LoopbackTransport): same
(Broker.setTub): refactor some code out of negotiate.py
* foolscap/negotiate.py (Negotiation.switchToBanana): same
(Negotiation.loopbackDecision): new method to determine params for
a loopback connection
* foolscap/test/test_loopback.py: enable all tests, add a check to
make sure we can connect to ourselves twice
* foolscap/referenceable.py (RemoteReferenceTracker.getRef): the
weakref this holds may have become stale, so check that we both
have self.ref *and* that self.ref() is not None to decide whether
we must re-create the RemoteReference. This fixes a bug in which
two calls to Tub.getReference() for the same URL would result in
the second call getting None.
(RemoteReferenceTracker._handleRefLost): only send a decref
message if we haven't already re-created the RemoteReference
* foolscap/test/test_pb.py (TestService.testConnect3): modify this
test to validate the 'call Tub.getReference() twice' bug is fixed
2007-01-15 Brian Warner <warner@lothar.com>
* foolscap/test/test_loopback.py (ConnectToSelf): Add tests to
validate that we can connect to our own Tub properly. This test
does not yet pass for authenticated Tubs: the negotiation hangs
until the 30 second timeout is reached. To fix this requires
special-casing such connections to use a different kind of Broker,
one that wires transport.write to eventual(rcvr.dataReceived) and
skips negotiation completely.
2007-01-10 Brian Warner <warner@lothar.com>
* doc/using-pb.xhtml: fix some "pb" references to mention
"Foolscap" instead
* doc/schema.xhtml: same
2007-01-09 Brian Warner <warner@lothar.com>
* foolscap/pb.py (Listener.removeTub): disownServiceParent is not
guaranteed to return a Deferred, so don't try to make removeTub do
so either. I saw a failure related to this, but was unable to
analzye it well enough to reproduce it or write a test case.
(Tub.stopListeningOn): tolerate removeTub returning synchronously
(Tub.stopService): same
2007-01-04 Brian Warner <warner@lothar.com>
* foolscap/negotiate.py (Negotiation.dataReceived): when sending
an error message to the far end inside the decision block, make
sure the error text itself has no newlines, since that would break
the format of the block, and probably cause all sorts of
confusion.
* foolscap/ipb.py (IRemotelyCallable.doRemoteCall): remote calls now
accept positional args
2007-01-04 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: bump revision to 0.0.6+ while between
releases
* misc/{sid|sarge|dapper}/debian/changelog: same
2006-12-18 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.0.6
* misc/{sid|sarge|dapper}/debian/changelog: same
2006-12-18 Brian Warner <warner@lothar.com>
* misc/{sid|sarge|dapper}/debian/rules: include copyable.xhtml
* NEWS: update for 0.0.6
* foolscap/negotiate.py (Negotiation): Send less data. When
sending a range (both for the fundamental banana negotiation
version and for the initial vocab table index), send it in a
single line with "0 1" rather than two separate min and max lines.
This brings the hello message down to about 105 bytes and improves
the benefit of using a negotiated initial-vocab-table-range rather
than an early (but post-negotiation) SET-VOCAB banana sequence.
* foolscap/schema.py: add RemoteInterfaceConstraints. This works
by declaring an argument as, e.g., RIFoo, which means that this
argument must be passed a RemoteReference that is connected to a
remote Referenceable which implements RIFoo. This works as a
return value constraint too.
(Constraint.checkObject): add inbound=
argument to this method, so RemoteInterfaceConstraint can work
properly
(InterfaceConstraint): split this into local and remote forms
(LocalInterfaceConstraint): only check real local objects, not
RemoteReferences. This isn't really useful yet, but eventually
schemas will turn into proper local Guards and then it will be.
(RemoteInterfaceConstraint): only check RemoteReferences. The
check performed must be different on inbound and outbound (since
we'll see a RemoteReference when inbound=True, and a Referenceable
when inbound=False).
(makeConstraint): distinguish between Interfaces and
RemoteInterfaces, so we can figure out whether to use
LocalInterfaceConstraint or RemoteInterfaceConstraint
(callable): get rid of this, the functionality has been absorbed
into RemoteMethodSchema.initFromMethod
* foolscap/broker.py (Broker._doCall): use inbound= argument
(Broker._callFinished): same
* foolscap/referenceable.py (RemoteReference._callRemote): same
* foolscap/slicer.py (ReferenceUnslicer.receiveChild): same
* foolscap/test/test_schema.py: same
* foolscap/test/test_interfaces.py: rearrange, add tests for
RemoteInterfaces.
(LocalTypes): there are tests for Interfaces too, but without local
Guards they're disabled for now.
* foolscap/test/common.py: refactoring
* foolscap/schema.py (makeConstraint): map None to Nothing(),
which only accepts None. This is pretty handy for methods which
are always supposed to return None.
* foolscap/schema.py (RemoteMethodSchema.checkResults): don't
annotate any Violations here.. leave that up to the caller
* foolscap/broker.py (Broker._callFinished): update the annotation
* foolscap/test/test_pb.py: update to match
* foolscap/tokens.py (Violation.prependLocation): add new methods
to Violations for easier annotation of where they occurred
(Violation.appendLocation): same
(Violation.__str__): remove the "at" from the location text
* foolscap/test/test_pb.py: update to match
* foolscap/broker.py (Broker._callFinished): if the outbound
return value violates the schema, annotate the Violation to
indicate the object and method that was responsible.
2006-12-15 Brian Warner <warner@lothar.com>
* foolscap/call.py (CopiedFailure): clean up a bit, make it match
the current Failure class better
* doc/using-pb.xhtml: document positional arguments
* foolscap/call.py: pass both positional and keyword arguments to
remote methods. Previously only keyword arguments were accepted.
This is a pretty far-reaching change, and introduces a
compatibility barrier.
(ArgumentSlicer): send both positional args and kwargs in a
separate container
(CallSlicer): move arg-sending out of CallSlicer
(InboundDelivery.isRunnable): make InboundDelivery itself
responsible for determining when it is runnable, instead of
leaving that up to the CallUnslicer. The InboundDelivery is always
referenceable, making the resulting object delivery simpler.
(ArgumentUnslicer): move arg-receiving out of CallUnslicer. All of
the schema-checking takes place here. Simplify the are-we-ready
tests.
(CallUnslicer): most of the code has moved out. The (call)
sequence is now ('call', reqID, objID, methname, (args)).
* foolscap/broker.py (Broker.scheduleCall): simplify, allow posargs
* foolscap/referenceable.py (Referenceable.doRemoteCall): deliver
posargs to the target method as well as kwargs
(RemoteReference): same, stop trying to pre-map posargs into kwargs,
no longer require a RemoteInterface to use posargs
* foolscap/vocab.py (vocab_v1): add 'arguments' to the v1 vocab
list. This is a compatibilty barriers, and changes like this are
only allowed between releases. Once 0.0.6 is out we should leave
the v1 list alone and make any additions to v2 instead.
* foolscap/schema.py (RemoteMethodSchema): allow posargs, deal
correctly with a mixture of posargs and kwargs
* foolscap/test/test_schema.py (Arguments): test the
RemoteMethodSchema class
* foolscap/test/test_pb.py (TestCall.testCall1a): new tests of
posargs and mixed posargs/kwargs
(TestService.tearDown): use flushEventualQueue for cleanup
* foolscap/test/test_interfaces.py (TestInterface): change a few
things to match RemoteMethodSchema's new interfaces
* foolscap/eventual.py (flushEventualQueue): allow this method to
accept a single argument, which it ignores. This enables it to be
used easily as a Deferred callback/errback, such as in a Trial
tearDown method. The recommended usage is: d = clean_stuff();
d.addBoth(flushEventualQueue); return d
2006-12-11 Brian Warner <warner@lothar.com>
* foolscap/vocab.py: add code to negotiate an initial set of words
with which to pre-fill the VOCAB token list. Each side proposes a
range and they use the highest common index (and they exchange a
short hash of the list itself to guard against disagreements).
This serve to compress the protocol traffic by maybe 50% over the
longer run.
* foolscap/negotiate.py: send the 'initial-vocab-table-min' and
'-max' keys in the offer, and 'initial-vocab-table-index' in the
decision (and in the Banana params)
* foolscap/broker.py (Broker.__init__): populate the table
* foolscap/banana.py (Banana.populateVocabTable): new method
* foolscap/test/test_banana.py (Sliceable.testAdapter): todo items
* foolscap/referenceable.py
(RemoteReferenceOnly.notifyOnDisconnect): document this method.
* foolscap/broker.py (Broker.shutdown): cancel all disconnect
watchers upon shutdown
* foolscap/pb.py (Tub.stopService): same
* foolscap/negotiate.py (Negotiation.evaluateHello): if we spot an
<=0.0.5 peer, mention that fact in our error message, to
distinguish this case from some completely non-Foolscapish
protocol trying to talk to us.
2006-12-10 Brian Warner <warner@lothar.com>
* foolscap/negotiate.py (TubConnectorClientFactory.__repr__):
annotate the string form to include which Tub we're connecting to.
This makes the default factory's "BlahFactory Starting" log
messages more interesting to look at.
* foolscap/referenceable.py (TubRef.getTubID): support method
(NoAuthTubRef.getTubID): same
2006-12-01 Brian Warner <warner@lothar.com>
* foolscap/referenceable.py (RemoteReference.callRemote): use
defer.maybeDeferred to rearrange and simplify. Clarify the
comments about the various phases of commitment.
* foolscap/call.py (AnswerUnslicer.checkToken): when re-raising an
exception, use bareword 'raise' rather than explicitly re-raising
the same exception instance with 'raise v'. Both forms get the
right instance, but the latter loses the earlier stack trace.
* foolscap/schema.py (RemoteMethodSchema.checkResults): same
(RemoteMethodSchema.checkAllArgs): same
* foolscap/referenceable.py (RemoteReference.callRemote): same
* foolscap/test/test_interfaces.py (TestInterface.testStack): new
test to verify that the Failure you get when you violate outbound
method arguments actually includes the call to callRemote.
* foolscap/schema.py (StringConstraint.checkObject): make the Violation
message more useful
(InterfaceConstraint.checkObject): same, by printing the repr() of the
object that didn't meet the constraint. I'm not sure if this could be
considered to leak sensitive information or not.
(ClassConstraint.checkObject): same
(RemoteMethodSchema.checkAllArgs): record which argument caused the
problem in the Violation
* foolscap/referenceable.py (RemoteReference.callRemote): add
RemoteInterface and method name to the Violation when a caller
violates their outbound constraint
* foolscap/tokens.py (Violation.setLocation,getLocation): make it
easier to modify an existing location value
* foolscap/test/test_interfaces.py (TestInterface.testFail): verify
that RemoteFailures pass a StringConstraint schema
* foolscap/test/test_copyable.py: remove unused imports, from pyflakes
* foolscap/test/test_pb.py: same
* foolscap/test/test_reconnector.py: same
* foolscap/test/test_registration.py: same
* foolscap/test/test_interfaces.py: split the RemoteInterface
tests out to a separate file
* foolscap/test/test_pb.py: split them from here
* foolscap/test/common.py: factor out some common utility classes
2006-11-30 Brian Warner <warner@lothar.com>
* foolscap/negotiate.py (Negotiation.dataReceived): when sending
an error block, set banana-decision-version to '1' so the
recipient knows that it's safe to interpret the 'error' key.
Thanks to Rob Kinninmont for the catch.
2006-11-27 Brian Warner <warner@lothar.com>
* foolscap/negotiate.py (Negotiation._evaluateNegotiationVersion1):
ignore extra keys in the offer, since a real v2 (and beyond) offer
will have all sorts of extra keys.
* foolscap/test/test_negotiate.py (NegotiationV2): test it by
putting extra keys in the offer
2006-11-26 Brian Warner <warner@lothar.com>
* foolscap/negotiate.py (Negotiation): change negotiation
protocol: now each end sends a minVersion/maxVersion pair, using
banana-negotiation-min-version and banana-negotiation-max-version,
indicating that they can handle all versions between those
numbers, inclusive. The deciding end finds the highest version
number that fits in the ranges of both ends, and includes it in
the banana-decision-version key of the decision block. This is an
incompatible protocol change, but should make it easier (i.e.
possible) to have compatible protocol changes in the future.
Thanks to Zooko for suggesting this approach.
(Negotiation.evaluateNegotiationVersion1): each negotiation
version gets is own methods
(Negotiation.acceptDecisionVersion1): same
(TubConnectorClientFactory.buildProtocol): allow the Tub to make
us use other Negotiation classes, for testing
* foolscap/pb.py (Listener.__init__): same, use the class from the
Tub that first caused the Listener to be created
* foolscap/broker.py (Broker.__init__): record the
banana-decision-version value, so tests can check it
* foolscap/test/test_negotiate.py (Future): test it
2006-11-17 Brian Warner <warner@lothar.com>
* foolscap/pb.py: remove unused and dodgy urlparse stuff
* doc/using-pb.xhtml: move and expand the section on Copyable and
other pass-by-copy things into a new file
* doc/copyable.xhtml: new document. Thanks to Ricky Iacovou for
the registerCopier examples.
* doc/listings/copyable-{receive|send}.py: new examples
* doc/stylesheet.css, doc/stylesheet-unprocessed.css
* doc/template.tpl: docs utilities
* Makefile: add 'make docs' target
* foolscap/__init__.py: export registerCopier and
registerRemoteCopyFactory
* foolscap/copyable.py (Copyable): The new preferred Copyable
usage is to have a class-level attribute named "typeToCopy" which
holds the unique string. This must match the class-level
"copytype" attribute of the corresponding RemoteCopy class.
Copyable subclasses (or ICopyable adapters) may still implement
getTypeToCopy(), but the default just returns self.typeToCopy .
Most significantly, we no longer automatically use the
fully-qualified classname: instead we *require* that the class
definition include "typeToCopy". Feel free to use any stable and
globally-unique string here.
(RemoteCopyClass): Require that RemoteCopy subclasses set their
"copytype" attribute, and use it for auto-registration. These
subclasses can still use "copytype=None" to inhibit
auto-registration. They no longer auto-register with the
fully-qualified classname.
* foolscap/referenceable.py (SturdyRef): match this change
* foolscap/test/test_copyable.py: same
2006-11-16 Brian Warner <warner@lothar.com>
* foolscap/negotiate.py (Negotiation.dataReceived): include the
error message in the '500 Internal Server Error' string.
(Negotiation.handlePLAINTEXTClient): include the full '500
Internal Server Error' string in the reported exception. These two
changes make it easier to spot mismatched TubIDs. Thanks to Rob
Kinninmont for the suggestion.
2006-11-14 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: bump revision to 0.0.5+ while between
releases
* misc/{sid|sarge|dapper}/debian/changelog: same
2006-11-04 Brian Warner <warner@lothar.com>
* NEWS: update for 0.0.5
* foolscap/__init__.py: release Foolscap-0.0.5
* misc/{sid|sarge|dapper}/debian/changelog: same
* MANIFEST.in: add debian packaging files to source tarball
2006-11-01 Brian Warner <warner@lothar.com>
* foolscap/pb.py (Tub.setOption): new API to set options. Added
logRemoteFailures and logLocalFailures, which cause failed
callRemotes to be sent to the twisted log via log.msg . The
defaults are False, which means that failures are only reported
through the caller's Deferred.errback .
Setting logRemoteFailures to True means that the client's log will
contain a record of every callRemote that it sent to someone else
that failed on the far side. This can be implemented on a
per-callRemote basis by just doing d.addErrback(log.err)
everywhere, but often there are reasons (like debugging) for
logging failures that are completely independent of the desired
error-handling path. These log messages have a REMOTE: prefix to
make it very clear that the stack trace being shown is *not*
occurring on the local system, but rather on some remote one.
Setting logLocalFailures to True means that the server's log will
contain a record of every callRemote that someone sent to it which
failed on that server. This cannot be implemented with
addErrbacks, since normally the server does not care about the
methods it is running for other people's benefit. This option is
purely for debugging purposes. These log messages have a LOCAL:
prefix to make it clear that the stack trace is happening locally,
but on behalf of some remote caller.
* foolscap/call.py (PendingRequest.fail): improve the logging,
make it conditional on logRemoteFailures, add the REMOTE: prefix
(InboundDelivery): put more information into the InboundDelivery,
move logLocalFailures logging into it
(CallUnslicer.receiveClose): put the .runnable flag on the
InboundDelivery object instead of on the CallUnslicer
* foolscap/broker.py (Broker): pass the InboundDelivery around
instead of the CallUnslicer that it points to.
(Broker.callFailed): Add logLocalFailures checking here.
* foolscap/reconnector.py: oops, add missing import that would break
any actual reconnection attempts
2006-10-31 Brian Warner <warner@lothar.com>
* misc/sarge/debian/control: add sarge packaging
* misc/dapper/debian/control: update dependencies, add Recommends
on pyopenssl
* misc/sid/debian/control: same
* Makefile: add 'debian-sarge' target
* misc/dapper/debian: move debian packaging up a level
* misc/sid/debian: same
* Makefile: same
* foolscap/__init__.py (__version__): bump to 0.0.4+ while between
releases
* misc/debs/sid/debian/changelog: same
* misc/debs/dapper/debian/changelog: same
2006-10-26 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.0.4
* misc/debs/sid/debian/changelog: same
* misc/debs/dapper/debian/changelog: same
2006-10-26 Brian Warner <warner@lothar.com>
* setup.py: fix project URL
* MANIFEST.in: include misc/debs/* in the source tarball
* NEWS: update for 0.0.4
* foolscap/test/test_reconnector.py: verify that the Reconnector's
callbacks are properly interleaved with any notifyOnDisconnect
callbacks the user might have registered. A Reconnector cb that
uses notifyOnDisconnect should see a strictly-ordered sequence of
connect, disconnect, connect, disconnect.
2006-10-25 Brian Warner <warner@lothar.com>
* foolscap/referenceable.py
(RemoteReferenceOnly.notifyOnDisconnect): accept args/kwargs to
pass to the callback. Return a marker that can be passed to
dontNotifyOnDisconnect() to de-register the callback.
* foolscap/broker.py (Broker.notifyOnDisconnect): same
(Broker.connectionLost): fire notifyOnDisconnect callbacks in a
separate turn, using eventually(), so that problems or
side-effects in one call cannot affect other calls or the
connectionLost process
* foolscap/test/test_pb.py (TestCall.testDisconnect4): test it
* foolscap/pb.py (Tub.registerReference): undo that, make
registerReference *always* create a strongref to the target, but
split some of the work out to an internal function which makes the
weakrefs. Tub.registerReference() is the API that application code
uses to publish an object (make it reachable) *and* have the Tub
keep it alive for you. I'm not sure I can think of a use case for
making it reachable but *not* wanting the Tub to keep it alive. If
you want to make it reachable but still ephemeral, just pass it
over the wire.
(Tub._assignName): new method to make weakrefs and assign names.
(Tub.getOrCreateURLForReference): renamed from getURLForReference.
Changed to assign a name if possible and one didn't already exist.
BEHAVIOR CHANGE: This causes *all* objects passed over the wire,
whether explicitly registered or just implicitly passed along, to
be shareable as gifts (assuming the Tub is reachable and has a
location, of course).
* foolscap/referenceable.py (ReferenceableTracker.getURL): update
* foolscap/test/test_registration.py (Registration.testWeak): use
_assignName instead of registerReference
* foolscap/test/test_gifts.py (Gifts.testOrdering): test it
2006-10-25 Brian Warner <warner@lothar.com>
* foolscap/pb.py (Tub.registerReference): add a strong= argument
which means the Tub should keep the registered object alive. If
strong=False, the tub uses a weakref, so that when the application
and all remote peers forget about the object, the Tub will too.
strong= defaults to True to match the previous behavior, but this
might change in the future, and/or it might become a property to
be set on the Tub.
* foolscap/test/test_registration.py: new tests for it
* foolscap/test/test_pb.py (TestService.testStatic): disable this
test, since static data (like tuples) are not weakreffable. The
registration of static data is an outstanding issue.
* foolscap/pb.py (Tub.connectTo): provide a new method, sets up a
repeating connection to a given url (with randomized exponential
backoff) that will keep firing a callback each time a new
connection is made. This is the foolscap equivalent of
ReconnectingClientFactory, and is the repeating form of
getReference(). Thanks to AllMyData.com for sponsoring this work.
* foolscap/reconnector.py (Reconnector): implement it here
* foolscap/test/test_reconnector.py: test it
* doc/using-pb.xhtml: update to reflect that we now have secure
PBURLs and TubIDs, and that methods are delivered in-order (at
least within a Tub-to-Tub connection) even in the face of gifts.
* misc/debs/dapper/debian/rules (binary-indep): remove obsolete
reference to the old python-twisted-pb2 package
* foolscap/referenceable.py (YourReferenceSlicer.slice): assert
that we actually have a URL to give out, since otherwise the error
will confusingly show up on the far end (as a Violation). This
occurs when we (as Alice) try to introduce Carol to a Bob that was
not explicitly registered in Bob's Tub, such that Bob does not
have a URL to give out.
* foolscap/pb.py (Tub): tubID is no longer a parameter to Tub,
since it is always computed from the certificate
(UnauthenticatedTub): but it *is* a parameter here, since there
is no certificate
* foolscap/broker.py (Broker.getMyReferenceByCLID): relax the
assertion to (int,long), since eventually clids will overrun a
31-bit integer. Thanks to Rob Kinninmont for the catch.
(Broker.remote_decref): same
2006-10-10 Brian Warner <warner@lothar.com>
* misc/debs: add some debian packaging, separate directories for
sid and dapper because sid has pycentral and dapper is still in
the versioned-python-package era
* Makefile: simple Makefile to remind me how to create .debs
2006-10-05 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: bump to 0.0.3+ while between releases
2006-10-05 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.0.3
* NEWS: update for 0.0.3 release
2006-10-05 Brian Warner <warner@lothar.com>
* foolscap/test/test_gifts.py (Gifts): split out the Introduction
tests from test_pb.py
(Gifts.testOrdering): test the ordering of messages around a gift.
Doing [send(1), send(2, carol), send(3)] should result in Bob seeing
[1, (2,carol), 3] in that order. Before the recent ordering fix,
the presence of the gift would delay message delivery, resulting in
something like [1, 3, (2,carol)]
* foolscap/test/test_pb.py: same
* foolscap/call.py (CallUnslicer): fix ordering of message
delivery in the face of Gifts. Each inbound method call gets
unserialized into an InboundDelivery/CallUnslicer pair, which gets
put on a queue. Messages get pulled off the queue in order, but
only when the head of the queue is ready (i.e. all of its
arguments are available, which means any pending Gifts have been
retrieved).
(InboundDelivery): same
(CallUnslicer.describe): stop losing useful information
* foolscap/broker.py (Broker.doNextCall): add inboundDeliveryQueue
to implement all this
* foolscap/test/test_pb.py (TestCall.testFailWrongArgsRemote1):
match the change to CallUnslicer.describe
* foolscap/referenceable.py (TheirReferenceUnslicer.receiveClose):
don't bother returning ready_deferred, since we're returning an
unreferenceable Deferred anyway.
* foolscap/test/test_pb.py (Test3Way): put off the check that
Alice's gift table is empty until we're sure she's received the
'decgift' message. Add a note about a race condition that we have
to work around in a weird way to avoid spurious test failures
until I implement sendOnly (aka callRemoteOnly).
2006-10-04 Brian Warner <warner@lothar.com>
* foolscap/test/test_banana.py (ThereAndBackAgain.testIdentity):
use an actual tuple. Obviously I wasn't thinking when I first
wrote this and tried to use "(x)" to construct a one-item tuple.
2006-10-02 Brian Warner <warner@lothar.com>
* everything: fix most of the pyflakes warnings. Some of the
remaining ones are actual bugs where I need to finish implementing
something.
* foolscap/slicers/*.py: move most Slicers/Unslicers out to separate
files, since slicer.py was way too big
* foolscap/slicers/allslicers.py: new module to pull them all in.
banana.py imports this to make sure all the auto-registration hooks
get triggered.
* everything: rearrange imports to match
* setup.py: add new sub-package
2006-10-01 Brian Warner <warner@lothar.com>
* foolscap/slicer.py: rearrange the internals, putting the
corresponding Slicer and Unslicer for each type next to each other
* foolscap/slicer.py: move all "unsafe" Slicers and Unslicers out to
storage.py where it belongs
* foolscap/storage.py: same
* foolscap/test/test_banana.py: fix some imports to match
* foolscap/test/test_pb.py: same
* foolscap/slicer.py (ReplaceVocabSlicer): clean up VOCAB
handling: add the ('add-vocab') sequence to incrementally add to
the receiving end's incomingVocabulary table, fix the race
condition that would have caused problems for strings that were
serialized after the setOutgoingVocabulary() call was made but
before the ('set-vocab') sequence was actually emitted. Lay the
groundwork for adaptive tokenization and negotiated vocab table
presets. Other classes involved are AddVocabSlicer,
AddVocabUnslicer, and ReplaceVocabUnslicer.
(BananaUnslicerRegistry): handle the add-vocab and set-vocab
sequences with a registry rather than special-casing them.
* foolscap/storage.py (UnsafeRootUnslicer): same, add the
BananaUnslicerRegistry
* foolscap/banana.py (setOutgoingVocabulary): make it safe
to call this function at any time, as it merely schedules
an update. Change the signature to accept a list of strings
that should be tokenized rather than expecting the caller to
choose the index values as well.
(addToOutgoingVocabulary): new function to tokenize a single
string, also safe to call at any time
(outgoingVocabTableWasReplaced):
(allocateEntryInOutgoingVocabTable):
(outgoingVocabTableWasAmended): new functions for use by the
Slicers that are sending the 'set-vocab' and 'add-vocab' sequences
(Banana.maybeVocabizeString): reserve a place for adaptize
tokenizing
* foolscap/test/test_banana.py: match the changes
* foolscap/broker.py: s/topRegistry/topRegistries/, since it is
actually a list of Registries. Same for openRegistry and
openRegistries
* foolscap/slicer.py: same
* foolscap/storage.py: same
* foolscap/test/test_banana.py: same
* foolscap/slicer.py (BuiltinSetSlicer): use a different test to
look for python < 2.4, one which doesn't make pyflakes complain
about using __builtins__
2006-09-30 Brian Warner <warner@lothar.com>
* foolscap/promise.py (Promise): implement a simpler syntax, at
the encouragement of Zooko and others: now p.foo(args) does an
eventual-send. This is a simpler form of send(p).foo(args) . Added
_then and _except methods to do simple callback/errback handling.
You can still do send() and sendOnly() on either immediate values
or Promises: this shortcut only helps with send() on a Promise.
You can still do when() on a Promise, which is more flexible
because it returns a Deferred. The new syntax gives you a more
dataflow-ish style of coding, which might be confusing in some
ways but can also make the overall code much much easier to read.
* foolscap/test/test_promise.py: update tests
* foolscap/test/common.py (HelperTarget.remote_defer): replace
callLater(0) with fireEventually()
* foolscap/test/test_banana.py (ErrorfulSlicer.next): same
(EncodeFailureTest.tearDown): use flushEventualQueue() for cleanup
* foolscap/crypto.py (CertificateError): In Twisted >2.5, this
exception is defined in twisted.internet.error, and it is
sometimes raised by the SSL transport (in getPeerCertificate), and
we need to catch it. In older versions, we define it ourselves
even though it will never be raised, so that the code which
catches it doesn't have to have weird conditionals.
* foolscap/negotiate.py (Negotiation.handleENCRYPTED): catch the
CertificateError exception (which indicates that we have an
encrypted but unauthenticated connection: the other end did not
supply a certificate). In older versions of twisted's SSL code,
this was just indicated by having getPeerCertificate() return
None.
* foolscap/test/test_negotiate.py: re-enable all negotiation tests
* foolscap/pb.py (UnauthenticatedTub): change the API and docs to
refer to "Unauthenticated" tubs rather than "Unencrypted" ones,
since that's really the choice you get to make. We use encrypted
connections whenever possible; what you get to control is whether
we use keys to provide secure identification and introduction.
* foolscap/__init__.py: same, export UnauthenticatedTub instead of
UnencryptedTub
* foolscap/negotiate.py: same
* foolscap/referenceable.py: same
* foolscap/test/test_negotiate.py: same
* doc/listings/pb1server.py: update examples to match
* doc/using-pb.xhtml: same
2006-09-26 Brian Warner <warner@lothar.com>
* foolscap/pb.py (Tub): rename PBService to Tub, make it always
be encrypted
(UnencryptedTub): new class for unencrypted tubs
* all: fix everything else (code, docs, tests) to match
* foolscap/ipb.py (ITub): new interface to mark a Tub
2006-09-24 Brian Warner <warner@lothar.com>
* foolscap/referenceable.py (RemoteReferenceTracker._refLost): now
that we have eventually(), use it to avoid the ugly bug-inducing
indeterminacies that result from weakref callbacks being fired in
the middle of other operations.
* foolscap/promise.py (Promise._resolve): I think I figured out
chained Promises. In the process, I made it illegal to call
_break after the Promise has already been resolved. This also
means that _resolve() can only be called once. We'll figure
out breakable Far references later.
* foolscap/test/test_promise.py (Chained): tests for them
* foolscap/broker.py (Broker.getRemoteInterfaceByName): fix a bunch
of typos caught by pyflakes. Left a couple of ones in there that I
haven't figured out how to fix yet.
* foolscap/slicer.py (InstanceUnslicer.receiveChild): same
* foolscap/schema.py (RemoteMethodSchema.initFromMethod): same
* foolscap/pb.py (Listener.addTub): same
* foolscap/debug.py (TokenBanana.reportReceiveError): same
* foolscap/copyable.py: same
* foolscap/test/common.py: same
* foolscap/test/test_pb.py (TestReferenceable.NOTtestRemoteRef1):
same
* foolscap/eventual.py: move eventual-send handling out to a
separate file. This module now provides eventually(cb),
d=fireEventually(), and d=flushEventualQueue() (for use by
unit tests, not user code).
* foolscap/negotiate.py: update to match
* foolscap/test/common.py: same
* foolscap/test/test_pb.py: same
* foolscap/test/test_eventual.py: new tests for eventually()
* foolscap/promise.py: rework Promise handling, now it behaves
like I want it to (although chained Promises aren't working yet)
* foolscap/test/test_promise.py: rework tests
2006-09-16 Brian Warner <warner@lothar.com>
* foolscap/crypto.py: fall back to using our own sslverify.py if
Twisted doesn't provide one (i.e. Twisted-2.4.x).
* foolscap/sslverify.py: copy from the Divmod tree
2006-09-14 Brian Warner <warner@lothar.com>
* foolscap/banana.py: remove #! line from non-script
* foolscap/remoteinterface.py: same
* foolscap/tokens.py: same
* foolscap/test/test_schema.py: same
* foolscap/__init__.py: bump to 0.0.2+ while between releases
2006-09-14 Brian Warner <warner@lothar.com>
* foolscap/__init__.py: release Foolscap-0.0.2
2006-09-14 Brian Warner <warner@lothar.com>
* doc/using-pb.xhtml: update pb3 example to match current usage, show
an example of using encrypted Tubs
* doc/listings/pb3calculator.py: same
* doc/listings/pb3user.py: same
* foolscap/__init__.py: rearrange the API: now 'import foolscap'
is the preferred entry point, rather than 'from foolscap import pb'.
* foolscap/pb.py: stop importing things just to make them available
to people who import foolscap.pb
* all: same, update docs, examples, tests
* all: rename newpb to 'Foolscap'
* setup.py: fix packages= to get tests too
2006-05-15 Brian Warner <warner@lothar.com>
* test_zz_resolve.py: rename test file, I'd like to sit at the end
of the tests rather than at the beginning. This is to investigate
ticket #1390.
* test_negotiate.py (Crossfire): oops, a cut-and-paste error
resulted in two CrossfireReverse tests and zero Crossfire tests.
Fixed this to enable the possibly-never-run real CrossfireReverse
test case.
(top): disable all negotiation tests unless NEWPB_TEST_NEGOTIATION
is set in the environment, since they are sensitive to system load
and the intermittent buildbot failures are annoying.
2006-05-05 Brian Warner <warner@lothar.com>
* release-twisted: add 'pb' subproject
* twisted/python/dist.py: same
* twisted/pb/__init__.py: set version to 0.0.1
* twisted/pb/topfiles/setup.py: fix subproject name, set version 0.0.1
2006-04-29 Brian Warner <warner@lothar.com>
* topfiles/README, topfiles/NEWS: prepare for 0.0.1 release
* setup.py: fix up description, project name
* test_ZZresolve.py: add some instrumentation to try and debug the
occasional all-connection-related-tests-fail problem, which I
suspect involves the threadpool being broken.
2006-02-28 Brian Warner <warner@lothar.com>
* sslverify.py: update to latest version (r5075) from Vertex SVN,
to fix a problem reported on OS-X with python2.4 . Removed the
test-case-name tag to prevent troubles with buildbot on systems
that don't also have vertex installed. I need to find a better
solution for this in the long run: I don't want newpb to depend
upon Vertex, but I also don't want to duplicate code.
2006-02-27 Brian Warner <warner@lothar.com>
* debug.py (encodeTokens): return a Deferred rather than use
deferredResult
2006-02-02 Brian Warner <warner@lothar.com>
* test_negotiate.py: skip pb-vs-web tests when we don't have
twisted.web, thanks to <marienz> for the patch.
2006-01-26 Brian Warner <warner@lothar.com>
* test/test_banana.py (ErrorfulSlicer.next): don't use callLater()
with non-zero timeout
* test/test_promise.py (TestPromise.test2): same
* test/common.py (HelperTarget.remote_defer): same
2006-01-25 Brian Warner <warner@lothar.com>
* copyable.py: refactor ICopyable and IRemoteCopy to make it
possible to register adapters for third-party classes.
(RemoteCopy): allow RemoteCopy to auto-register with the
fully-qualified classname. This is only useful if you inherit from
both pb.Copyable and pb.RemoteCopy at the same time, otherwise the
sender and receiver will be using different names so they won't
match up.
* broker.py (PBRootUnslicer.open): now that registerRemoteCopy is
done purely in terms of Unslicers, remove all the special-case code
that handled IRemoteCopy
(PBRootSlicer.slicerForObject): since zope.interface won't do
transitive adaptation, manually handle the
ThirdPartyClass -> ICopyable -> ISlicer case
* test/test_copyable.py: clean up, improve comments
(MyRemoteCopy3Unslicer): update to match new RemoteCopyUnslicer
behavior. This needs to be documented and made easier. Also switch
from pb.registerRemoteCopy to registerRemoteCopyUnslicerFactory,
which is a mouthful.
(Registration): split this out, update to match new debug tables
in copyable.py
(Adaptation): test ICopyable adapters
2006-01-23 Brian Warner <warner@lothar.com>
* common.py: remove missing test_gift from the test-case-name tag,
not sure how that got in there
* test/test_copyable.py: split Copyable tests out of test_pb.py
* test/common.py: factor out some more common test utility pieces
* copyable.py: add suitable test-case-name tag
* base32.py: rename Base32.py to base32.py, to match Twisted
naming conventions
* crypto.py: same
* pb.py: same
2006-01-02 Brian Warner <warner@lothar.com>
* negotiate.py (eventually): add glyph's eventual-send operator,
based upon a queue cranked by callLater(0).
(flushEventualQueue): provide a way to flush that queue, so tests
know when to finish.
* test/test_pb.py: switch to negotiate.eventually
* test/__init__.py: add test-case-name tag
2005-12-31 Brian Warner <warner@lothar.com>
* test_gift.py (TestOrderedGifts.testGift): verify that the
presence of a gift (a third-party reference) in the arguments of a
method does not cause that method to be run out-of-order. Marked
TODO because at the moment they *are* run out-of-order.
* common.py (RIHelper.append): new method
* referenceable.py (TheirReferenceUnslicer.ackGift): ignore errors
that involve losing the connection, since if these happen, the
giver will decref the gift reference anyway. This removes some
spurious log.errs and makes the unit tests happier.
2005-12-30 Brian Warner <warner@lothar.com>
* test_negotiate.py (Versus.testVersusHTTPServerEncrypted): stall
for a second after the test completes, to give the HTTP server a
moment to tear down its socket. Otherwise trial flunks the test
because of the lingering socket. I don't care for the arbitrary
1.0-second delay, but twisted.web doesn't give me any convenient
way to wait for it to shut down. (this test was only failing under
the gtk2 reactor, but I think this was an unlucky timing thing).
(Versus.testVersusHTTPServerUnencrypted): same
* negotiate.py (eventually): add an eventual-send operator
(Negotiation.negotiationFailed): fire connector.negotiationFailed
through eventually(), to give us a chance to loseConnection
beforehand. This helps the unit tests clean up better.
* negotiation.py (eventually): change the eventual-send operator
to (ab)use reactor.callFromThread instead of callLater(0). exarkun
warned me, but I didn't listen: callLater(0) does not guarantee
relative ordering of sequentially-scheduled calls, and the windows
reactors in fact execute them in random order. Obviously I'd like
the reactor to provide a clearly-defined method for this purpose.
* test_pb.py (eventually): same
(Loopback.write): same. It was the reordering of these _write
calls that was breaking the unit tests on windows so badly.
(Loopback.loseConnection): same
2005-12-29 Brian Warner <warner@lothar.com>
* test_pb.py (Loopback): fix plan-coordination bug by deferring
all write() and loseConnection() calls until the next reactor
turn, using reactor.callLater(0) as an 'eventual send' operator.
This avoids an infinite-mutual-recursion hang that confuses
certain test failures. Tests which use this Loopback must call
flush() and wait on the returned Deferred before finishing.
(TargetMixin): do proper setup/teardown of Loopback
(TestCall.testDisconnect2): use proper CONNECTION_LOST exception
(TestCall.testDisconnect3): same
(TestReferenceable.testArgs1): rename some tests
(TestReferenceable.testArgs2): test sending shared objects in
multiple arguments of a single method call
(TestReferenceable.testAnswer1): test shared objects in the return
value of a method call
(TestReferenceable.testAnswer2): another test for return values
* call.py (CallUnslicer): inherit from ScopedUnslicer, so
arguments that reference shared objects will accurately reproduce
the object graph
(AnswerUnslicer): same, for answers that have shared objects
(ErrorUnslicer): same, just in case serialized Failures do too
* slicer.py (ImmutableSetSlicer): set trackReferences=False, since
immutable objects are never shared, so don't require reference
tracking
* banana.py (Banana.sendError): do loseConnection() in sendError
rather than inside dataReceived.
2005-12-26 Brian Warner <warner@lothar.com>
* slicer.py (ScopedSlicer.registerReference): track references
with a (obj,refid) pair instead of just refid. This insures that
the object being tracked stays alive until the scope is retired,
preventing some ugly bugs that result from dead object id() values
being reused. These bugs would only happen if the object graph
changes during serialization (which you aren't supposed to do),
but this is a cheap fix that limits the damage that could happen.
In particular, it should fix a test failure on the OS-X buildslave
that results from a unit test that is violating this object-graph
-shouldn't-change prohibition.
* banana.py (StorageBanana): refactor storage-related things,
moving them from banana.py and slicer.py into the new storage.py .
This includes UnsafeRootSlicer, StorageRootSlicer,
UnsafeRootUnslicer, and StorageRootUnslicer. Also provide a simple
serialize()/unserialize() pair in twisted.pb.storage, which will
be the primary interface for simple pickle.dumps()-like
serialization.
2005-12-24 Brian Warner <warner@lothar.com>
* slicer.py: remove #!, add test-case-name
(SetSlicer): define this unconditionally, now that python2.2 is no
longer supported.
(BuiltinSetSlicer): just like SetSlicer, used when there is a builtin
'set' type (python2.4 and higher)
(ImmutableSetSlicer): define this unconditionally
(SetUnslicer): same
(ImmutableSetUnslicer): same
* test_banana.py (TestBananaMixin.looptest): make it easier to
test roundtrip encode/decode pairs that don't *quite* re-create
the original object
(TestBananaMixin.loop): clear the token stream for each test
(ThereAndBackAgain.test_set): verify that python2.4's builtin
'set' type is serialized as a sets.Set
* all: drop python2.2 compatibility, now that Twisted no longer
supports it
2005-12-22 Brian Warner <warner@lothar.com>
* pb.py (Listener.getPortnum): more python2.2 fixes, str in str
(PBService.__init__): same, bool issues
* test/test_banana.py: same, use failUnlessSubstring
* test/test_negotiate.py: same
* test/test_pb.py: same
* negotiate.py: same, str in str stuff
* broker.py: don't import itertools, for python2.2 compatibility
* sslverify.py: same
2005-12-20 Brian Warner <warner@lothar.com>
* test/test_banana.py: remove all remaining uses of
deferredResult/deferredError
* test/test_pb.py: same
2005-12-09 Brian Warner <warner@lothar.com>
* pb.py (PBService.__init__): switch to SHA-1 for TubID digests
* negotiate.py (Negotiation.evaluateHello): same
* crypto.py (digest32): same
2005-12-08 Brian Warner <warner@lothar.com>
* pb.py (PBService): allow all Tubs to share the same RandomPool
2005-10-10 Brian Warner <warner@lothar.com>
* lots: overhaul negotiation, add lots of new tests. Implement
shared Listeners, correct handling of both encrypted and
non-encrypted Tubs, follow multiple locationHints correctly. More
docs, update encrypted-tub examples to match new usage.
2005-09-15 Brian Warner <warner@lothar.com>
* test_pb.py: remove some uses of deferredResult/deferredError
2005-09-14 Brian Warner <warner@lothar.com>
* pb.py (PBService.generateSwissnumber): use PyCrypto RNG if
available, otherwise use the stdlib 'random' module. Create a
160-bit swissnumber by default, this can be changed by the
NAMEBITS class attribute.
(PBService.__init__): use a random 32-bit number as a TubID when
we aren't using crypto and an SSL certificate
* Base32.py: copy module from the waterken.org Web-Calculus
python implementation
* test/test_crypto.py (TestService.getRef): let it register a
random swissnumber instead of a well-known name
* crypto.py: Implement encrypted PB connections, so PB-URLs are
closer to being secure capabilities. This file contains utility
functions.
* sslverify.py: some pyOpenSSL wrappers, copied from Divmod's
Vertex/vertex/sslverify.py
* test/test_crypto.py: test case for encrypted connections
* pb.py (PBServerFactory.buildProtocol): accomodate missing tubID,
this needs to be re-thought when I do the "what if we aren't using
crypto" pass.
(PBServerFactory.clientConnectionMade): get the remote_tubid from
a .theirTubID attribute, not the negotiated connection parameters,
which won't include tub IDs anyway)
(PBClientFactory.buildProtocol): if we're using crypto, tell the
other side we want an encrypted connection
(PBService.__init__): add useCrypto= parameter, currently defaults
to False. This should switch to =True soon.
(PBService.createCertificate): if useCrypto=True, create an SSL
certificate for the Tub.
* ipb.py (DeadReferenceError): actually define it somewhere
* broker.py (Broker.handleNegotiation_v1): cleanup, make the
different negotiation-parameter dictionaries distinct, track the
['my-tub-id'] field of each end more carefully. Start a TLS
session when both ends want it.
(Broker.startTLS): method to actually start the TLS session. This
is called on both sides (client and server), the t.i.ssl
subclasses figure out which is which and inform SSL appropriately.
(Broker.acceptNegotiation): Make a PB-specific form. Start TLS if
the server tells us to. When the second (encrypted) negotiation
block arrives, verify that the TubID we're looking for matches
both what they claim and what their SSL certificate contains.
(Broker.freeYourReference): ignore DeadReferenceErrors too
* banana.py (Banana.__init__): each instance must have its own
copy of self.negotiationOffer, rather than setting it at the class
level
(Banana.negotiationDataReceived): let both handleNegotiation() and
acceptNegotiation() return a 'done' flag, if False then the
negotiation is re-started
(Banana.handleNegotiation): make handleNegotiation_v1 responsible
for setting self.negotiationResults
(Banana.handleNegotiation_v1): same
(Banana.acceptNegotiation): same
2005-09-09 Brian Warner <warner@lothar.com>
* broker.py: big sanity-cleanup of RemoteInterface usage. Only
allow a single RemoteInterface on any given pb.Referenceable.
Tub.getReference() now only takes a string-form method name, so
the rr.callRemote(RIFoo['bar'], *args) form is gone, and the one
RemoteInterface associated with the RemoteReference (is available)
will be checked. Tub.getReference() no longer takes an interface
name: you request an object, and then later find out what it
implements (rather than specifying your expectations ahead of
time). Gifts (i.e. 'their-reference' sequences) no longer have an
interfacename.. that is left up to the actual owner of the
reference, who will provide it in the 'my-reference' sequence.
* call.py, pb.py, referenceable.py, remoteinterface.py: same
* test/test_pb.py: update to match, still needs some cleanup
2005-09-08 Brian Warner <warner@lothar.com>
* setup.py, twisted/pb/topfiles: add "PB" sub-project
* banana.py (Banana.sendFailed): oops, loseConnection() doesn't
take an argument
* copyable.py (RemoteCopyClass): make it possible to disable
auto-registration of RemoteCopy classes
* test/test_pb.py (TestCopyable.testRegistration): test it
* referenceable.py (CallableSlicer): make it possible to publish
callables (bound methods in particular) as secure capabilities.
They are handled very much like pb.Referenceable, but with a
negative CLID number and a slightly different callRemote()
codepath.
* broker.py (Broker.getTrackerForMyCall): same
(Broker.getTrackerForYourReference): same, use a
RemoteMethodReferenceTracker for negative CLID values
(Broker.doCall): callables are distinguished by having a
methodname of 'None', and are dispatched differently
* call.py (CallUnslicer.checkToken): accept INT/NEG for the object
ID (the CLID), but not string (leftover from old scheme)
(CallUnslicer.receiveChild): handle negative CLIDs specially
* test/test_pb.py (TestCallable): tests for it all
(TestService.getRef): refactor
(TestService.testStatic): verify that we can register static data
too, at least stuff that can be hashed. We need to decide whether
it would be useful to publish non-hashable static data too.
2005-09-05 Brian Warner <warner@lothar.com>
* pb.py (PBService): move to using tubIDs as the primary identity
key for a Tub, replacing the baseURL with a .location attribute.
Look up references by name instead of by URL, and start using
SturdyRefs locally instead of URLs whenever possible.
(PBService.getReference): accept either a SturdyRef or a URL
(RemoteTub.__init__): take a list of locationHints instead of a
single location. The try-all-of-them code is not yet written, nor
is the optional redirect-following.
(RemoteTub.getReference): change the inter-Tub protocol to pass a
name over the wire instead of a full URL. The Broker is already
connected to a specific Tub (multiple Tubs sharing the same port
will require separate Brokers), and by this point the location
hints have already served their purpose, so the name is the only
appropriate thing left to send.
* broker.py (RIBroker.getReferenceByName): match that change to
the inter-Tub protocol: pass name over the wire, not URL
(Broker.getYourReferenceByName): same
(Broker.remote_getReferenceByName): same
* referenceable.py (RemoteReferenceOnly): replace getURL with
getSturdyRef, since the SturdyRef can be stringified into a URL if
necessary
(SturdyRef): new class. When these are sent over the wire, they
appear at the far end as an identical SturdyRef; if you want them
to appear as a live reference, send sr.asLiveRef() instead.
* test/test_pb.py (TestService.testRegister): match changes
(Test3Way.setUp): same
(HelperTarget.__init__): add some debugging annotations
* test/test_sturdyref.py: new test
* doc/pb/using-pb.xhtml: update to match new usage, explain PB
URLs and secure identifiers
* doc/pb/listings/pb1server.py: same
* doc/pb/listings/pb1client.py: same
* doc/pb/listings/pb2calculator.py: same
* doc/pb/listings/pb2user.py: same
2005-05-12 Brian Warner <warner@lothar.com>
* doc/pb/using-pb.xhtml: document RemoteInterface, Constraints,
most of Copyable (still need examples), Introductions (third-party
references).
* doc/pb/listings/pb2calculator.py, pb2user.py: demostrate
bidirectional references, using service.Application
2005-05-10 Brian Warner <warner@lothar.com>
* broker.py (Broker.freeYourReference): also ignore ConnectionLost
errors
* doc/pb/listings/pb1client.py, pb1server.py: use reactor.run()
* doc/pb/using-pb.xhtml: add shell output for examples
* doc/pb/using-pb.xhtml: started writing usage docs
* banana.py (Banana.dataReceived): add .connectionAbandoned, don't
accept inbound data if it has been set. I don't trust
.loseConnection to work right away, and sending multiple
negotiation error messages is bad.
(Banana.negotiationDataReceived): split out negotiation stuff to a
separate method. Improve failure-reporting code to make sure we
either report a problem with a negotation block, or with an ERROR
token, not both, and not with multiple ERROR tokens. Catch errors
in the upper-level bananaVersionNegotiated() call. Make sure we
only send a response if we're the server. Report negotiation errors
with NegotiationError, not BananaError.
(Banana.reportReceiveError): rearrange a bit, accept a Failure
object. Don't do transport.loseConnection here, do it in whatever
calls reportReceiveError
* debug.py (TokenBanana.reportReceiveError): match signature change
(TokenStorageBanana.reportReceiveError): same
* test/test_banana.py: match changes
* tokens.py (NegotiationError): new exception
* broker.py (Broker.handleNegotiation_v1): use the negotiation
block to exchange TubIDs.
(Broker.connectionFailed): tell the factory if negotiation failed
(Broker.freeYourReference): ignore lost-connection errors, call
freeYourReferenceTracker even if the connection was lost, since
in that case the reference has gone away anyway.
(Broker.freeYourReferenceTracker): don't explode if the keys were
already deleted, since .connectionLost will clear everything
before the decref-ack mechanism gets a chance to delete them.
* referenceable.py (RemoteReferenceTracker.__repr__): stringify
these with more useful information.
* pb.py (PBServerFactory.buildProtocol): copy .debugBanana flag
into the new Broker (both .debugSend and .debugReceive)
(PBServerFactory.clientConnectionMade): survive a missing TubID
(PBClientFactory.negotiationFailed): notify all onConnect watchers
2005-05-08 Brian Warner <warner@lothar.com>
* test_pb.py (TestService): test the use of PBService without
RemoteInterfaces too
2005-05-04 Brian Warner <warner@lothar.com>
* broker.py (Broker): add tables to track gifts (third-party
references)
(PBOpenRegistry): add their-reference entry
(RIBroker.decgift): new method to release pending gifts
* call.py (PendingRequest): add some debugging hints
(CallUnslicer): accept deferred arguments, don't invoke the method
until all arguments are available
* pb.py (PBService.listenOn): return the Service, for testing
(PBService.generateUnguessableName): at least make them unique,
if not actually unguessable
(top): remove old URL code, all is now PBService
* referenceable.py (RemoteReferenceOnly.__repr__): include the
URL, if available
(RemoteReference.callRemote): set .methodName on the
PendingRequest, to make debugging easier
(YourReferenceSlicer.slice): handle third-party references
(TheirReferenceUnslicer): accept third-party references
* schema.py (Nothing): a constraint which only accepts None
* test/test_pb.py (Test3Way): validate third-party reference gifts
2005-04-28 Brian Warner <warner@lothar.com>
* tokens.py (IReferenceable): move to flavors.py
* flavors.py (IReferenceable): add it, mark Referenceable as
implementing it.
* pb.py (PBServerFactory): make root= optional
(PBService): new class. In the future, all PB uses will go through
this service, rather than using factories and connectTCPs directly.
The service uses urlparse to map PB URLs to target hosts.
* test_pb.py (TestService): start adding tests for PBService
2005-04-26 Brian Warner <warner@lothar.com>
* banana.py: add preliminary newpb connection negotiation
* test_banana.py: start on tests for negotiation, at least verify
that newpb-newpb works, and that newpb-http and http-newpb fail.
2005-04-16 Brian Warner <warner@lothar.com>
* banana.py (Banana.handleData): handle -2**31 properly
* test_banana.py (ThereAndBackAgain.test_bigint): test it properly
* flavors.py: python2.2 compatibility: __future__.generators
* pb.py: same
* schema.py (TupleConstraint.maxSize): don't use sum()
(AttributeDictConstraint.maxSize): same
(makeConstraint): in 2.2, 'bool' is a function, not a type, and
there is no types.BooleanType
* slicer.py: __future__.generators, and the 'sets' module might not
be available
(SetSlicer): only define it if 'sets' is available
(SetUnslicer): same
* test_banana.py: __future__.generators, 'sets' might not exist,
(EncodeFailureTest.failUnlessIn): 2.2 can't do 'str in str', only
'char in str', so use str.find() instead
(InboundByteStream2.testConstrainedBool): skip bool constraints
unless we have a real BooleanType
(ThereAndBackAgain.test_set): skip sets unless they're supported
* test_schema.py (ConformTest.testBool): skip on 2.2
(CreateTest.testMakeConstraint): same
* test_pb.py: __future__.generators, use str.find()
* test_banana.py (DecodeTest.test_ref2): accomodate python2.4,
which doesn't try to be quite as clever as python2.3 when
comparing complex object graphs with ==
(DecodeTest.test_ref5): same. Do the comparison by hand.
(DecodeTest.test_ref6): same, big gnarly validation phase
* test_pb.py (TestReferenceUnslicer.testNoInterfaces): update to
new signature for receiveClose()
(TestReferenceUnslicer.testInterfaces): same
(TestCall.testFail1): deferredError doesn't seem to like
CopiedFailure all that much. Use retrial's return-a-deferred
support instead.
(MyRemoteCopy3Unslicer.receiveClose): same
(TestCall.testFail2): same
(TestCall.testFail3): same
(TestFactory): clean up both server and client sockets, to avoid
the "unclean reactor" warning from trial
(Test3Way.tearDown): clean up client sockets
* tokens.py (receiveClose): fix documentation
* pb.py (CopiedFailure): make CopiedFailure old-style, since you
can't raise new-style instances as exceptions, and CopiedFailure
may have its .trap() method invoked, which does 'raise self'.
(CopiedFailure.__str__): make it clear that this is a
CopiedFailure, not a normal Failure.
(callRemoteURL_TCP): Add a _gotReferenceCallback argument, to
allow test cases to clean up their client connections.
* flavors.py (RemoteCopyOldStyle): add an old-style base class, so
CopiedFailure can be old-style. Make RemoteCopy a new-style
derivative.
* test_banana.py (DecodeTest.test_instance): fix the
manually-constructed class names to reflect their new location in
the tree (test_banana to twisted.pb.test.test_banana)
(EncodeFailureTest.test_instance_unsafe): same
* twisted/pb/*: move newpb from Sandbox/warner into the 'newpb'
branch, distributed out in twisted/pb/ and doc/pb/
* twisted/pb: add __init__.py files to make it a real module
* twisted/pb/test/test_*.py: fix up import statements
2005-03-22 Brian Warner <warner@twistedmatrix.com>
* flavors.py: implement new signature
* pb.py: same
* test_pb.py: same
* test_banana.py (BrokenDictUnslicer.receiveClose): new signature
(ErrorfulUnslicer.receiveChild): same
(ErrorfulUnslicer.receiveClose): same
(FailingUnslicer.receiveChild): same
* slicer.py: implement new receiveChild/receiveClose signature.
Require that ready_deferred == None for now.
(ListUnslicer.receiveChild): put "placeholder" in the list instead
of the Deferred
(TupleUnslicer.start): change the way we keep track of
not-yet-constructable tuples, using a counter of unreferenceable
children instead of counting the Deferred placeholders in the list
(TupleUnslicer.receiveChild): put "placeholder" in the list
instead of the Deferred
* banana.py (Banana.reportReceiveError): when debugging, log the
exception in a way that doesn't cause trial to think the test
failed.
(Banana.handleToken): implement new receiveChild signature
(Banana.handleClose): same
* debug.py (LoggingBananaMixin.handleToken): same
* tokens.py (IUnslicer.receiveChild): new signature for
receiveClose and receiveChild, they now pass a pair of (obj,
ready_deferred), where obj is still object-or-deferred, but
ready_deferred is non-None when the object will not be ready to
use until some other event takes place (like a "slow" global
reference is established).
# Local Variables:
# add-log-time-format: add-log-iso8601-time-string
# End:
|