File: HISTORY.txt

package info (click to toggle)
zodb 1:3.10.7-1
  • links: PTS, VCS
  • area: main
  • in suites: buster, sid, stretch
  • size: 3,988 kB
  • ctags: 5,875
  • sloc: python: 33,695; ansic: 7,673; xml: 474; sh: 20; makefile: 18
file content (3997 lines) | stat: -rw-r--r-- 157,017 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
2479
2480
2481
2482
2483
2484
2485
2486
2487
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
2529
2530
2531
2532
2533
2534
2535
2536
2537
2538
2539
2540
2541
2542
2543
2544
2545
2546
2547
2548
2549
2550
2551
2552
2553
2554
2555
2556
2557
2558
2559
2560
2561
2562
2563
2564
2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649
2650
2651
2652
2653
2654
2655
2656
2657
2658
2659
2660
2661
2662
2663
2664
2665
2666
2667
2668
2669
2670
2671
2672
2673
2674
2675
2676
2677
2678
2679
2680
2681
2682
2683
2684
2685
2686
2687
2688
2689
2690
2691
2692
2693
2694
2695
2696
2697
2698
2699
2700
2701
2702
2703
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
2717
2718
2719
2720
2721
2722
2723
2724
2725
2726
2727
2728
2729
2730
2731
2732
2733
2734
2735
2736
2737
2738
2739
2740
2741
2742
2743
2744
2745
2746
2747
2748
2749
2750
2751
2752
2753
2754
2755
2756
2757
2758
2759
2760
2761
2762
2763
2764
2765
2766
2767
2768
2769
2770
2771
2772
2773
2774
2775
2776
2777
2778
2779
2780
2781
2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801
2802
2803
2804
2805
2806
2807
2808
2809
2810
2811
2812
2813
2814
2815
2816
2817
2818
2819
2820
2821
2822
2823
2824
2825
2826
2827
2828
2829
2830
2831
2832
2833
2834
2835
2836
2837
2838
2839
2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853
2854
2855
2856
2857
2858
2859
2860
2861
2862
2863
2864
2865
2866
2867
2868
2869
2870
2871
2872
2873
2874
2875
2876
2877
2878
2879
2880
2881
2882
2883
2884
2885
2886
2887
2888
2889
2890
2891
2892
2893
2894
2895
2896
2897
2898
2899
2900
2901
2902
2903
2904
2905
2906
2907
2908
2909
2910
2911
2912
2913
2914
2915
2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
2927
2928
2929
2930
2931
2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942
2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957
2958
2959
2960
2961
2962
2963
2964
2965
2966
2967
2968
2969
2970
2971
2972
2973
2974
2975
2976
2977
2978
2979
2980
2981
2982
2983
2984
2985
2986
2987
2988
2989
2990
2991
2992
2993
2994
2995
2996
2997
2998
2999
3000
3001
3002
3003
3004
3005
3006
3007
3008
3009
3010
3011
3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
3027
3028
3029
3030
3031
3032
3033
3034
3035
3036
3037
3038
3039
3040
3041
3042
3043
3044
3045
3046
3047
3048
3049
3050
3051
3052
3053
3054
3055
3056
3057
3058
3059
3060
3061
3062
3063
3064
3065
3066
3067
3068
3069
3070
3071
3072
3073
3074
3075
3076
3077
3078
3079
3080
3081
3082
3083
3084
3085
3086
3087
3088
3089
3090
3091
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101
3102
3103
3104
3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
3146
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
3172
3173
3174
3175
3176
3177
3178
3179
3180
3181
3182
3183
3184
3185
3186
3187
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
3202
3203
3204
3205
3206
3207
3208
3209
3210
3211
3212
3213
3214
3215
3216
3217
3218
3219
3220
3221
3222
3223
3224
3225
3226
3227
3228
3229
3230
3231
3232
3233
3234
3235
3236
3237
3238
3239
3240
3241
3242
3243
3244
3245
3246
3247
3248
3249
3250
3251
3252
3253
3254
3255
3256
3257
3258
3259
3260
3261
3262
3263
3264
3265
3266
3267
3268
3269
3270
3271
3272
3273
3274
3275
3276
3277
3278
3279
3280
3281
3282
3283
3284
3285
3286
3287
3288
3289
3290
3291
3292
3293
3294
3295
3296
3297
3298
3299
3300
3301
3302
3303
3304
3305
3306
3307
3308
3309
3310
3311
3312
3313
3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
3324
3325
3326
3327
3328
3329
3330
3331
3332
3333
3334
3335
3336
3337
3338
3339
3340
3341
3342
3343
3344
3345
3346
3347
3348
3349
3350
3351
3352
3353
3354
3355
3356
3357
3358
3359
3360
3361
3362
3363
3364
3365
3366
3367
3368
3369
3370
3371
3372
3373
3374
3375
3376
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387
3388
3389
3390
3391
3392
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404
3405
3406
3407
3408
3409
3410
3411
3412
3413
3414
3415
3416
3417
3418
3419
3420
3421
3422
3423
3424
3425
3426
3427
3428
3429
3430
3431
3432
3433
3434
3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461
3462
3463
3464
3465
3466
3467
3468
3469
3470
3471
3472
3473
3474
3475
3476
3477
3478
3479
3480
3481
3482
3483
3484
3485
3486
3487
3488
3489
3490
3491
3492
3493
3494
3495
3496
3497
3498
3499
3500
3501
3502
3503
3504
3505
3506
3507
3508
3509
3510
3511
3512
3513
3514
3515
3516
3517
3518
3519
3520
3521
3522
3523
3524
3525
3526
3527
3528
3529
3530
3531
3532
3533
3534
3535
3536
3537
3538
3539
3540
3541
3542
3543
3544
3545
3546
3547
3548
3549
3550
3551
3552
3553
3554
3555
3556
3557
3558
3559
3560
3561
3562
3563
3564
3565
3566
3567
3568
3569
3570
3571
3572
3573
3574
3575
3576
3577
3578
3579
3580
3581
3582
3583
3584
3585
3586
3587
3588
3589
3590
3591
3592
3593
3594
3595
3596
3597
3598
3599
3600
3601
3602
3603
3604
3605
3606
3607
3608
3609
3610
3611
3612
3613
3614
3615
3616
3617
3618
3619
3620
3621
3622
3623
3624
3625
3626
3627
3628
3629
3630
3631
3632
3633
3634
3635
3636
3637
3638
3639
3640
3641
3642
3643
3644
3645
3646
3647
3648
3649
3650
3651
3652
3653
3654
3655
3656
3657
3658
3659
3660
3661
3662
3663
3664
3665
3666
3667
3668
3669
3670
3671
3672
3673
3674
3675
3676
3677
3678
3679
3680
3681
3682
3683
3684
3685
3686
3687
3688
3689
3690
3691
3692
3693
3694
3695
3696
3697
3698
3699
3700
3701
3702
3703
3704
3705
3706
3707
3708
3709
3710
3711
3712
3713
3714
3715
3716
3717
3718
3719
3720
3721
3722
3723
3724
3725
3726
3727
3728
3729
3730
3731
3732
3733
3734
3735
3736
3737
3738
3739
3740
3741
3742
3743
3744
3745
3746
3747
3748
3749
3750
3751
3752
3753
3754
3755
3756
3757
3758
3759
3760
3761
3762
3763
3764
3765
3766
3767
3768
3769
3770
3771
3772
3773
3774
3775
3776
3777
3778
3779
3780
3781
3782
3783
3784
3785
3786
3787
3788
3789
3790
3791
3792
3793
3794
3795
3796
3797
3798
3799
3800
3801
3802
3803
3804
3805
3806
3807
3808
3809
3810
3811
3812
3813
3814
3815
3816
3817
3818
3819
3820
3821
3822
3823
3824
3825
3826
3827
3828
3829
3830
3831
3832
3833
3834
3835
3836
3837
3838
3839
3840
3841
3842
3843
3844
3845
3846
3847
3848
3849
3850
3851
3852
3853
3854
3855
3856
3857
3858
3859
3860
3861
3862
3863
3864
3865
3866
3867
3868
3869
3870
3871
3872
3873
3874
3875
3876
3877
3878
3879
3880
3881
3882
3883
3884
3885
3886
3887
3888
3889
3890
3891
3892
3893
3894
3895
3896
3897
3898
3899
3900
3901
3902
3903
3904
3905
3906
3907
3908
3909
3910
3911
3912
3913
3914
3915
3916
3917
3918
3919
3920
3921
3922
3923
3924
3925
3926
3927
3928
3929
3930
3931
3932
3933
3934
3935
3936
3937
3938
3939
3940
3941
3942
3943
3944
3945
3946
3947
3948
3949
3950
3951
3952
3953
3954
3955
3956
3957
3958
3959
3960
3961
3962
3963
3964
3965
3966
3967
3968
3969
3970
3971
3972
3973
3974
3975
3976
3977
3978
3979
3980
3981
3982
3983
3984
3985
3986
3987
3988
3989
3990
3991
3992
3993
3994
3995
3996
3997

3.9.7 (2010-09-28)
==================

Bugs Fixed
----------

- Changes in way that garbage collection treats dictionaries in Python
  2.7 broke the object/connection cache implementation.
  (https://bugs.launchpad.net/zodb/+bug/641481)

  Python 2.7 wasn't officially supported, but we were releasing
  binaries for it, so ...

- Logrotation/repoening via a SIGUSR2 signal wasn't implemented.
  (https://bugs.launchpad.net/zodb/+bug/143600)

- When using multi-databases, cache-management operations on a
  connection, cacheMinimize and cacheGC, weren't applied to
  subconnections.

3.9.6 (2010-09-21)
==================

Bugs Fixed
----------

- Updating blobs in save points could cause spurious "invalidations
  out of order" errors.  https://bugs.launchpad.net/zodb/+bug/509801

  (Thanks to Christian Zagrodnick for chasing this down.)

- If a ZEO client process was restarted while invalidating a ZEO cache
  entry, the cache could be left in a stage when there is data marked
  current that should be invalidated, leading to persistent conflict
  errors.

- Corrupted or invalid cache files prevented ZEO clients from
  starting. Now, bad cache files are moved aside.

- Invalidations of object records in ZEO caches, where the
  invalidation transaction ids matched the cached transaction ids
  should have been ignored.

- Shutting down a process while committing a transaction or processing
  invalidations from the server could cause ZEO persistent client
  caches to have invalid data.  This, in turn caused stale data to
  remain in the cache until it was updated.

- Conflict errors didn't invalidate ZEO cache entries.

- When objects were added in savepoints and either the savepoint was
  rolled back (https://bugs.launchpad.net/zodb/+bug/143560) or the
  transaction was aborted
  (https://mail.zope.org/pipermail/zodb-dev/2010-June/013488.html)
  The objects' _p_oid and _p_jar variables weren't cleared, leading to
  surprizing errors.

- Objects added in transactions that were later aborted could have
  _p_changed still set (https://bugs.launchpad.net/zodb/+bug/615758).

- ZEO extension methods failed when a client reconnected to a
  storage. (https://bugs.launchpad.net/zodb/+bug/143344)

- On Mac OS X, clients that connected and disconnected quickly could
  cause a ZEO server to stop accepting connections, due to a failure
  to catch errors in the initial part of the connection process.

  The failure to properly handle exceptions while accepting
  connections is potentially problematic on other platforms.

  Fixes: https://bugs.launchpad.net/zodb/+bug/135108

- Passing keys or values outside the range of 32-bit ints on 64-bit
  platforms led to undetected overflow errors. Now these cases cause
  Type errors to be raised.

  https://bugs.launchpad.net/zodb/+bug/143237

- BTree sets and tree sets didn't correctly check values passed to
  update or to constructors, causing Python to exit under certain
  circumstances.

- The verbose mode of the fstest was broken.
  (https://bugs.launchpad.net/zodb/+bug/475996)

3.9.5 (2010-04-23)
==================

Bugs Fixed
----------

- Fixed bug in cPickleCache's byte size estimation logic.
  (https://bugs.launchpad.net/zodb/+bug/533015)

- Fixed a serious bug that caused cache failures when run
  with Python optimization turned on.

  https://bugs.launchpad.net/zodb/+bug/544305

- Fixed a bug that caused savepoint rollback to not properly
  set object state when objects implemented _p_invalidate methods
  that reloaded ther state (unghostifiable objects).

  https://bugs.launchpad.net/zodb/+bug/428039

- cross-database wekrefs weren't handled correctly.

  https://bugs.launchpad.net/zodb/+bug/435547

- The mkzeoinst script was fixed to tell people to
  install and use the mkzeoinstance script. :)

3.9.4 (2009-12-14)
==================

Bugs Fixed
----------

- A ZEO threading bug could cause transactions to read inconsistent
  data.  (This sometimes caused an AssertionError in
  Connection._setstate_noncurrent.)

- DemoStorage.loadBefore sometimes returned invalid data which
  would trigger AssertionErrors in ZODB.Connection.

- History support was broken when using stprages that work with ZODB
  3.8 and 3.9.

- zope.testing was an unnecessary non-testing dependency.

- Internal ZEO errors were logged at the INFO level, rather
  than at the error level.

- The FileStorage backup and restore script, repozo, gave a
  deprecation warning under Python 2.6.

- C Header files weren't installed correctly.

- The undo implementation was incorrect in ways that could cause
  subtle missbehaviors.

3.9.3 (2009-10-23)
==================

Bugs Fixed
----------

- 2 BTree bugs, introduced by a bug fix in 3.9.0c2, sometimes caused
  deletion of keys to be improperly handled, resulting in data being
  available via iteraation but not item access.

3.9.2 (2009-10-13)
==================

Bugs Fixed
----------

- ZEO manages a separate thread for client network IO.  It created
  this thread on import, which caused problems for applications that
  implemented daemon behavior by forking.  Now, the client thread
  isn't created until needed.

- File-storage pack clean-up tasks that can take a long time
  unnecessarily blocked other activity.

- In certain rare situations, ZEO client connections would hang during
  the initial connection setup.

3.9.1 (2009-10-01)
==================

Bugs Fixed
----------

- Conflict errors committing blobs caused ZEO servers to stop committing
  transactions.

3.9.0 (2009-09-08)
==================

New Features (in more or less reverse chronological order)
----------------------------------------------------------

- The Database class now has an ``xrefs`` keyword argument and a
  corresponding allow-implicit-cross-references configuration option.
  which default to true.  When set to false, cross-database references
  are disallowed.

- Added support for RelStorage.

- As a convenience, the connection root method for returning the root
  object can now *also* be used as an object with attributes mapped to
  the root-object keys.

- Databases have a new method, ``transaction``, that can be used with the
  Python (2.5 and later) ``with`` statement::

     db = ZODB.DB(...)
     with db.transaction() as conn:
          # ... do stuff with conn

  This uses a private transaction manager for the connection.
  If control exits the block without an error, the transaction is
  committed, otherwise, it is aborted.

- Convenience functions ZODB.connection and ZEO.connection provide a
  convenient way to open a connection to a database.  They open a
  database and return a connection to it. When the connection is
  closed, the database is closed as well.

- The ZODB.config databaseFrom... methods now support
  multi-databases. If multiple zodb sections are used to define
  multiple databases, the databases are connected in a multi-database
  arrangement and the first of the defined databases is returned.

- The zeopack script has gotten a number of improvements:

  - Simplified command-line interface. (The old interface is still
    supported, except that support for ZEO version 1 servers has been
    dropped.)

  - Multiple storages can be packed in sequence.

    - This simplifies pack scheduling on servers serving multiple
      databases.

    - All storages are packed to the same time.

  - You can now specify a time of day to pack to.

  - The script will now time out if it can't connect to s storage in
    60 seconds.

- The connection now estimates the object size based on its pickle size
  and informs the cache about size changes.

  The database got additional configurations options (`cache-size-bytes`
  and `historical-cache-size-bytes`) to limit the
  cache size based on the estimated total size of cached objects.
  The default values are 0 which has the interpretation "do not limit
  based on the total estimated size".
  There are corresponding methods to read and set the new configuration
  parameters.

- Connections now have a public ``opened`` attribute that is true when
  the connection is open, and false otherwise.  When true, it is the
  seconds since the epoch (time.time()) when the connection was
  opened. This is a renaming of the previous ``_opened`` private
  variable.

- FileStorage now supports blobs directly.

- You can now control whether FileStorages keep .old files when packing.

- POSKeyErrors are no longer logged by ZEO servers, because they are
  really client errors.

- A new storage interface, IExternalGC, to support external garbage
  collection, http://wiki.zope.org/ZODB/ExternalGC, has been defined
  and implemented for FileStorage and ClientStorage.

- As a small convenience (mainly for tests), you can now specify
  initial data as a string argument to the Blob constructor.

- ZEO Servers now provide an option, invalidation-age, that allows
  quick verification of ZEO clients have been disconnected for less
  than a given time even if the number of transactions the client
  hasn't seen exceeds the invalidation queue size. This is only
  recommended if the storage being served supports efficient iteration
  from a point near the end of the transaction history.

- The FileStorage iterator now handles large files better.  When
  iterating from a starting transaction near the end of the file, the
  iterator will scan backward from the end of the file to find the
  starting point.  This enhancement makes it practical to take
  advantage of the new storage server invalidation-age option.

- Previously, database connections were managed as a stack.  This
  tended to cause the same connection(s) to be used over and over.
  For example, the most used connection would typically be the only
  connection used.  In some rare situations, extra connections could
  be opened and end up on the top of the stack, causing extreme memory
  wastage.  Now, when connections are placed on the stack, they sink
  below existing connections that have more active objects.

- There is a new pool-timeout database configuration option to specify that
  connections unused after the given time interval should be garbage
  collection.  This will provide a means of dealing with extra
  connections that are created in rare circumstances and that would
  consume an unreasonable amount of memory.

- The Blob open method now supports a new mode, 'c', to open committed
  data for reading as an ordinary file, rather than as a blob file.
  The ordinary file may be used outside the current transaction and
  even after the blob's database connection has been closed.

- ClientStorage now provides blob cache management. When using
  non-shared blob directories, you can set a target cache size and the
  cache will periodically be reduced try to keep it below the target size.

  The client blob directory layout has changed.  If you have existing
  non-shared blob directories, you will have to remove them.

- ZODB 3.9 ZEO clients can connect to ZODB 3.8 servers.  ZODB ZEO clients
  from ZODB 3.2 on can connect to ZODB 3.9 servers.

- When a ZEO cache is stale and would need verification, a
  ZEO.interfaces.StaleCache event is published (to zope.event).
  Applications may handle this event and take action such as exiting
  the application without verifying the cache or starting cold.

- There's a new convenience function, ZEO.DB, for creating databases
  using ZEO Client Storages.  Just call ZEO.DB with the same arguments
  you would otherwise pass to ZEO.ClientStorage.ClientStorage::

    import ZEO
    db = ZEO.DB(('some_host', 8200))

- Object saves are a little faster

- When configuring storages in a storage server, the storage name now
  defaults to "1".  In the overwhelmingly common case that a single
  storage, the name can now be omitted.

- FileStorage now provides optional garbage collection.  A 'gc'
  keyword option can be passed to the pack method.  A false value
  prevents garbage collection.

- The FileStorage constructor now provides a boolean pack_gc option,
  which defaults to True, to control whether garbage collection is
  performed when packing by default. This can be overridden with the
  gc option to the pack method.

  The ZConfig configuration for FileStorage now includes a pack-gc
  option, corresponding to the pack_gc constructor argument.

- The FileStorage constructor now has a packer keyword argument that
  allows an alternative packer to be supplied.

  The ZConfig configuration for FileStorage now includes a packer
  option, corresponding to the packer constructor argument.

- MappingStorage now supports multi-version concurrency control and
  iteration and provides a better storage implementation example.

- DemoStorage has a number of new features:

  - The ability to use a separate storage, such as a file storage to
    store changes

  - Blob support

  - Multi-version concurrency control and iteration

  - Explicit support for demo-storage stacking via push and pop methods.

- Wen calling ZODB.DB to create a database, you can now pass a file
  name, rather than a storage to use a file storage.

- Added support for copying and recovery of blob storages:

  - Added a helper function, ZODB.blob.is_blob_record for testing whether
    a data record is for a blob.  This can be used when iterating over a
    storage to detect blob records so that blob data can be copied.

    In the future, we may want to build this into a blob-aware
    iteration interface, so that records get blob file attributes
    automatically.

  - Added the IBlobStorageRestoreable interfaces for blob storages
    that support recovery via a restoreBlob method.

  - Updated ZODB.blob.BlobStorage to implement
    IBlobStorageRestoreable and to have a copyTransactionsFrom method
    that also copies blob data.

- New `ClientStorage` configuration option `drop_cache_rather_verify`.
  If this option is true then the ZEO client cache is dropped instead of
  the long (unoptimized) verification. For large caches, setting this
  option can avoid effective down times in the order of hours when
  the connection to the ZEO server was interrupted for a longer time.

- Cleaned-up the storage iteration API and provided an iterator implementation
  for ZEO.

- Versions are no-longer supported.

- Document conflict resolution (see ZODB/ConflictResolution.txt).

- Support multi-database references in conflict resolution.

- Make it possible to examine oid and (in some situations) database
  name of persistent object references during conflict resolution.

- Moved the 'transaction' module out of ZODB.
  ZODB depends upon this module, but it must be installed separately.

- ZODB installation now requires setuptools.

- Added `offset` information to output of `fstail`
  script. Added test harness for this script.

- Added support for read-only, historical connections based
  on datetimes or serials (TIDs).  See
  src/ZODB/historical_connections.txt.

- Removed the ThreadedAsync module.

- Now depend on zc.lockfile

Bugs Fixed
----------

- CVE-2009-2701: Fixed a vulnerability in ZEO storage servers when
  blobs are available. Someone with write access to a ZEO server
  configured to support blobs could read any file on the system
  readable by the server process and remove any file removable by the
  server process.

- BTrees (and TreeSets) kept references to internal keys.
  https://bugs.launchpad.net/zope3/+bug/294788

- BTree Sets and TreeSets don't support the standard set add method.
  (Now either add or the original insert method can be used to add an
  object to a BTree-based set.)

- The runzeo script didn't work without a configuration file.
  (https://bugs.launchpad.net/zodb/+bug/410571)

- Officially deprecated PersistentDict
  (https://bugs.launchpad.net/zodb/+bug/400775)

- Calling __setstate__ on a persistent object could under certain
  uncommon cause the process to crash.
  (https://bugs.launchpad.net/zodb/+bug/262158)

- When committing transactions involving blobs to ClientStorages with
  non-shared blob directories, a failure could occur in tpc_finish if
  there was insufficient disk space to copy the blob file or if the
  file wasn't available.  https://bugs.launchpad.net/zodb/+bug/224169

- Savepoint blob data wasn't properly isolated. If multiple
  simultaneous savepoints in separate transactions modified the same
  blob, data from one savepoint would overwrite data for another.

- Savepoint blob data wasn't cleaned up after a transaction abort.
  https://bugs.launchpad.net/zodb/+bug/323067

- Opening a blob with modes 'r+' or 'a' would fail when the blob had no
  committed changes.

- PersistentList's sort method did not allow passing of keyword parameters.
  Changed its sort parameter list to match that of its (Python 2.4+)
  UserList base class.

- Certain ZEO server errors could cause a client to get into a state
  where it couldn't commit transactions.
  https://bugs.launchpad.net/zodb/+bug/374737

- Fixed vulnerabilities in the ZEO network protocol that allow:

  - CVE-2009-0668 Arbitrary Python code execution in ZODB ZEO storage servers
  - CVE-2009-0669 Authentication bypass in ZODB ZEO storage servers

  The vulnerabilities only apply if you are using ZEO to share a
  database among multiple applications or application instances and if
  untrusted clients are able to connect to your ZEO servers.

- Fixed the setup test command.  It previously depended on private
  functions in zope.testing.testrunner that don't exist any more.

- ZEO client threads were unnamed, making it hard to debug thread
  management.

- ZEO protocol 2 support was broken.  This caused very old clients to
  be unable to use new servers.

- zeopack was less flexible than it was before.  -h should default to
  local host.

- The "lawn" layout was being selected by default if the root of
  the blob directory happened to contain a hidden file or directory
  such as ".svn".  Now hidden files and directories are ignored
  when choosing the default layout.

- BlobStorage was not compatible with MVCC storages because the
  wrappers were being removed by each database connection.  Fixed.

- Saving indexes for large file storages failed (with the error:
  RuntimeError: maximum recursion depth exceeded).  This can cause a
  FileStorage to fail to start because it gets an error trying to save
  its index.

- Sizes of new objects weren't added to the object cache size
  estimation, causing the object-cache size limiting feature to let
  the cache grow too large when many objects were added.

- Deleted records weren't removed when packing file storages.

- Fixed analyze.py and added test.

- fixed Python 2.6 compatibility issue with ZEO/zeoserverlog.py

- using hashlib.sha1 if available in order to avoid DeprecationWarning
  under Python 2.6

- made runzeo -h work

- The monitor server didn't correctly report the actual number of
  clients.

- Packing could return spurious errors due to errors notifying
  disconnected clients of new database size statistics.

- Undo sometimes failed for FileStorages configured to support blobs.

- Starting ClientStorages sometimes failed with non-new but empty
  cache files.

- The history method on ZEO clients failed.

- Fix for bug #251037: Make packing of blob storages non-blocking.

- Fix for bug #220856: Completed implementation of ZEO authentication.

- Fix for bug #184057: Make initialisation of small ZEO client file cache
  sizes not fail.

- Fix for bug #184054: MappingStorage used to raise a KeyError during `load`
  instead of a POSKeyError.

- Fixed bug in Connection.TmpStore: load() would not defer to the backend
  storage for loading blobs.

- Fix for bug #181712: Make ClientStorage update `lastTransaction` directly
  after connecting to a server, even when no cache verification is necessary.

- Fixed bug in blob filesystem helper: the `isSecure` check was inverted.

- Fixed bug in transaction buffer: a tuple was unpacked incorrectly in
  `clear`.

- Bugfix the situation in which comparing persistent objects (for
  instance, as members in BTree set or keys of BTree) might cause data
  inconsistency during conflict resolution.

- Fixed bug 153316: persistent and BTrees were using `int`
  for memory sizes which caused errors on x86_64 Intel Xeon machines
  (using 64-bit Linux).

- Fixed small bug that the Connection.isReadOnly method didn't
  work after a savepoint.

- Bug #98275: Made ZEO cache more tolerant when invalidating current
  versions of objects.

- Fixed a serious bug that could cause client I/O to stop
  (hang). This was accompanied by a critical log message along the
  lines of: "RuntimeError: dictionary changed size during iteration".

- Fixed bug #127182: Blobs were subclassable which was not desired.

- Fixed bug #126007: tpc_abort had untested code path that was
  broken.

- Fixed bug #129921: getSize() function in BlobStorage could not
  deal with garbage files

- Fixed bug in which MVCC would not work for blobs.

- Fixed bug in ClientCache that occurred with objects larger than the total
  cache size.

- When an error occured attempting to lock a file and logging of said error was
  enabled.

- FileStorages previously saved indexes after a certain
  number of writes.  This was done during the last phase of two-phase
  commit, which made this critical phase more subject to errors than
  it should have been.  Also, for large databases, saves were done so
  infrequently as to be useless.  The feature was removed to reduce
  the chance for errors during the last phase of two-phase commit.

- File storages previously kept an internal object id to
  transaction id mapping as an optimization. This mapping caused
  excessive memory usage and failures during the last phase of
  two-phase commit. This optimization has been removed.

- Refactored handling of invalidations on ZEO clients to fix
  a possible ordering problem for invalidation messages.

- On many systems, it was impossible to create more than 32K
  blobs. Added a new blob-directory layout to work around this
  limitation.

- Fixed bug that could lead to memory errors due to the use
  of a Python dictionary for a mapping that can grow large.

- Fixed bug #251037: Made packing of blob storages non-blocking.

- Fixed a bug that could cause InvalidObjectReference errors
  for objects that were explicitly added to a database if the object
  was modified after a savepoint that added the object.

- Fixed several bugs that caused ZEO cache corruption when connecting
  to servers. These bugs affected both persistent and non-persistent caches.

- Improved the the ZEO client shutdown support to try to
  avoid spurious errors on exit, especially for scripts, such as zeopack.

- Packing failed for databases containing cross-database references.

- Cross-database references to databases with empty names
  weren't constructed properly.

- The zeo client cache used an excessive amount of memory, causing applications
  with large caches to exhaust available memory.

- Fixed a number of bugs in the handling of persistent ZEO caches:

  - Cache records are written in several steps.  If a process exits
    after writing begins and before it is finishes, the cache will be
    corrupt on restart.  The way records are written was changed to
    make cache record updates atomic.

  - There was no lock file to prevent opening a cache multiple times
    at once, which would lead to corruption.  Persistent caches now
    use lock files, in the same way that file storages do.

  - A bug in the cache-opening logic led to cache failure in the
    unlikely event that a cache has no free blocks.

- When using ZEO Client Storages, Errors occured when trying to store
  objects too big to fit in the ZEO cache file.

- Fixed bug in blob filesystem helper: the `isSecure` check was inverted.

- Fixed bug in transaction buffer: a tuple was unpacked incorrectly in
  `clear`.

- Fixed bug in Connection.TmpStore: load() would not defer to the
  back-end storage for loading blobs.

- Fixed bug #190884: Wrong reference to `POSKeyError` caused NameError.

- Completed implementation of ZEO authentication. This fixes issue 220856.


What's new in ZODB 3.8.0
========================

General
-------

- (unreleased) Fixed setup.py use of setuptools vs distutils, so .c and .h
  files are included in the bdist_egg.

- The ZODB Storage APIs have been documented and cleaned up.

- ZODB versions are now officially deprecated and support for them
  will be removed in ZODB 3.9.  (They have been widely recognized as
  deprecated for quite a while.)

- Changed the automatic garbage collection when opening a connection to only
  apply the garbage collections on those connections in the pool that are
  closed. (This fixed issue 113923.)

ZEO
---

- (3.8a1) ZEO's strategoes for avoiding client cache verification were
  improved in the case that servers are restarted.  Before, if
  transactions were committed after the restart, clients that were up
  to date or nearly up to date at the time of the restart and then
  connected had to verify their caches.  Now, it is far more likely
  that a client that reconnects soon after a server restart won't have
  to verify its cache.

- (3.8a1) Fixed a serious bug that could cause clients that disconnect from and
  reconnect to a server to get bad invalidation data if the server
  serves multiple storages with active writes.

- (3.8a1) It is now theoretically possible to use a ClientStorage in a storage
  server.  This might make it possible to offload read load from a
  storage server at the cost of increasing write latency.  This should
  increase write throughput by offloading reads from the final storage
  server.  This feature is somewhat experimental.  It has tests, but
  hasn't been used in production.

Transactions
------------

- (3.8a1) Add a doom() and isDoomed() interface to the transaction module.

  First step towards the resolution of
  http://www.zope.org/Collectors/Zope3-dev/655

  A doomed transaction behaves exactly the same way as an active transaction
  but raises an error on any attempt to commit it, thus forcing an abort.

  Doom is useful in places where abort is unsafe and an exception cannot be
  raised.  This occurs when the programmer wants the code following the doom to
  run but not commit. It is unsafe to abort in these circumstances as a
  following get() may implicitly open a new transaction.

  Any attempt to commit a doomed transaction will raise a DoomedTransaction
  exception.

- (3.8a1) Clean up the ZODB imports in transaction.

  Clean up weird import dance with ZODB. This is unnecessary since the
  transaction module stopped being imported in ZODB/__init__.py in rev 39622.

- (3.8a1) Support for subtransactions has been removed in favor of
  save points.

Blobs
-----

- (3.8b1) Updated the Blob implementation in a number of ways.  Some
  of these are backward incompatible with 3.8a1:

  o The Blob class now lives in ZODB.blob

  o The blob openDetached method has been replaced by the committed method.

- (3.8a1) Added new blob feature. See the ZODB/Blobs directory for
  documentation.

  ZODB now handles (reasonably) large binary objects efficiently. Useful to
  use from a few kilobytes to at least multiple hundred megabytes.

BTrees
------

- (3.8a1) Added support for 64-bit integer BTrees as separate types.  

  (For now, we're retaining compile-time support for making the regular
  integer BTrees 64-bit.)

- (3.8a1) Normalize names in modules so that BTrees, Buckets, Sets, and
  TreeSets can all be accessed with those names in the modules (e.g.,
  BTrees.IOBTree.BTree).  This is in addition to the older names (e.g.,
  BTrees.IOBTree.IOBTree).  This allows easier drop-in replacement, which
  can especially be simplify code for packages that want to support both
  32-bit and 64-bit BTrees.

- (3.8a1) Describe the interfaces for each module and actually declare
  the interfaces for each.

- (3.8a1) Fix module references so klass.__module__ points to the Python
  wrapper module, not the C extension.

- (3.8a1) introduce module families, to group all 32-bit and all 64-bit
  modules.


What's new in ZODB3 3.7.0
==========================
Release date: 2007-04-20

Packaging
---------

- (3.7.0b3) ZODB is now packaged without it's dependencies

  ZODB no longer includes copies of dependencies such as 
  ZConfig, zope.interface and so on.  It now treats these as
  dependencies.  If ZODB is installed with easy_install or
  zc.buildout, the dependencies will be installed automatically.


- (3.7.0b3) ZODB is now a buildout

  ZODB checkouts are now built and tested using zc.buildout.

- (3.7b4) Added logic to avoid spurious errors from the logging system
  on exit.

- (3.7b2) Removed the "sync" mode for ClientStorage.  

  Previously, a ClientStorage could be in either "sync" mode or "async"
  mode.  Now there is just "async" mode.  There is now a dedicicated
  asyncore main loop dedicated to ZEO clients.

  Applications no-longer need to run an asyncore main loop to cause
  client storages to run in async mode.  Even if an application runs an
  asyncore main loop, it is independent of the loop used by client
  storages. 

  This addresses a test failure on Mac OS X,
  http://www.zope.org/Collectors/Zope3-dev/650, that I believe was due
  to a bug in sync mode. Some asyncore-based code was being called from
  multiple threads that didn't expect to be.

  Converting to always-async mode revealed some bugs that weren't caught
  before because the tests ran in sync mode.  These problems could
  explain some problems we've seen at times with clients taking a long
  time to reconnect after a disconnect.

  Added a partial heart beat to try to detect lost connections that
  aren't otherwise caught,
  http://mail.zope.org/pipermail/zodb-dev/2005-June/008951.html, by
  perioidically writing to all connections during periods of inactivity.

Connection management
---------------------

- (3.7a1) When more than ``pool_size`` connections have been closed,
  ``DB`` forgets the excess (over ``pool_size``) connections closed first.
  Python's cyclic garbage collection can take "a long time" to reclaim them
  (and may in fact never reclaim them if application code keeps strong
  references to them), but such forgotten connections can never be opened
  again, so their caches are now cleared at the time ``DB`` forgets them.
  Most applications won't notice a difference, but applications that open
  many connections, and/or store many large objects in connection caches,
  and/or store limited resources (such as RDB connections) in connection
  caches may benefit.

BTrees
------

- Support for 64-bit integer keys and values has been provided as a
  compile-time option for the "I" BTrees (e.g. IIBTree).

Documentation
-------------

- (3.7a1) Thanks to Stephan Richter for converting many of the doctest
  files to ReST format.  These are now chapters in the Zope 3 apidoc too.

IPersistent
-----------

- (3.7a1) The documentation for ``_p_oid`` now specifies the concrete
  type of oids (in short, an oid is either None or a non-empty string).

Testing
-------

- (3.7b2) Fixed test-runner output truncation.

  A bug was fixed in the test runner that caused result summaries to be
  omitted when running on Windows.

Tools
-----

- (3.7a1) The changeover from zLOG to the logging module means that some
  tools need to perform minimal logging configuration themselves. Changed
  the zeoup script to do so and thus enable it to emit error messages.

BTrees
------

- (3.7a1) Suppressed warnings about signedness of characters when
  compiling under GCC 4.0.x.  See http://www.zope.org/Collectors/Zope/2027.

Connection
----------

- (3.7a1) An optimization for loading non-current data (MVCC) was
  inadvertently disabled in ``_setstate()``; this has been repaired.

persistent
----------

- (3.7a1) Suppressed warnings about signedness of characters when
  compiling under GCC 4.0.x.  See http://www.zope.org/Collectors/Zope/2027.

- (3.7a1) PersistentMapping was inadvertently pickling volatile attributes
  (http://www.zope.org/Collectors/Zope/2052).

After Commit hooks
------------------

- (3.7a1) Transaction objects have a new method,
  ``addAfterCommitHook(hook, *args, **kws)``.  Hook functions
  registered with a transaction are called after the transaction
  commits or aborts. For example, one might want to launch non
  transactional or asynchrnonous code after a successful, or aborted,
  commit. See ``test_afterCommitHook()`` in
  ``transaction/tests/test_transaction.py`` for a tutorial doctest,
  and the ``ITransaction`` interface for details.


What's new in ZODB3 3.6.2?
==========================
Release date: 15-July-2006


DemoStorage
-----------

- (3.6.2) DemoStorage was unable to wrap base storages who did not have
          an '_oid' attribute:  most notably, ZEO.ClientStorage
          (http://www.zope.org/Collectors/Zope/2016).

Following is combined news from internal releases (to support ongoing
Zope2 / Zope3 development).  These are the dates of the internal releases:

- 3.6.1 27-Mar-2006
- 3.6.0 05-Jan-2006
- 3.6b6 01-Jan-2006
- 3.6b5 18-Dec-2005
- 3.6b4 04-Dec-2005
- 3.6b3 06-Nov-2005
- 3.6b2 25-Oct-2005
- 3.6b1 24-Oct-2005
- 3.6a4 07-Oct-2005
- 3.6a3 07-Sep-2005
- 3.6a2 06-Sep-2005
- 3.6a1 04-Sep-2005

Removal of Features Deprecated in ZODB 3.4
------------------------------------------

(3.6b2) ZODB 3.6 no longer contains features officially deprecated in the
ZODB 3.4 release.  These include:

- ``get_transaction()``.  Use ``transaction.get()`` instead.
  ``transaction.commit()`` is a shortcut spelling of
  ``transaction.get().commit()``, and ``transaction.abort()``
  of ``transaction.get().abort()``.  Note that importing ZODB no longer
  installs ``get_transaction`` as a name in Python's ``__builtin__``
  module either.

- The ``begin()`` method of ``Transaction`` objects.  Use the ``begin()``
  method of a transaction manager instead.  ``transaction.begin()`` is
  a shortcut spelling to call the default transaction manager's ``begin()``
  method.

- The ``dt`` argument to ``Connection.cacheMinimize()``.

- The ``Connection.cacheFullSweep()`` method.  Use ``cacheMinimize()``
  instead.

- The ``Connection.getTransaction()`` method.  Pass a transaction manager
  to ``DB.open()`` instead.

- The ``Connection.getLocalTransaction()`` method.  Pass a transaction
  manager to ``DB.open()`` instead.

- The ``cache_deactivate_after`` and ``version_cache_deactivate_after``
  arguments to the ``DB`` constructor.

- The ``temporary``, ``force``, and ``waitflag`` arguments
  to ``DB.open()``.  ``DB.open()`` no longer blocks (there's no longer
  a fixed limit on the number of open connections).

- The ``transaction`` and ``txn_mgr``arguments to ``DB.open()``.  Use
  the ``transaction_manager`` argument instead.

- The ``getCacheDeactivateAfter``, ``setCacheDeactivateAfter``,
  ``getVersionCacheDeactivateAfter`` and ``setVersionCacheDeactivateAfter``
  methods of ``DB``.

Persistent
----------

- (3.6.1) Suppressed warnings about signedness of characters when
  compiling under GCC 4.0.x.  See http://www.zope.org/Collectors/Zope/2027.

- (3.6a4) ZODB 3.6 introduces a change to the basic behavior of Persistent
  objects in a particular end case.  Before ZODB 3.6, setting
  ``obj._p_changed`` to a true value when ``obj`` was a ghost was ignored:
  ``obj`` remained a ghost, and getting ``obj._p_changed`` continued to
  return ``None``.  Starting with ZODB 3.6, ``obj`` is activated instead
  (unghostified), and its state is changed from the ghost state to the
  changed state.  The new behavior is less surprising and more robust.

- (3.6b5) The documentation for ``_p_oid`` now specifies the concrete
  type of oids (in short, an oid is either None or a non-empty string).

Commit hooks
------------

- (3.6a1) The ``beforeCommitHook()`` method has been replaced by the new
  ``addBeforeCommitHook()`` method, with a more-robust signature.
  ``beforeCommitHook()`` is now deprecated, and will be removed in ZODB 3.8.
  Thanks to Julien Anguenot for contributing code and tests.

Connection management
---------------------

- (3.6b6) When more than ``pool_size`` connections have been closed,
  ``DB`` forgets the excess (over ``pool_size``) connections closed first.
  Python's cyclic garbage collection can take "a long time" to reclaim them
  (and may in fact never reclaim them if application code keeps strong
  references to them), but such forgotten connections can never be opened
  again, so their caches are now cleared at the time ``DB`` forgets them.
  Most applications won't notice a difference, but applications that open
  many connections, and/or store many large objects in connection caches,
  and/or store limited resources (such as RDB connections) in connection
  caches may benefit.

ZEO
---

- (3.6a4) Collector 1900.  In some cases of pickle exceptions raised by
  low-level ZEO communication code, callers of ``marshal.encode()`` could
  attempt to catch an exception that didn't actually exist, leading to an
  erroneous ``AttributeError`` exception.  Thanks to Tres Seaver for the
  diagnosis.

BaseStorage
-----------

- (3.6a4) Nothing done by ``tpc_abort()`` should raise an exception.
  However, if something does (an error case), ``BaseStorage.tpc_abort()``
  left the commit lock in the acquired state, causing any later attempt
  to commit changes hang.

Multidatabase
-------------

- (3.6b1) The ``database_name`` for a database in a multidatabase
  collection can now be specified in a config file's ``<zodb>`` section,
  as the value of the optional new ``database_name`` key.  The
  ``.databases`` attribute cannot be specified in a config file, but
  can be passed as the optional new ``databases`` argument to the
  ``open()`` method of a ZConfig factory for type ``ZODBDatabase``.
  For backward compatibility, Zope 2.9 continues to allow using the
  name in its ``<zodb_db name>`` config section as the database name
  (note that ``<zodb_db>`` is defined by Zope, not by ZODB -- it's a
  Zope-specific extension of ZODB's ``<zodb>`` section).

PersistentMapping
-----------------

- (3.6.1) PersistentMapping was inadvertently pickling volatile attributes
  (http://www.zope.org/Collectors/Zope/2052).

- (3.6b4) ``PersistentMapping`` makes changes by a ``pop()`` method call
  persistent now (http://www.zope.org/Collectors/Zope/2036).

- (3.6a1) The ``PersistentMapping`` class has an ``__iter__()`` method
  now, so that objects of this type work well with Python's iteration
  protocol.  For example, if ``x`` is a ``PersistentMapping`` (or
  Python dictionary, or BTree, or ``PersistentDict``, ...), then
  ``for key in x:`` iterates over the keys of ``x``, ``list(x)`` creates
  a list containing ``x``'s keys, ``iter(x)`` creates an iterator for
  ``x``'s keys, and so on.

Tools
-----

- (3.6b5) The changeover from zLOG to the logging module means that some
  tools need to perform minimal logging configuration themselves. Changed
  the zeoup script to do so and thus enable it to emit error messages.

BTrees
------

- (3.6.1) Suppressed warnings about signedness of characters when
  compiling under GCC 4.0.x.  See http://www.zope.org/Collectors/Zope/2027.

- (3.6a1) BTrees and Buckets now implement the ``setdefault()`` and ``pop()``
  methods. These are exactly like Python's dictionary methods of the same
  names, except that ``setdefault()`` requires both arguments (and Python is
  likely to change to require both arguments too -- defaulting the
  ``default`` argument to ``None`` has no viable use cases).  Thanks to
  Ruslan Spivak for contributing code, tests, and documentation.

- (3.6a1) Collector 1873.  It wasn't possible to construct a BTree or Bucket
  from, or apply their update() methods to, a PersistentMapping or
  PersistentDict.  This works now.

ZopeUndo
--------

- (3.6a4) Collector 1810. A previous bugfix (#1726) broke listing undoable
  transactions for users defined in a non-root acl_users folder. Zope logs
  a acl_users path together with a username (separated by a space) and this
  previous fix failed to take this into account.

Connection
----------

- (3.6b5) An optimization for loading non-current data (MVCC) was
  inadvertently disabled in ``_setstate()``; this has been repaired.

Documentation
-------------

- (3.6b3) Thanks to Stephan Richter for converting many of the doctest
  files to ReST format.  These are now chapters in the Zope 3 apidoc too.

- (3.6b4) Several misspellings of "occurred" were repaired.

Development
-----------

- (3.6a1) The source code for the old ExtensionClass-based Persistence
  package moved, from ZODB to the Zope 2.9 development tree.  ZODB 3.5
  makes no use of Persistence, and, indeed, the Persistence package could
  not be compiled from a ZODB release, since some of the C header files
  needed appear only in Zope.

- (3.6a3) Re-added the ``zeoctl`` module, for the same reasons
  ``mkzeoinst`` was re-added (see below).

- (3.6a2) The ``mkzeoinst`` module was re-added to ZEO, because Zope3
  has a script that expects to import it from there.  ZODB's ``mkzeoinst``
  script was rewritten to invoke the ``mkzeoinst`` module.

``transact``
------------

- (3.6b4) Collector 1959:  The undocumented ``transact`` module no
  longer worked.  It remains undocumented and untested, but thanks to
  Janko Hauser it's possible that it works again ;-).

What's new in ZODB3 3.5.1?
==========================
Release date: 26-Sep-2005

Following is combined news from internal releases (to support ongoing
Zope3 development).  These are the dates of the internal releases:

- 3.5.1b2 07-Sep-2005
- 3.5.1b1 06-Sep-2005

Build
-----

- (3.5.1b2) Re-added the ``zeoctl`` module, for the same reasons
  ``mkzeoinst`` was re-added (see below).

- (3.5.1b1) The ``mkzeoinst`` module was re-added to ZEO, because Zope3
  has a script that expects to import it from there.  ZODB's ``mkzeoinst``
  script was rewritten to invoke the ``mkzeoinst`` module.

ZopeUndo
--------

- (3.5.1) Collector 1810. A previous bugfix (#1726) broke listing undoable
  transactions for users defined in a non-root acl_users folder. Zope logs
  a acl_users path together with a username (separated by a space) and this
  previous fix failed to take this into account.


What's new in ZODB3 3.5.0?
==========================
Release date: 31-Aug-2005

Following is combined news from internal releases (to support ongoing
Zope3 development).  These are the dates of the internal releases:

- 3.5a7 11-Aug-2005
- 3.5a6 04-Aug-2005
- 3.5a5 19-Jul-2005
- 3.5a4 14-Jul-2005
- 3.5a3 17-Jun-2005
- 3.5a2 16-Jun-2005
- 3.5a1 10-Jun-2005

Savepoints
----------

- (3.5.0) As for deprecated subtransaction commits, the intent was
  that making a savepoint would invoke incremental garbage collection on
  Connection memory caches, to try to reduce the number of objects in
  cache to the configured cache size.  Due to an oversight, this didn't
  happen, and stopped happening for subtransaction commits too.  Making a
  savepoint (or doing a subtransaction commit) does invoke cache gc now.

- (3.5a3) When a savepoint is made, the states of objects modified so far
  are saved to a temporary storage (an instance of class ``TmpStore``,
  although that's an internal implementation detail).  That storage needs
  to implement the full storage API too, but was missing the ``loadBefore()``
  method needed for MVCC to retrieve non-current revisions of objects.  This
  could cause spurious errors if a transaction with a pending savepoint
  needed to fetch an older revision of some object.

- (3.5a4) The ``ISavepoint`` interface docs said you could roll back to a
  given savepoint any number of times (until the transaction ends, or until
  you roll back to an earlier savepoint's state), but the implementation
  marked a savepoint as invalid after its first use.  The implementation has
  been repaired, to match the docs.

ZEO client cache
----------------

- (3.5a6) Two memory leaks in the ZEO client cache were repaired, a
  major one involving ``ZEO.cache.Entry`` objects, and a minor one involving
  empty lists.

Subtransactions are deprecated
------------------------------

- (3.5a4) Subtransactions are deprecated, and will be removed in ZODB 3.7.
  Use savepoints instead.  Savepoints are more powerful, and code using
  subtransactions does not mix well with code using savepoints (a
  subtransaction commit forces all current savepoints to become unusable, so
  code using subtransactions can hurt newer code trying to use savepoints).
  In general, a subtransaction commit done just to free memory can be changed
  from::

      transaction.commit(1)

  to::

      transaction.savepoint(True)

  That is, make a savepoint, and forget it.  As shown, it's best to pass
  ``True`` for the optional ``optimistic`` argument in this case:  because
  there's no possibility of asking for a rollback later, there's no need
  to insist that all data managers support rollback.

  In rarer cases, a subtransaction commit is followed later by a
  subtransaction abort.  In that case, change the initial::

      transaction.commit(1)

  to::

      sp = transaction.savepoint()

  and in place of the subtransaction abort::

      transaction.abort(1)

  roll back the savepoint instead::

      sp.rollback()

- (3.5a4) Internal uses of subtransactions (transaction ``commit()`` or
  ``abort()`` passing a true argument) were rewritten to use savepoints
  instead.

Multi-database
--------------

- (3.5a1) Preliminary support for persistent cross-database references has
  been added.  See ``ZODB/cross-database-references.txt`` for an
  introduction.

Tools
-----

- (3.5a6, 3.5a7) Collector #1847.  The ZEO client cache tracing and simulation
  tools weren't updated to work with ZODB 3.3, and the introduction of
  MVCC required major reworking of the tracing and simulation code.  These
  tools are in a working state again, although so far lightly tested on
  just a few applications.  In ``doc/ZEO/``, see the heavily revised
  ``trace.txt`` and ``cache.txt``.

- (3.5a5) Collector #1846:  If an uncommitted transaction was found,
  fsrecover.py fell into an infinite loop.

Windows
-------

- (3.5a6) As developed in a long thread starting at
  http://mail.zope.org/pipermail/zope/2005-July/160433.html
  there appears to be a race bug in the Microsoft Windows socket
  implementation, rarely visible in ZEO when multiple processes try to
  create an "asyncore trigger" simultaneously.  Windows-specific code in
  ``ZEO/zrpc/trigger.py`` changed to work around this bug when it occurs.

ThreadedAsync.LoopCallback
--------------------------

- (3.5a5) This once again physically replaces Python's ``asyncore.loop``
  function with its own loop function, because it turns out Zope relied on
  the seemingly unused ``LoopCallback.exit_status`` global, which was
  removed in the change described below.  Python's ``asyncore.loop`` is again
  not invoked, so any breakpoints or debugging prints added to that are again
  "lost".

- (3.5a4) This replaces Python's ``asyncore.loop`` function with its own, in
  order to get notified when ``loop()`` is first called.  The signature of
  ``asyncore.loop`` changed in Python 2.4, but ``LoopCallback.loop``'s
  signature didn't change to match.  The code here was repaired to be
  compatible with both old and new signatures, and also repaired to invoke
  Python's ``asyncore.loop()`` instead of replacing it entirely (so, for
  example, debugging prints added to Python's ``asyncore.loop`` won't be
  lost anymore).


FileStorage
-----------

- (3.5a4) Collector #1830.  In some error cases when reading a FileStorage
  index, the code referenced an undefined global.

- (3.5a4) Collector #1822.  The ``undoLog()`` and ``undoInfo()`` methods
  were changed in 3.4a9 to return the documented results.  Alas, some pieces
  of (non-ZODB) code relied on the actual behavior.  When the ``first`` and
  ``last`` arguments are both >= 0, these methods now treat them as if they
  were Python slice indices, including the `first` index but excluding the
  ``last`` index.  This matches former behavior, although it contradicts older
  ZODB UML documentation.  The documentation in
  ``ZODB.interfaces.IStorageUndoable`` was changed to match the new intent.

- (3.5a2) The ``_readnext()`` method now returns the transaction size as
  the value of the "size" key.  Thanks to Dieter Maurer for the patch, from
  http://mail.zope.org/pipermail/zodb-dev/2003-October/006157.html. "This is
  very valuable when you want to spot strange transaction sizes via Zope's
  'Undo' tab".

BTrees
------

- (3.5.a5) Collector 1843.  When a non-integer was passed to a method like
  ``keys()`` of a Bucket or Set with integer keys, an internal error code
  was overlooked, leading to everything from "delayed errors" to segfaults.
  Such cases raise TypeError now, as intended.

- (3.5a4) Collector 1831.  The BTree ``minKey()`` and ``maxKey()`` methods
  gave a misleading message if no key satisfying the constraints existed in a
  non-empty tree.

- (3.5a4) Collector 1829.  Clarified that the ``minKey()`` and ``maxKey()``
  methods raise an exception if no key exists satsifying the constraints.

- (3.5a4) The ancient ``convert.py`` script was removed.  It was intended to
  convert "old" BTrees to "new" BTrees, but the "old" BTree implementation
  was removed from ZODB years ago.


What's new in ZODB3 3.4.1?
==========================
Release date: 09-Aug-2005

Following are dates of internal releases (to support ongoing Zope 2
development) since ZODB 3.4's last public release:

- 3.4.1b5 08-Aug-2005
- 3.4.1b4 07-Aug-2005
- 3.4.1b3 04-Aug-2005
- 3.4.1b2 02-Aug-2005
- 3.4.1b1 26-Jul-2005
- 3.4.1a6 19-Jul-2005
- 3.4.1a5 12-Jul-2005
- 3.4.1a4 08-Jul-2005
- 3.4.1a3 02-Jul-2005
- 3.4.1a2 29-Jun-2005
- 3.4.1a1 27-Jun-2005

Savepoints
----------

- (3.4.1a1) When a savepoint is made, the states of objects modified so far
  are saved to a temporary storage (an instance of class ``TmpStore``,
  although that's an internal implementation detail).  That storage needs
  to implement the full storage API too, but was missing the ``loadBefore()``
  method needed for MVCC to retrieve non-current revisions of objects.  This
  could cause spurious errors if a transaction with a pending savepoint
  needed to fetch an older revision of some object.

- (3.4.1a5) The ``ISavepoint`` interface docs said you could roll back to a
  given savepoint any number of times (until the transaction ends, or until
  you roll back to an earlier savepoint's state), but the implementation
  marked a savepoint as invalid after its first use.  The implementation has
  been repaired, to match the docs.

- (3.4.1b4) Collector 1860:  use an optimistic savepoint in ExportImport
  (there's no possiblity of rollback here, so no need to insist that the
  data manager support rollbacks).

ZEO client cache
----------------

- (3.4.1b3) Two memory leaks in the ZEO client cache were repaired, a
  major one involving ``ZEO.cache.Entry`` objects, and a minor one involving
  empty lists.

Subtransactions
---------------

- (3.4.1a5) Internal uses of subtransactions (transaction ``commit()`` or
  ``abort()`` passing a true argument) were rewritten to use savepoints
  instead.  Application code is strongly encouraged to do this too:
  subtransactions are weaker, will be deprecated soon, and do not mix well
  with savepoints (when you do a subtransaction commit, all current
  savepoints are made unusable).  In general, a subtransaction commit
  done just to free memory can be changed from::

      transaction.commit(1)

  to::

      transaction.savepoint(True)

  That is, make a savepoint, and forget it.  As shown, it's best to pass
  ``True`` for the optional ``optimistic`` argument in this case:  because
  there's no possibility of asking for a rollback later, there's no need
  to insist that all data managers support rollback.

  In rarer cases, a subtransaction commit is followed later by a
  subtransaction abort.  In that case, change the initial::

      transaction.commit(1)

  to::

      sp = transaction.savepoint()

  and in place of the subtransaction abort::

      transaction.abort(1)

  roll back the savepoint instead::

      sp.rollback()

FileStorage
-----------

- (3.4.1a3) Collector #1830.  In some error cases when reading a FileStorage
  index, the code referenced an undefined global.

- (3.4.1a2) Collector #1822.  The ``undoLog()`` and ``undoInfo()`` methods
  were changed in 3.4a9 to return the documented results.  Alas, some pieces
  of (non-ZODB) code relied on the actual behavior.  When the `first` and
  `last` arguments are both >= 0, these methods now treat them as if they
  were Python slice indices, including the `first` index but excluding the
  `last` index.  This matches former behavior, although it contradicts older
  ZODB UML documentation.  The documentation in
  ``ZODB.interfaces.IStorageUndoable`` was changed to match the new intent.

- (3.4.1a1) The ``UndoSearch._readnext()`` method now returns the transaction
  size as the value of the "size" key.  Thanks to Dieter Maurer for the
  patch, from
  http://mail.zope.org/pipermail/zodb-dev/2003-October/006157.html. "This is
  very valuable when you want to spot strange transaction sizes via Zope's
  'Undo' tab".

ThreadedAsync.LoopCallback
--------------------------

- (3.4.1a6) This once again physically replaces Python's ``asyncore.loop``
  function with its own loop function, because it turns out Zope relied on
  the seemingly unused ``LoopCallback.exit_status`` global, which was
  removed in the change described below.  Python's ``asyncore.loop`` is again
  not invoked, so any breakpoints or debugging prints added to that are again
  "lost".

- (3.4.1a1) This replaces Python's ``asyncore.loop`` function with its own,
  in order to get notified when ``loop()`` is first called.  The signature of
  ``asyncore.loop`` changed in Python 2.4, but ``LoopCallback.loop``'s
  signature didn't change to match.  The code here was repaired to be
  compatible with both old and new signatures, and also repaired to invoke
  Python's ``asyncore.loop()`` instead of replacing it entirely (so, for
  example, debugging prints added to Python's ``asyncore.loop`` won't be lost
  anymore).

Windows
-------

- (3.4.1b2) As developed in a long thread starting at
  http://mail.zope.org/pipermail/zope/2005-July/160433.html
  there appears to be a race bug in the Microsoft Windows socket
  implementation, rarely visible in ZEO when multiple processes try to
  create an "asyncore trigger" simultaneously.  Windows-specific code in
  ``ZEO/zrpc/trigger.py`` changed to work around this bug when it occurs.


Tools
-----

- (3.4.1b1 thru 3.4.1b5) Collector #1847.  The ZEO client cache tracing and
  simulation tools weren't updated to work with ZODB 3.3, and the
  introduction of MVCC required major reworking of the tracing and simulation
  code.  These tools are in a working state again, although so far lightly
  tested on just a few applications.  In ``doc/ZEO/``, see the heavily revised
  ``trace.txt`` and ``cache.txt``.

- (3.4.1a6) Collector #1846:  If an uncommitted transaction was found,
  fsrecover.py fell into an infinite loop.


DemoStorage
-----------

- (3.4.1a1) The implementation of ``undoLog()`` was wrong in several ways;
  repaired.

BTrees
------

- (3.4.1a6) Collector 1843.  When a non-integer was passed to a method like
  ``keys()`` of a Bucket or Set with integer keys, an internal error code
  was overlooked, leading to everything from "delayed errors" to segfaults.
  Such cases raise TypeError now, as intended.

- (3.4.1a4) Collector 1831.  The BTree ``minKey()`` and ``maxKey()`` methods
  gave a misleading message if no key satisfying the constraints existed in a
  non-empty tree.

- (3.4.1a3) Collector 1829.  Clarified that the ``minKey()`` and ``maxKey()``
  methods raise an exception if no key exists satsifying the constraints.


What's new in ZODB3 3.4?
========================
Release date: 09-Jun-2005

Following is combined news from the "internal releases" (to support
ongoing Zope 2.8 and Zope3 development) since the last public ZODB 3.4
release.  These are the dates of the internal releases:

- 3.4c2 06-Jun-2005
- 3.4c1 03-Jun-2005
- 3.4b3 27-May-2005
- 3.4b2 26-May-2005

Connection, DB
--------------

- (3.4b3) ``.transaction_manager`` is now a public attribute of
  IDataManager, and is the instance of ITransactionManager used by the
  data manager as its transaction manager.  There was previously no way
  to ask a data manager which transaction manager it was using.  It's
  intended that ``transaction_manager`` be treated as read-only.

- (3.4b3) For sanity, the ``txn_mgr`` argument to ``DB.open()``,
  ``Connection.__init__()``, and ``Connection._setDB()`` has been renamed
  to ``transaction_manager``.  ``txn_mgr`` is still accepted, but is
  deprecated and will be removed in ZODB 3.6.  Any code that was using
  the private ``._txn_mgr`` attribute of ``Connection`` will break
  immediately.

Development
-----------

- (3.4b2) ZODB's ``test.py`` is now a small driver for the shared
  ``zope.testing.testrunner``.  See the latter's documentation
  for command-line arguments.

Error reporting
---------------

- (3.4c1) In the unlikely event that ``referencesf()`` reports an unpickling
  error (for example, a corrupt database can cause this), the message it
  produces no longer contains unprintable characters.

Tests
-----

- (3.4c2) ``checkCrossDBInvalidations`` suffered spurious failures too often
  on slow and/or busy machines.  The test is willing to wait longer for
  success now.


What's new in ZODB3 3.4b1?
==========================
Release date: 19-May-2005

What follows is combined news from the "internal releases" (to support
ongoing Zope 2.8 and Zope3 development) since the last public ZODB 3.4
release.  These are the dates of the internal releases:

- 3.4b1 19-May-2005
- 3.4a9 12-May-2005
- 3.4a8 09-May-2005
- 3.4a7 06-May-2005
- 3.4a6 05-May-2005
- 3.4a5 25-Apr-2005
- 3.4a4 23-Apr-2005
- 3.4a3 13-Apr-2005
- 3.4a2 03-Apr-2005


transaction
-----------

- (3.4a7) If the first activity seen by a new ``ThreadTransactionManager`` was
  an explicit ``begin()`` call, then synchronizers registered after that (but
  still during the first transaction) were not communicated to the
  transaction object.  As a result, the ``afterCompletion()`` methods of
  registered synchronizers weren't called when the first transaction ended.

- (3.4a6) Doing a subtransaction commit erroneously processed invalidations,
  which could lead to an inconsistent view of the database.  For example, let
  T be the transaction of which the subtransaction commit was a part.  If T
  read a persistent object O's state before the subtransaction commit, did not
  commit new state of its own for O during its subtransaction commit, and O
  was modified before the subtransaction commit by a different transaction,
  then the subtransaction commit processed an invalidation for O, and the
  state T read for O originally was discarded in T.  If T went on to access O
  again, it saw the newly committed (by a different transaction) state for O::

      o_attr = O.some_attribute
      get_transaction().commit(True)
      assert o_attr == O.some_attribute

  could fail, and despite that T never modifed O.

- (3.4a4) Transactions now support savepoints.  Savepoints allow changes to be
  periodically checkpointed within a transaction.  You can then rollback to a
  previously created savepoint.  See ``transaction/savepoint.txt``.

- (3.4a6) A ``getBeforeCommitHooks()`` method was added.  It returns an
  iterable producing the registered beforeCommit hooks.

- (3.4a6) The ``ISynchronizer`` interface has a new ``newTransaction()``
  method. This is invoked whenever a transaction manager's ``begin()`` method
  is called.  (Note that a transaction object's (as opposed to a transaction
  manager's) ``begin()`` method is deprecated, and ``newTransaction()`` is
  not called when using the deprecated method.)

- (3.4a6) Relatedly, ``Connection`` implements ``ISynchronizer``, and
  ``Connection``'s ``afterCompletion()`` and ``newTransaction()`` methods now
  call ``sync()`` on the underlying storage (if the underlying storage has
  such a method), in addition to processing invalidations.  The practical
  implication is that storage synchronization will be done automatically now,
  whenever a transaction is explicitly started, and after top-level
  transaction commit or abort.  As a result, ``Connection.sync()`` should
  virtually never be needed anymore, and will eventually be deprecated.

- (3.4a3) Transaction objects have a new method, ``beforeCommitHook(hook,
  *args, **kws)``.  Hook functions registered with a transaction are called
  at the start of a top-level commit, before any of the work is begun, so a
  hook function can perform any database operations it likes.  See
  ``test_beforeCommitHook()`` in ``transaction/tests/test_transaction.py``
  for a tutorial doctest, and the ``ITransaction`` interface for details.
  Thanks to Florent Guillaume for contributing code and tests.

- (3.4a3) Clarifications were made to transaction interfaces.

Support for ZODB4 savepoint-aware data managers has been dropped
----------------------------------------------------------------

- (3.4a4) In adding savepoint support, we dropped the attempted support for
  ZODB4 data managers that support savepoints.  We don't think that this will
  affect anyone.

ZEO
---

- (3.4a4) The ZODB and ZEO version numbers are now the same.  Concretely::

      import ZODB, ZEO
      assert ZODB.__version__ == ZEO.version

  no longer fails.  If interested, see the README file for details about
  earlier version numbering schemes.

- (3.4b1) ZConfig version 2.3 adds new socket address types, for smoother
  default behavior across platforms.  The hostname portion of
  socket-binding-address defaults to an empty string, which acts like
  INADDR_ANY on Windows and Linux (bind to any interface).  The hostname
  portion of socket-connection-address defaults to "127.0.0.1" (aka
  "localhost").  In config files, the types of ``zeo`` section keys
  ``address`` and ``monitor-address`` changed to socket-binding-address,
  and the type of the ``zeoclient`` section key ``server`` changed to
  socket-connection-address.

- (3.4a4) The default logging setup in ``runzeo.py`` was broken.  It was
  changed so that running ``runzeo.py`` from a command line now, and without
  using a config file, prints output to the console much as ZODB 3.2 did.

ZEO on Windows
--------------

Thanks to Mark Hammond for these ``runzeo.py`` enhancements on Windows:

- (3.4b1) Collector 1788:  Repair one of the new features below.

- (3.4a4) A pid file (containing the process id as a decimal string) is
  created now for a ZEO server started via ``runzeo.py``.  External programs
  can read the pid from this file and derive a "signal name" used in a new
  signal-emulation scheme for Windows.  This is only necessary on Windows,
  but the pid file is created on all platforms that implement
  ``os.getpid()``, as long as the ``pid-filename`` option is set, or
  environment variable ``INSTANCE_HOME`` is defined.  The ``pid-filename``
  option can be set in a ZEO config file, or passed as the new ``--pid-file``
  argument to ``runzeo.py``.

- (3.4a4) If available, ``runzeo.py`` now uses Zope's new 'Signal' mechanism
  for Windows, to implement clean shutdown and log rotation handlers for
  Windows. Note that the Python in use on the ZEO server must also have the
  Python Win32 extensions installed for this to be useful.

Tools
-----

- (3.4a4) ``fsdump.py`` now displays the size (in bytes) of data records.
  This actually went in several months go, but wasn't noted here at the time.
  Thanks to Dmitry Vasiliev for contributing code and tests.

FileStorage
-----------

- (3.4a9) The ``undoLog()`` and ``undoInfo()`` methods almost always returned
  a wrong number of results, one too many if ``last < 0`` (the default is
  such a case), or one too few if ``last >= 0``.  These have been repaired,
  new tests were added, and these methods are now documented in
  ``ZODB.interfaces.IStorageUndoable``.

- (3.4a2) A ``pdb.set_trace()`` call was mistakenly left in method
  ``FileStorage.modifiedInVersion()``.

ZConfig
-------

- (3.4b1) The "standalone" release of ZODB now includes ZConfig version 2.3.

DemoStorage
-----------

- (3.4a4) Appropriate implementations of the storage API's ``registerDB()``
  and ``new_oid()`` methods were added, delegating to the base storage.  This
  was needed to support wrapping a ZEO client storage as a ``DemoStorage``
  base storage, as some new Zope tests want to do.

BaseStorage
-----------

- (3.4a4) ``new_oid()``'s undocumented ``last=`` argument was removed.  It
  was used only for internal recursion, and injured code sanity elsewhere
  because not all storages included it in their ``new_oid()``'s signature.
  Straightening this out required adding ``last=`` everywhere, or removing it
  everywhere. Since recursion isn't actually needed, and there was no other
  use for ``last=``, removing it everywhere was the obvious choice.

Tests
-----

- (3.4a3) The various flavors of the ``check2ZODBThreads`` and
  ``check7ZODBThreads`` tests are much less likely to suffer sproadic
  failures now.

- (3.4a2) The test ``checkOldStyleRoot`` failed in Zope3, because of an
  obscure dependence on the ``Persistence`` package (which Zope3 doesn't use).

ZApplication
------------

- (3.4a8) The file ``ZApplication.py`` was moved, from ZODB to Zope(2).  ZODB
  and Zope3 don't use it, but Zope2 does.

- (3.4a7) The ``__call__`` method didn't work if a non-None ``connection``
  string argument was passed.  Thanks to Stefan Holek for noticing.


What's new in ZODB3 3.4a1?
==========================
Release date: 01-Apr-2005

transaction
-----------

- ``get_transaction()`` is officially deprecated now, and will be removed
  in ZODB 3.6.  Use the ``transaction`` package instead.   For example,
  instead of::

      import ZODB
      ...
      get_transaction().commit()

  do::

      import transaction
      ...
      transaction.commit()

DB
--

- There is no longer a hard limit on the number of connections that
  ``DB.open()`` will create.  In other words, ``DB.open()`` never blocks
  anymore waiting for an earlier connection to close, and ``DB.open()``
  always returns a connection now (while it wasn't documented, it was
  possible for ``DB.open()`` to return ``None`` before).

  ``pool_size`` continues to default to 7, but its meaning has changed:
  if more than ``pool_size`` connections are obtained from ``DB.open()``
  and not closed, a warning is logged; if more than twice ``pool_size``, a
  critical problem is logged.  ``pool_size`` should be set to the maximum
  number of connections from the ``DB`` instance you expect to have open
  simultaneously.

  In addition, if a connection obtained from ``DB.open()`` becomes
  unreachable without having been explicitly closed, when Python's garbage
  collection reclaims that connection it no longer counts against the
  ``pool_size`` thresholds for logging messages.

  The following optional arguments to ``DB.open()`` are deprecated:
  ``transaction``, ``waitflag``, ``force`` and ``temporary``.  If one
  is specified, its value is ignored, and ``DeprecationWarning`` is
  raised.  In ZODB 3.6, these optional arguments will be removed.

- Lightweight support for "multi-databases" is implemented.  These are
  collections of named DB objects and associated open Connections, such
  that the Connection for any DB in the collection can be obtained from
  a Connection from any other DB in the collection.  See the new test
  file ZODB/tests/multidb.txt for a tutorial doctest.  Thanks to Christian
  Theune for his work on this during the PyCon 2005 ZODB sprint.

ZEO compatibility
-----------------

There are severe restrictions on using ZEO servers and clients at or after
ZODB 3.3 with ZEO servers and clients from ZODB versions before 3.3.  See the
reworked ``Compatibility`` section in ``README.txt`` for details.  If
possible, it will be easiest to move clients and servers to 3.3+
simultaneously.  With care, it's possible to use a 3.3+ ZEO server with
pre-3.3 ZEO clients, but not possible to use a pre-3.3 ZEO server with 3.3+
ZEO clients.

BTrees
------

- A new family of BTree types, in the ``IFBTree`` module, map
  signed integers (32 bits) to C floats (also 32 bits).  The
  intended use is to help construct search indices, where, e.g.,
  integer word or document identifiers map to scores of some
  kind.  This is easier than trying to work with scaled integer
  scores in an ``IIBTree``, and Zope3 has moved to ``IFBTrees``
  for these purposes in its search code.

FileStorage
-----------

- Addded a record iteration protocol to FileStorage.  You can use the
  record iterator to iterate over all current revisions of data
  pickles in the storage.

  In order to support calling via ZEO, we don't implement this as an
  actual iterator.  An example of using the record iterator protocol
  is as follows::

      storage = FileStorage('anexisting.fs')
      next_oid = None
      while True:
          oid, tid, data, next_oid = storage.record_iternext(next_oid)
          # do something with oid, tid and data
          if next_oid is None:
              break

  The behavior of the iteration protocol is now to iterate over all
  current records in the database in ascending oid order, although
  this is not a promise to do so in the future.


Tools
-----

New tool fsoids.py, for heavy debugging of FileStorages; shows all
uses of specified oids in the entire database (e.g., suppose oid 0x345620
is missing -- did it ever exist?  if so, when?  who referenced it?  when
was the last transaction that modified an object that referenced it?
which objects did it reference?  what kind of object was it?).
ZODB/test/testfsoids.py is a tutorial doctest.


fsIndex
-------

Efficient, general implementations of ``minKey()`` and ``maxKey()`` methods
were added.  ``fsIndex`` is a special hybrid kind of BTree used to implement
FileStorage indices.  Thanks to Chris McDonough for code and tests.


What's new in ZODB3 3.3.1?
==========================
Release date: DD-MMM-2005

Tests
-----

The various flavors of the ``check2ZODBThreads`` and ``check7ZODBThreads``
tests are much less likely to suffer sproadic failures now.


What's new in ZODB3 3.3.1c1?
============================
Release date: 01-Apr-2005

BTrees
------

Collector #1734: BTrees conflict resolution leads to index inconsistencies.

Silent data loss could occur due to BTree conflict resolution when one
transaction T1 added a new key to a BTree containing at least three buckets,
and a concurrent transaction T2 deleted all keys in the bucket to which the
new key was added.  Conflict resolution then created a bucket containing the
newly added key, but the bucket remained isolated, disconnected from the
BTree. In other words, the committed BTree didn't contain the new key added by
T1.  Conflict resolution doesn't have enough information to repair this,
so ``ConflictError`` is now raised in such cases.


ZEO
---

Repaired subtle race conditions in establishing ZEO connections, both client-
and server-side.  These account for intermittent cases where ZEO failed
to make a connection (or reconnection), accompanied by a log message showing
an error caught in ``asyncore`` and having a traceback ending with:

    ``UnpicklingError: invalid load key, 'Z'.``

or:

    ``ZRPCError: bad handshake '(K\x00K\x00U\x0fgetAuthProtocol)t.'``

or:

    ``error: (9, 'Bad file descriptor')``

or an ``AttributeError``.

These were exacerbated when running the test suite, because of an unintended
busy loop in the test scaffolding, which could starve the thread trying to
make a connection.  The ZEO reconnection tests may run much faster now,
depending on platform, and should suffer far fewer (if any) intermittent
"timed out waiting for storage to connect" failures.

ZEO protocol and compatibility
------------------------------

ZODB 3.3 introduced multiversion concurrency control (MVCC), which required
changes to the ZEO protocol.  The first 3.3 release should have increased
the internal ZEO protocol version number (used by ZEO protocol negotiation
when a client connects), but neglected to.  This has been repaired.

Compatibility between pre-3.3 and post-3.3 ZEO clients and servers remains
very limited.  See the newly updated ``Compatibility`` section in
``README.txt`` for details.

FileStorage
-----------

- The ``.store()`` and ``.restore()`` methods didn't update the storage's
  belief about the largest oid in use when passed an oid larger than the
  largest oid the storage already knew about.  Because ``.restore()`` in
  particular is used  by ``copyTransactionsFrom()``, and by the first stage
  of ZRS recovery, a large database could be created that believed the only
  oid in use was oid 0 (the special oid reserved for the root object).  In
  rare cases, it could go on from there assigning duplicate oids to new
  objects, starting over from oid 1 again.  This has been repaired.  A
  new ``set_max_oid()`` method was added to the ``BaseStorage`` class so
  that derived storages can update the largest oid in use in a threadsafe
  way.

- A FileStorage's index file tried to maintain the index's largest oid as a
  separate piece of data, incrementally updated over the storage's lifetime.
  This scheme was more complicated than necessary, so was also more brittle
  and slower than necessary.  It indirectly participated in a rare but
  critical bug:  when a FileStorage was created via
  ``copyTransactionsFrom()``, the "maximum oid" saved in the index file was
  always 0.  Use that FileStorage, and it could then create "new" oids
  starting over at 0 again, despite that those oids were already in use by
  old objects in the database.  Packing a FileStorage has no reason to
  try to update the maximum oid in the index file either, so this kind of
  damage could (and did) persist even across packing.

  The index file's maximum-oid data is ignored now, but is still written
  out so that ``.index`` files can be read by older versions of ZODB.
  Finding the true maximum oid is done now by exploiting that the main
  index is really a kind of BTree (long ago, this wasn't true), and finding
  the largest key in a BTree is inexpensive.

- A FileStorage's index file could be updated on disk even if the storage
  was opened in read-only mode.  That bug has been repaired.

- An efficient ``maxKey()`` implementation was added to class ``fsIndex``.


Pickle (in-memory Connection) Cache
-----------------------------------

You probably never saw this exception:

    ``ValueError: Can not re-register object under a different oid``

It's been changed to say what it meant:

    ``ValueError: A different object already has the same oid``

This happens if an attempt is made to add distinct objects to the cache
that have the same oid (object identifier).  ZODB should never do this,
but it's possible for application code to force such an attempt.

PersistentMapping and PersistentList
------------------------------------

Backward compatibility code has been added so that the sanest of the
ZODB 3.2 dotted paths for ``PersistentMapping`` and ``PersistentList``
resolve.  These are still preferred:

- ``from persistent.list import PersistentList``
- ``from persistent.mapping import PersistentMapping``

but these work again too:

- ``from ZODB.PersistentList import PersistentList``
- ``from ZODB.PersistentMapping import PersistentMapping``

BTrees
------

The BTrees interface file neglected to document the optional
``excludemin`` and ``excludemax`` arguments to the ``keys()``, ``values()``
and ``items()`` methods.  Appropriate changes were merged in from the
ZODB4 BTrees interface file.

Tools
-----

- ``mkzeoinst.py``'s default port number changed from to 9999 to 8100, to
  match the example in Zope's ``zope.conf``.

fsIndex
-------

An efficient ``maxKey()`` method was implemented for the ``fsIndex`` class.
This makes it possible to determine the largest oid in a ``FileStorage``
index efficiently, directly, and reliably, replacing a more delicate scheme
that tried to keep track of this by saving an oid high water mark in the
index file and incrementally updating it.


What's new in ZODB3 3.3.1a1?
============================
Release date: 11-Jan-2005

ZEO client cache
----------------

- Collector 1536:  The ``cache-size`` configuration option for ZEO clients
  was being ignored.  Worse, the client cache size was only one megabyte,
  much smaller than the advertised default of 20MB.  Note that the default
  is carried over from a time when gigabyte disks were expensive and rare;
  20MB is also too small on most modern machines.

- Fixed a nasty bug in cache verification.  A persistent ZEO cache uses a
  disk file, and, when active, has some in-memory data structures too to
  speed operation.  Invalidations processed as part of startup cache
  verification were reflected in the in-memory data structures, but not
  correctly in the disk file.  So if an object revision was invalidated as
  part of verification, the object wasn't loaded again before the connection
  was closed, and the object revision remained in the cache file until the
  connection was closed, then the next time the cache file was opened it
  could believe that the stale object revision in the file was actually
  current.

- Fixed a bug wherein an object removed from the client cache didn't
  properly mark the file slice it occupied as being available for reuse.

ZEO
---

Collector 1503:  excessive logging.  It was possible for a ZEO client to
log "waiting for cache verification to finish" messages at a very high
rate, producing gigabytes of such messages in short order.
``ClientStorage._wait_sync()`` was changed to log no more than one
such message per 5 minutes.

persistent
----------

Collector #1350:  ZODB has a default one-thread-per-connection model, and
two threads should never do operations on a single connection
simultaneously.  However, ZODB can't detect violations, and this happened
in an early stage of Zope 2.8 development.  The low-level ``ghostify()``
and ``unghostify()`` routines in ``cPerisistence.c`` were changed to give
some help in detecting this when it happens.  In a debug build, both abort
the process if thread interference is detected.  This is extreme, but
impossible to overlook.  In a release build, ``unghostify()`` raises
``SystemError`` if thread damage is detected; ``ghostify()`` ignores the
problem in a release build (``ghostify()`` is supposed to be so simple that
it "can't fail").

ConflictError
-------------

New in 3.3, a ``ConflictError`` exception may attempt to insert the path to
the object's class in its message.  However, a ZEO server may not have
access to application class implementations, and then the attempt by the
server to raise ``ConflictError`` could raise ``ImportError`` instead while
trying to determine the object's class path.  This was confusing.  The code
has been changed to obtain the class path from the object's pickle, without
trying to import application modules or classes.

FileStorage
-----------

Collector 1581:  When an attempt to pack a corrupted ``Data.fs`` file was
made, it was possible for the pack routine to die with a reference to an
undefined global while it was trying to raise ``CorruptedError``.  It
raises ``CorruptedError``, as it always intended, in these cases now.

Install
-------

The C header file ``ring.h`` is now installed.

Tools
-----

- ``BTrees.check.display()`` now displays the oids (if any) of the
  BTree's or TreeSet's constituent objects.


What's new in ZODB3 3.3?
========================
Release date: 06-Oct-2004

ZEO
---

The encoding of RPC calls between server and client was being done
with protocol 0 ("text mode") pickles, which could require sending
four times as many bytes as necessary.  Protocol 1 pickles are used
now.  Thanks to Andreas Jung for the diagnosis and cure.

ZODB/component.xml
------------------

``cache-size`` parameters were changed from type ``integer`` to
type ``byte-size``.  This allows you to specify, for example,
"``cache-size 20MB``" to get a 20 megabyte cache.

transaction
-----------

The deprecation warning for ``Transaction.begin()`` was changed to
point to the caller, instead of to ``Transaction.begin()`` itself.

Connection
----------

Restored Connection's private ``_opened`` attribute.  This was still
referenced by ``DB.connectionDebugInfo()``, and Zope 2 calls the latter.

FileStorage
-----------

Collector #1517: History tab for ZPT does not work. ``FileStorage.history()``
was reading the user, description, and extension fields out of the object
pickle, due to starting the read at a wrong location.  Looked like
cut-and-paste repetition of the same bug in ``FileStorage.FileIterator``
noted in the news for 3.3c1.

What's new in ZODB3 3.3 release candidate 1?
============================================
Release date: 14-Sep-2004

Connection
----------

ZODB intends to raise ``ConnnectionStateError`` if an attempt is made to
close a connection while modifications are pending (the connection is
involved in a transaction that hasn't been ``abort()``'ed or
``commit()``'ed).  It was missing the case where the only pending
modifications were made in subtransactions.  This has been fixed.  If an
attempt to close a connection with pending subtransactions is made now::

    ConnnectionStateError: Cannot close a connection with a pending subtransaction

is raised.

transaction
-----------

- Transactions have new, backward-incompatible behavior in one respect:
  if a ``Transaction.commit()``, ``Transaction.commit(False)``, or
  ``Transaction.commit(True)`` raised an exception, prior behavior was that
  the transaction effectively aborted, and a new transaction began.
  A primary bad consequence was that, if in a sequence of subtransaction
  commits, one of the commits failed but the exception was suppressed,
  all changes up to and including the failing commit were lost, but
  later subtransaction commits in the sequence got no indication that
  something had gone wrong, nor did the final (top level) commit.  This
  could easily lead to inconsistent data being committed, from the
  application's point of view.

  The new behavior is that a failing commit "sticks" until explicitly
  cleared.  Now if an exception is raised by a ``commit()`` call (whether
  subtransaction or top level) on a Transaction object ``T``:

    - Pending changes are aborted, exactly as they were for a failing
      commit before.

    - But ``T`` remains the current transaction object (if ``tm`` is ``T``'s
      transaction manger, ``tm.get()`` continues to return ``T``).

    - All subsequent attempts to do ``T.commit()``, ``T.join()``, or
      ``T.register()`` raise the new ``TransactionFailedError`` exception.
      Note that if you try to modify a persistent object, that object's
      resource manager (usually a ``Connection`` object) will attempt to
      ``join()`` the failed transaction, and ``TransactionFailedError``
      will be raised right away.

  So after a transaction or subtransaction commit fails, that must be
  explicitly cleared now, either by invoking ``abort()`` on the transaction
  object, or by invoking ``begin()`` on its transaction manager.

- Some explanations of new transaction features in the 3.3a3 news
  were incorrect, and this news file has been retroactively edited to
  repair that.  See news for 3.3a3 below.

- If ReadConflictError was raised by an attempt to load an object with a
  ``_p_independent()`` method that returned false, attempting to commit the
  transaction failed to (re)raise ReadConflictError for that object.  Note
  that ZODB intends to prevent committing a transaction in which a
  ReadConflictError occurred; this was an obscure case it missed.

- Growing pains:  ZODB 3.2 had a bug wherein ``Transaction.begin()`` didn't
  abort the current transaction if the only pending changes were in a
  subtransaction.  In ZODB 3.3, it's intended that a transaction manager be
  used to effect ``begin()`` (instead of invoking ``Transaction.begin()``),
  and calling ``begin()`` on a transaction manager didn't have this old
  bug.  However, ``Transaction.begin()`` still exists in 3.3, and it had a
  worse bug:  it never aborted the transaction (not even if changes were
  pending outside of subtransactions). ``Transaction.begin()`` has been
  changed to abort the transaction. ``Transaction.begin()`` is also
  deprecated.  Don't use it.  Use ``begin()`` on the relevant transaction
  manager instead.  For example,

      >>> import transaction
      >>> txn = transaction.begin()  # start a txn using the default TM

  if using the default ``ThreadTransactionManager`` (see news for 3.3a3
  below). In 3.3, it's intended that a single ``Transaction`` object is
  used for exactly one transaction.  So, unlike as in 3.2, when somtimes
  ``Transaction`` objects were reused across transactions, but sometimes
  weren't, when you do ``Transaction.begin()`` in 3.3 a brand new
  transaction object is created.  That's why this use is deprecated.  Code
  of the form:

      >>> txn = transaction.get()
      >>> ...
      >>> txn.begin()
      >>> ...
      >>> txn.commit()

  can't work as intended in 3.3, because ``txn`` is no longer the current
  ``Transaction`` object the instant ``txn.begin()`` returns.

BTrees
------

The BTrees __init__.py file is now just a comment.  It had been trying
to set up support for (long gone) "int sets", and to import an old
version of Zope's Interface package, which doesn't even ship with ZODB.
The latter in particular created problems, at least clashing with
PythonCAD's Interface package.

POSException
------------

Collector #1488 (TemporaryStorage -- going backward in time).  This
confusion was really due to that the detail on a ConflictError exception
didn't make sense.  It called the current revision "was", and the old
revision "now".  The detail is much more informative now.  For example,
if the exception said::

    ConflictError: database conflict error (oid 0xcb22,
    serial was 0x03441422948b4399, now 0x034414228c3728d5)

before, it now says::

    ConflictError: database conflict error (oid 0xcb22,
    serial this txn started with 0x034414228c3728d5 2002-04-14 20:50:32.863000,
    serial currently committed 0x03441422948b4399 2002-04-14 20:50:34.815000)

ConflictError
-------------

The undocumented ``get_old_serial()`` and ``get_new_serial()`` methods
were swapped (the first returned the new serial, and the second returned
the old serial).

Tools
-----

``FileStorage.FileIterator`` was confused about how to read a transaction's
user and description fields, which caused several tools to display
binary gibberish for these values.

``ZODB.utils.oid_repr()`` changed to add a leading "0x", and to strip
leading zeroes.  This is used, e.g., in the detail of a ``POSKeyError``
exception, to identify the missing oid.  Before, the output was ambiguous.
For example, oid 17 was displayed as 0000000000000011.  As a Python
integer, that's octal 9.  Or was it meant to be decimal 11?  Or was it
meant to be hex? Now it displays as 0x11.

fsrefs.py:

    When run with ``-v``, produced tracebacks for objects whose creation was
    merely undone.  This was confusing.  Tracebacks are now produced only
    if there's "a real" problem loading an oid.

    If the current revision of object O refers to an object P whose
    creation has been undone, this is now identified as a distinct case.

    Captured and ignored most attempts to stop it via Ctrl+C.  Repaired.

    Now makes two passes, so that an accurate report can be given of all
    invalid references.

``analyze.py`` produced spurious "len of unsized object" messages when
finding a data record for an object uncreation or version abort.  These
no longer appear.

``fsdump.py``'s ``get_pickle_metadata()`` function (which is used by several
tools) was confused about what to do when the ZODB pickle started with
a pickle ``GLOBAL`` opcode.  It actually loaded the class then, which it
intends never to do, leading to stray messages on stdout when the class
wasn't available, and leading to a strange return value even when it was
available (the repr of the type object was returned as "the module name",
and an empty string was returned as "the class name").  This has been
repaired.


What's new in ZODB3 3.3 beta 2
==============================
Release date: 13-Aug-2004

Transaction Managers
--------------------

Zope3-dev Collector #139: Memory leak involving buckets and connections

The transaction manager internals effectively made every Connection
object immortal, except for those explicitly closed.  Since typical
practice is not to close connections explicitly (and closing a DB
happens not to close the connections to it -- although that may
change), this caused massive memory leaks when many connections were
opened.  The transaction manager internals were reworked to use weak
references instead, so that connection memory (and other registered
synch objects) now get cleaned up when nothing other than the
transaction manager knows about them.

Storages
--------

Collector #1327: FileStorage init confused by time travel

If the system clock "went backwards" a long time between the times a
FileStorage was closed and reopened, new transaction ids could be
smaller than transaction ids already in the storage, violating a
key invariant.  Now transaction ids are guaranteed to be increasing
even when this happens.  If time appears to have run backwards at all
when a FileStorage is opened, a new message saying so is logged at
warning level; if time appears to have run backwards at least 30
minutes, the message is logged at critical level (and you should
investigate to find and repair the true cause).

Tools
-----

repozo.py:  Thanks to a suggestion from Toby Dickenson, backups
(whether incremental or full) are first written to a temp file now,
which is fsync'ed at the end, and only after that succeeds is the
file renamed to YYYY-MM-DD-HH-MM-SS.ext form.  In case of a system
crash during a repozo backup, this at least makes it much less
likely that a backup file with incomplete or incorrect data will be
left behind.

fsrefs.py:  Fleshed out the module docstring, and repaired a bug
wherein spurious error msgs could be produced after reporting a
problem with an unloadable object.

Test suite
----------

Collector #1397: testTimeStamp fails on FreeBSD

    The BSD distributions are unique in that their mktime()
    implementation usually ignores the input tm_isdst value.  Test
    checkFullTimeStamp() was sensitive to this platform quirk.

Reworked the way some of the ZEO tests use threads, so that unittest is
more likely to notice the real cause of a failure (which usually occurs in
a thread), and less likely to latch on to spurious problems resulting from
the real failure.


What's new in ZODB3 3.3 beta 1
==============================
Release date: 07-Jun-2004

3.3b1 is the first ZODB release built using the new zpkg tools:

    http://zope.org/Members/fdrake/zpkgtools/

This appears to have worked very well.  The structure of the tarball
release differs from previous releases because of it, and the set of
installed files includes some that were not installed in previous
releases.  That shouldn't create problems, so let us know if it does!
We'll fine-tune this for the next release.

BTrees
------

Fixed bug indexing BTreeItems objects with negative indexes.  This
caused reverse iteration to return each item twice.  Thanks to Casey
Duncan for the fix.

ZODB
----

Methods removed from the database (ZODB.DB.DB) class:  cacheStatistics(),
cacheMeanAge(), cacheMeanDeac(), and cacheMeanDeal().  These were
undocumented, untested, and unused.  The first always returned an empty
tuple, and the rest always returned None.

When trying to do recovery to a time earlier than that of the most recent
full backup, repozo.py failed to find the appropriate files, erroneously
claiming "No files in repository before <specified time>".  This has
been repaired.

Collector #1330:  repozo.py -R can create corrupt .fs.
When looking for the backup files needed to recreate a Data.fs file,
repozo could (unintentionally) include its meta .dat files in the list,
or random files of any kind created by the user in the backup directory.
These would then get copied verbatim into the reconstructed file, filling
parts with junk.  Repaired by filtering the file list to include only
files with the data extensions repozo.py creates (.fs, .fsz, .deltafs,
and .deltafsz).  Thanks to James Henderson for the diagnosis.

fsrecover.py couldn't work, because it referenced attributes that no
longer existed after the MVCC changes.  Repaired that, and added new
tests to ensure it continues working.

Collector #1309:  The reference counts reported by DB.cacheExtremeDetails()
for ghosts were one too small.  Thanks to Dieter Maurer for the diagnosis.

Collector #1208:  Infinite loop in cPickleCache.
If a persistent object had a __del__ method (probably not a good idea
regardless, but we don't prevent it) that referenced an attribute of
self, the code to deactivate objects in the cache could get into an
infinite loop:  ghostifying the object could lead to calling its __del__
method, the latter would load the object into cache again to
satsify the attribute reference, the cache would again decide that
the object should be ghostified, and so on.  The infinite loop no longer
occurs, but note that objects of this kind still aren't sensible (they're
effectively immortal).  Thanks to Toby Dickenson for suggesting a nice
cure.


What's new in ZODB3 3.3 alpha 3
===============================
Release date: 16-Apr-2004

transaction
-----------

There is a new transaction package, which provides new interfaces for
application code and for the interaction between transactions and
resource managers.

The top-level transaction package has functions ``commit()``, ``abort()``,
``get()``, and ``begin()``.  They should be used instead of the magic
``get_transaction()`` builtin, which will be deprecated.  For example:

    >>> get_transaction().commit()

should now be written as

    >>> import transaction
    >>> transaction.commit()

The new API provides explicit transaction manager objects.  A transaction
manager (TM) is responsible for associating resource managers with a
"current" transaction.  The default TM, implemented by class
``ThreadedTransactionManager``, assigns each thread its own current
transaction.  This default TM is available as ``transaction.manager``.  The
``TransactionManager`` class assigns all threads to the same transaction,
and is an explicit replacement for the ``Connection.setLocalTransaction()``
method:

A transaction manager instance can be passed as the transaction_manager
argument to ``DB.open()``.  If you do, the connection will use the specified
transaction manager instead of the default TM.  The current transaction is
obtained by calling ``get()`` on a TM. For example:

    >>> tm = transaction.TransactionManager()
    >>> cn = db.open(transaction_manager=tm)
    [...]
    >>> tm.get().commit()

The ``setLocalTransaction()`` and ``getTransaction()`` methods of
Connection are deprecated.  Use an explicit TM passed via
``transaction_manager=`` to ``DB.open()`` instead.  The
``setLocalTransaction()`` method still works, but it returns a TM instead of
a Transaction.

A TM creates Transaction objects, which are used for exactly one
transaction.  Transaction objects still have ``commit()``, ``abort()``,
``note()``, ``setUser()``, and ``setExtendedInfo()`` methods.

Resource managers, e.g. Connection or RDB adapter, should use a
Transaction's ``join()`` method instead of its ``register()`` method.  An
object that calls ``join()`` manages its own resources.  An object that
calls ``register()`` expects the TM to manage the objects.

Data managers written against the ZODB 4 transaction API are now
supported in ZODB 3.

persistent
----------

A database can now contain persistent weak references.  An object that
is only reachable from persistent weak references will be removed by
pack().

The persistence API now distinguishes between deactivation and
invalidation.  This change is intended to support objects that can't
be ghosts, like persistent classes.  Deactivation occurs when a user
calls _p_deactivate() or when the cache evicts objects because it is
full.  Invalidation occurs when a transaction updates the object.  An
object that can't be a ghost must load new state when it is
invalidated, but can ignore deactivation.

Persistent objects can implement a __getnewargs__() method that will
be used to provide arguments that should be passed to __new__() when
instances (including ghosts) are created.  An object that implements
__getnewargs__() must be loaded from storage even to create a ghost.

There is new support for writing hooks like __getattr__ and
__getattribute__.  The new hooks require that user code call special
persistence methods like _p_getattr() inside their hook.  See the ZODB
programming guide for details.

The format of serialized persistent references has changed; that is,
the on-disk format for references has changed.  The old format is
still supported, but earlier versions of ZODB will not be able to read
the new format.

ZODB
----

Closing a ZODB Connection while it is registered with a transaction,
e.g. has pending modifications, will raise a ConnnectionStateError.
Trying to load objects from or store objects to a closed connection
will also raise a ConnnectionStateError.

ZODB connections are synchronized on commit, even when they didn't
modify objects.  This feature assumes that the thread that opened the
connection is also the thread that uses it.  If not, this feature will
cause problems.  It can be disabled by passing synch=False to open().

New broken object support.

New add() method on Connection.  User code should not assign the
_p_jar attribute of a new persistent object directly; a deprecation
warning is issued in this case.

Added a get() method to Connection as a preferred synonym for
__getitem__().

Several methods and/or specific optional arguments of methods have
been deprecated.  The cache_deactivate_after argument used by DB() and
Connection() is deprecated.  The DB methods getCacheDeactivateAfter(),
getVersionCacheDeactivateAfter(), setCacheDeactivateAfter(), and
setVersionCacheDeactivateAfter() are also deprecated.

The old-style undo() method was removed from the storage API, and
transactionalUndo() was renamed to undo().

The BDBStorages are no longer distributed with ZODB.

Fixed a serious bug in the new pack implementation.  If pack was
called on the storage and passed a time earlier than a previous pack
time, data could be lost.  In other words, if there are any two pack
calls, where the time argument passed to the second call was earlier
than the first call, data loss could occur.  The bug was fixed by
causing the second call to raise a StorageError before performing any
work.

Fixed a rare bug in pack:  if a pack started during a small window of
time near the end of a concurrent transaction's commit, it was possible
for the pack attempt to raise a spurious

     CorruptedError: ... transaction with checkpoint flag set

exception.  This did no damage to the database, or to the transaction
in progress, but no pack was performed then.

By popular demand, FileStorage.pack() no longer propagates a

    FileStorageError:  The database has already been packed to a
    later time or no changes have been made since the last pack

exception.  Instead that message is logged (at INFO level), and
the pack attempt simply returns then (no pack is performed).

ZEO
---

Fixed a bug that prevented the -m / --monitor argument from working.

zdaemon
-------

Added a -m / --mask option that controls the umask of the subprocess.

zLOG
----

The zLOG backend has been removed.  zLOG is now just a facade over the
standard Python logging package.  Environment variables like
STUPID_LOG_FILE are no longer honored.  To configure logging, you need
to follow the directions in the logging package documentation.  The
process is currently more complicated than configured zLOG.  See
test.py for an example.

ZConfig
-------

This release of ZODB contains ZConfig 2.1.

More documentation has been written.

Make sure keys specified as attributes of the <default> element are
converted by the appropriate key type, and are re-checked for derived
sections.

Refactored the ZConfig.components.logger schema components so that a
schema can import just one of the "eventlog" or "logger" sections if
desired.  This can be helpful to avoid naming conflicts.

Added a reopen() method to the logger factories.

Always use an absolute pathname when opening a FileHandler.


Miscellaneous
-------------

The layout of the ZODB source release has changed.  All the source
code is contained in a src subdirectory.  The primary motivation for
this change was to avoid confusion caused by installing ZODB and then
testing it interactively from the source directory; the interpreter
would find the uncompiled ZODB package in the source directory and
report an import error.

A reference-counting bug was fixed, in the logic calling a modified
persistent object's data manager's register() method.  The primary symptom
was rare assertion failures in Python's cyclic garbage collection.

The Connection class's onCommitAction() method was removed.

Some of the doc strings in ZODB are now written for processing by
epydoc.

Several new test suites were written using doctest instead of the
standard unittest TestCase framework.

MappingStorage now implements getTid().

ThreadedAsync: Provide a way to shutdown the servers using an exit
status.

The mkzeoinstance script looks for a ZODB installation, not a Zope
installation.  The received wisdom is that running a ZEO server
without access to the appserver code avoids many mysterious problems.


What's new in ZODB3 3.3 alpha 2
===============================
Release date: 06-Jan-2004

This release contains a major overhaul of the persistence machinery,
including some user-visible changes.  The Persistent base class is now
a new-style class instead of an ExtensionClass.  The change enables
the use of features like properties with persistent object classes.
The Persistent base class is now contained in the persistent package.

The Persistence package is included for backwards compatibility.  The
Persistence package is used by Zope to provide special
ExtensionClass-compatibility features like a non-C3 MRO and an __of__
method.  ExtensionClass is not included with this release of ZODB3.
If you use the Persistence package, it will print a warning and import
Persistent from persistent.

In short, the new persistent package is recommended for non-Zope
applications.  The following dotted class names are now preferred over
earlier names:

- persistent.Persistent
- persistent.list.PersistentList
- persistent.mapping.PersistentMapping
- persistent.TimeStamp

The in-memory, per-connection object cache (pickle cache) was changed
to participate in garbage collection.  This should reduce the number
of memory leaks, although we are still tracking a few problems.

Multi-version concurrency control
---------------------------------

ZODB now supports multi-version concurrency control (MVCC) for
storages that support multiple revisions.  FileStorage and
BDBFullStorage both support MVCC.  In short, MVCC means that read
conflicts should almost never occur.  When an object is modified in
one transaction, other concurrent transactions read old revisions of
the object to preserve consistency.  In earlier versions of ZODB, any
access of the modified object would raise a ReadConflictError.

The ZODB internals changed significantly to accommodate MVCC.  There
are relatively few user visible changes, aside from the lack of read
conflicts.  It is possible to disable the MVCC feature using the mvcc
keyword argument to the DB open() method, ex.: db.open(mvcc=False).

ZEO
---

Changed the ZEO server and control process to work with a single
configuration file; this is now the default way to configure these
processes.  (It's still possible to use separate configuration files.)
The ZEO configuration file can now include a "runner" section used by
the control process and ignored by the ZEO server process itself.  If
present, the control process can use the same configuration file.

Fixed a performance problem in the logging code for the ZEO protocol.
The logging code could call repr() on arbitrarily long lists, even
though it only logged the first 60 bytes; worse, it called repr() even
if logging was currently disabled.  Fixed to call repr() on individual
elements until the limit is reached.

Fixed a bug in zrpc (when using authentication) where the MAC header
wasn't being read for large messages, generating errors while unpickling
commands sent over the wire. Also fixed the zeopasswd.py script, added
testcases and provided a more complete commandline interface.

Fixed a misuse of the _map variable in zrpc Connectio objects, which
are also asyncore.dispatcher objects.  This allows ZEO to work with
CVS Python (2.4). _map is used to indicate whether the dispatcher
users the default socket_map or a custom socket_map.  A recent change
to asyncore caused it to use _map in its add_channel() and
del_channel() methods, which presumes to be a bug fix (may get ported
to 2.3).  That causes our dubious use of _map to be a problem, because
we also put the Connections in the global socket_map.  The new
asyncore won't remove it from the global socket map, because it has a
custom _map.

The prefix used for log messages from runzeo.py was changed from
RUNSVR to RUNZEO.

Miscellaneous
-------------

ReadConflictError objects now have an ignore() method.  Normally, a
transaction that causes a read conflict can't be committed.  If the
exception is caught and its ignore() method called, the transaction
can be committed.  Application code may need this in advanced
applications.


What's new in ZODB3 3.3 alpha 1
===============================
Release date: 17-Jul-2003

New features of Persistence
---------------------------

The Persistent base class is a regular Python type implemented in C.
It should be possible to create new-style classes that inherit from
Persistent, and, thus, use all the new Python features introduced in
Python 2.2 and 2.3.

The __changed__() method on Persistent objects is no longer supported.

New features in BTrees
----------------------

BTree, Bucket, TreeSet and Set objects are now iterable objects, playing
nicely with the iteration protocol introduced in Python 2.2, and can
be used in any context that accepts an iterable object.  As for Python
dicts, the iterator constructed for BTrees and Buckets iterates
over the keys.

>>> from BTrees.OOBTree import OOBTree
>>> b = OOBTree({"one": 1, "two": 2, "three": 3, "four": 4})
>>> for key in b: # iterates over the keys
...    print key
four
one
three
two
>>> list(enumerate(b))
[(0, 'four'), (1, 'one'), (2, 'three'), (3, 'two')]
>>> i = iter(b)
>>> i.next()
'four'
>>> i.next()
'one'
>>> i.next()
'three'
>>> i.next()
'two'
>>>

As for Python dicts in 2.2, BTree and Bucket objects have new
.iterkeys(), .iteritems(), and .itervalues() methods.  TreeSet and Set
objects have a new .iterkeys() method.  Unlike as for Python dicts,
these new methods accept optional min and max arguments to effect
range searches.  While Bucket.keys() produces a list, Bucket.iterkeys()
produces an iterator, and similarly for Bucket values() versus
itervalues(), Bucket items() versus iteritems(), and Set keys() versus
iterkeys().  The iter{keys,values,items} methods of BTrees and the
iterkeys() method of Treesets also produce iterators, while their
keys() (etc) methods continue to produce BTreeItems objects (a form of
"lazy" iterator that predates Python 2.2's iteration protocol).

>>> sum(b.itervalues())
10
>>> zip(b.itervalues(), b.iterkeys())
[(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')]
>>>

BTree, Bucket, TreeSet and Set objects also implement the __contains__
method new in Python 2.2, which means that testing for key membership
can be done directly now via the "in" and "not in" operators:

>>> "won" in b
False
>>> "won" not in b
True
>>> "one" in b
True
>>>

All old and new range-search methods now accept keyword arguments,
and new optional excludemin and excludemax keyword arguments.  The
new keyword arguments allow doing a range search that's exclusive
at one or both ends (doesn't include min, and/or doesn't include
max).

>>> list(b.keys())
['four', 'one', 'three', 'two']
>>> list(b.keys(max='three'))
['four', 'one', 'three']
>>> list(b.keys(max='three', excludemax=True))
['four', 'one']
>>>

Other improvements
------------------

The exceptions generated by write conflicts now contain the name of
the conflicted object's class.  This feature requires support for the
storage.  All the standard storages support it.

What's new in ZODB3 3.2
========================
Release date: 08-Oct-2003

Nothing has changed since release candidate 1.

What's new in ZODB3 3.2 release candidate 1
===========================================
Release date: 01-Oct-2003

Added a summary to the Doc directory.  There are several new documents
in the 3.2 release, including "Using zdctl and zdrun to manage server
processes" and "Running a ZEO Server HOWTO."

Fixed ZEO's protocol negotiation mechanism so that a client ZODB 3.1
can talk to a ZODB 3.2 server.

Fixed a memory leak in the ZEO server.  The server was leaking a few
KB of memory per connection.

Fixed a memory leak in the ZODB object cache (cPickleCache).  The
cache did not release two references to its Connection, causing a
large cycle of objects to leak when a database was closed.

Fixed a bug in the ZEO code that caused it to leak socket objects on
Windows.  Specifically, fix the trigger mechanism so that both sockets
created for a trigger are closed.

Fixed a bug in the ZEO storage server that caused it to leave temp
files behind.  The CommitLog class contains a temp file, but it was
not closing the file.

Changed the order of setuid() and setgid() calls in zdrun, so that
setgid() is called first.

Added a timeout to the ZEO test suite that prevents hangs.  The test
suite creates ZEO servers with randomly assigned ports.  If the port
happens to be in use, the test suite would hang because the ZEO client
would never stop trying to connect.  The fix will cause the test to
fail after a minute, but should prevent the test runner from hanging.

The logging package was updated to include the latest version of the
logging package from Python CVS.  Note that this package is only
installed for Python 2.2.  In later versions of Python, it is
available in the Python standard library.

The ZEO1 directory was removed from the source distribution.  ZEO1 is
not supported, and we never intended to include it in the release.

What's new in ZODB3 3.2 beta 3
==============================
Release date: 23-Sep-2003

Note: The changes listed for this release include changes also made in
ZODB 3.1.x releases and ported to the 3.2 release.

This version of ZODB 3.2 is not compatible with Python 2.1.  Early
versions were explicitly designed to be compatible with Zope 2.6.
That plan has been dropped, because Zope 2.7 is already in beta
release.

Several of the classes in ZEO and ZODB now inherit from object, making
them new-style classes.  The primary motivation for the change was to
make it easier to debug memory leaks.  We don't expect any behavior to
change as a result.

A new feature to allow removal of connection pools for versions was
ported from Zope 2.6.  This feature is needed by Zope to avoid denial
of service attacks that allow a client to create an arbitrary number
of version pools.

Fixed several critical ZEO bugs.

- If several client transactions were blocked waiting for the storage
  and one of the blocked clients disconnected, the server would
  attempt to restart one of the other waiting clients.  Since the
  disconnected client did not have the storage lock, this could lead
  to deadlock.  It could also cause the assertion "self._client is
  None" to fail.

- If a storage server fails or times out between the vote and the
  finish, the ZEO cache could get populated with objects that didn't
  make it to the storage server.

- If a client loses its connection to the server near the end of a
  transaction, it is now guaranteed to get a ClientDisconnected error
  even if it reconnects before the transaction finishes.  This is
  necessary because the server will always abort the transaction.
  In some cases, the client would never see an error for the aborted
  transaction.

- In tpc_finish(), reordered the calls so that the server's tpc_finish()
  is called (and must succeed) before we update the ZEO client cache.

- The storage name is now prepended to the sort key, to ensure a
  unique global sort order if storages are named uniquely.  This
  can prevent deadlock in some unusual cases.

Fixed several serious flaws in the implementation of the ZEO
authentication protocol.

- The smac layer would accept a message without a MAC even after the
  session key was established.

- The client never initialized its session key, so it never checked
  incoming messages or created MACs for outgoing messags.

- The smac layer used a single HMAC instance for sending and receiving
  messages.  This approach could only work if client and server were
  guaranteed to process all messages in the same total order, which
  could only happen in simple scenarios like unit tests.

Fixed a bug in ExtensionClass when comparing ExtensionClass instances.
The code could raise RuntimeWarning under Python 2.3, and produce
incorrect results on 64-bit platforms.

Fixed bug in BDBStorage that could lead to DBRunRecoveryErrors when a
transaction was aborted after performing operations like commit
version or undo that create new references to existing pickles.

Fixed a bug in Connection.py that caused it to fail with an
AttributeError if close() was called after the database was closed.

The test suite leaves fewer log files behind, although it still leaves
a lot of junk.  The test.py script puts each tests temp files in a
separate directory, so it is easier to see which tests are causing
problems.  Unfortunately, it is still to tedious to figure out why the
identified tests are leaving files behind.

This release contains the latest and greatest version of the
BDBStorage.  This storage has still not seen testing in a production
environment, but it represents the current best design and most recent
code culled from various branches where development has occurred.

The Tools directory contains a number of small improvements, a few new
tools, and README.txt that catalogs the tools.  Many of the tools are
installed by setup.py; those scripts will now have a #! line set
automatically on Unix.

Fixed bugs in Tools/repozo.py, including a timing-dependent one that
could cause the following invocation of repozo to do a full backup when
an incremental backup would have sufficed.

A pair of new scripts from Jim Fulton can be used to synthesize
workloads and measure ZEO performance:  see zodbload.py and
zeoserverlog.py in the Tools directory.  Note that these require
Zope.

Tools/checkbtrees.py was strengthened in two ways:

- In addition to running the _check() method on each BTree B found,
  BTrees.check.check(B) is also run.  The check() function was written
  after checkbtrees.py, and identifies kinds of damage B._check()
  cannot find.

- Cycles in the object graph no longer lead to unbounded output.
  Note that preventing this requires remembering the oid of each
  persistent object found, which increases the memory needed by the
  script.

What's new in ZODB3 3.2 beta 2
==============================
Release date: 16-Jun-2003

Fixed critical race conditions in ZEO's cache consistency code that
could cause invalidations to be lost or stale data to be written to
the cache.  These bugs can lead to data loss or data corruption.
These bugs are relatively unlikely to be provoked in sites with few
conflicts, but the possibility of failure existed any time an object
was loaded and stored concurrently.

Fixed a bug in conflict resolution that failed to ghostify an object
if it was involved in a conflict.  (This code may be redundant, but it
has been fixed regardless.)

The ZEO server was fixed so that it does not perform any I/O until all
of a transactions' invalidations are queued.  If it performs I/O in the
middle of sending invalidations, it would be possible to overlap a
load from a client with the invalidation being sent to it.

The ZEO cache now handles invalidations atomically.  This is the same
sort of bug that is described in the 3.1.2b1 section below, but it
affects the ZEO cache.

Fixed several serious bugs in fsrecover that caused it to fail
catastrophically in certain cases because it thought it had found a
checkpoint (status "c") record when it was in the middle of the file.

Two new features snuck into this beta release.

The ZODB.transact module provides a helper function that converts a
regular function or method into a transactional one.

The ZEO client cache now supports Adaptable Persistence (APE).  The
cache used to expect that all OIDs were eight bytes long.

What's new in ZODB3 3.2 beta 1
==============================
Release date: 30-May-2003

ZODB
----

Invalidations are now processed atomically.  Each transaction will see
all the changes caused by an earlier transaction or none of them.
Before this patch, it was possible for a transaction to see invalid
data because it saw only a subset of the invalidations.  This is the
most likely cause of reported BTrees corruption, where keys were
stored in the wrong bucket.  When a BTree bucket splits, the bucket
and the bucket's parent are both modified.  If a transaction sees the
invalidation for the bucket but not the parent, the BTree in memory
will be internally inconsistent and keys can be put in the wrong
bucket.  The atomic invalidation fix prevents this problem.

A number of minor reference count fixes in the object cache were
fixed.  That's the cPickleCache.c file.

It was possible for a transaction that failed in tpc_finish() to lose
the traceback that caused the failure.  The transaction code was fixed
to report the original error as well as any errors that occur while
trying to recover from the original error.

The "other" argument to copyTransactionsFrom() only needs to have an
.iterator() method.  For convenience, change FileStorage's and
BDBFullStorage's iterator to have this method, which just returns
self.

Mount points are now visible from mounted objects.

Fixed memory leak involving database connections and caches.  When a
connection or database was closed, the cache and database leaked,
because of a circular reference involving the cache.  Fixed the cache
to explicitly clear out its contents when its connection is closed.

The ZODB cache has fewer methods.  It used to expose methods that
could mutate the dictionary, which allowed users to violate internal
invariants.

ZConfig
-------

It is now possible to configure ZODB databases and storages and ZEO
servers using ZConfig.

ZEO & zdaemon
-------------

ZEO now supports authenticated client connections.  The default
authentication protocol uses a hash-based challenge-response protocol
to prove identity and establish a session key for message
authentication.  The architecture is pluggable to allow third-parties
to developer better authentication protocols.

There is a new HOWTO for running a ZEO server.  The draft in this
release is incomplete, but provides more guidance than previous
releases.  See the file Doc/ZEO/howto.txt.


The ZEO storage server's transaction timeout feature was refactored
and made slightly more rebust.

A new ZEO utility script, ZEO/mkzeoinst.py, was added.  This creates a
standard directory structure and writes a configuration file with
mostly default values, and a bootstrap script that can be used to
manage and monitor the server using zdctl.py (see below).

Much work was done to improve zdaemon's zdctl.py and zdrun.py scripts.
(In the alpha 1 release, zdrun.py was called zdaemon.py, but
installing it in <prefix>/bin caused much breakage due to the name
conflict with the zdaemon package.)  Together with the new
mkzeoinst.py script, this makes controlling a ZEO server a breeze.

A ZEO client will not read from its cache during cache verification.
This fix was necessary to prevent the client from reading inconsistent
data.

The isReadOnly() method of a ZEO client was fixed to return the false
when the client is connected to a read-only fallback server.

The sync() method of ClientStorage and the pending() method of a zrpc
connection now do both input and output.

The short_repr() function used to generate log messages was fixed so
that it does not blow up creating a repr of very long tuples.

Storages
--------

FileStorage has a new pack() implementation that fixes several
reported problems that could lead to data loss.

Two small bugs were fixed in DemoStorage.  undoLog() did not handle
its arguments correctly and pack() could accidentally delete objects
created in versions.

Fixed trivial bug in fsrecover that prevented it from working at all.

FileStorage will use fsync() on Windows starting with Python 2.2.3.

FileStorage's commit version was fixed.  It used to stop after the
first object, leaving all the other objects in the version.

BTrees
------

Trying to store an object of a non-integer type into an IIBTree
or OIBTree could leave the bucket in a variety of insane states.  For
example, trying

    b[obj] = "I'm a string, not an integer"

where b is an OIBTree.  This manifested as a refcount leak in the test
suite, but could have been much worse (most likely in real life is that
a seemingly arbitrary existing key would "go missing").

When deleting the first child of a BTree node with more than one
child, a reference to the second child leaked.  This could cause
the entire bucket chain to leak (not be collected as garbage
despite not being referenced anymore).

Other minor BTree leak scenarios were also fixed.

Tools
-----

New tool zeoqueue.py for parsing ZEO log files, looking for blocked
transactions.

New tool repozo.py (originally by Anthony Baxter) for performing
incremental backups of Data.fs files.

The fsrecover.py script now does a better job of recovering from
errors the occur in the middle of a transaction record.  Fixed several
bugs that caused partial or total failures in earlier versions.


What's new in ZODB3 3.2 alpha 1
===============================
Release date: 17-Jan-2003

Most of the changes in this release are performance and stability
improvements to ZEO.  A major packaging change is that there won't be
a separate ZEO release.  The new ZConfig is a noteworthy addtion (see
below).

ZODB
----

An experimental new transaction API was added.  The Connection class
has a new method, setLocalTransaction().  ZODB applications can call
this method to bind transactions to connections rather than threads.
This is especially useful for GUI applications, which often have only
one thread but multiple independent activities within that thread
(generally one per window).  Thanks to Christian Reis for championing
this feature.

Applications that take advantage of this feature should not use the
get_transaction() function.  Until now, ZODB itself sometimes assumed
get_transaction() was the only way to get the transaction.  Minor
corrections have been added.  The ZODB test suite, on the other hand,
can continue to use get_transaction(), since it is free to assume that
transactions are bound to threads.

ZEO
---

There is a new recommended script for starting a storage server.  We
recommend using ZEO/runzeo.py instead of ZEO/start.py.  The start.py
script is still available in this release, but it will no longer be
maintained and will eventually be removed.

There is a new zdaemon implementation.  This version is a separate
script that runs an arbitrary daemon.  To run the ZEO server as a
daemon, you would run "zdrun.py runzeo.py".  There is also a simple
shell, zdctl.py, that can be used to manage a daemon.  Try
"zdctl.py -p runzeo.py".

There is a new version of the ZEO protocol in this release and a first
stab at protocol negotiation.  (It's a first stab because the protocol
checking supporting in ZODB 3.1 was too primitive to support anything
better.)  A ZODB 3.2 ZEO client can talk to an old server, but a ZODB
3.2 server can't talk to an old client.  It's safe to upgrade all the
clients first and upgrade the server last.  The ZEO client cache
format changed, so you'll need to delete persistent caches before
restarting clients.

The ZEO cache verification protocol was revised to require many fewer
messages in cases where a client or server restarts quickly.

The performance of full cache verification has improved dramatically.
Measurements from Jim were somewhere in 2x-5x.  The
implementation was fixed to use the very-fast getSerial() method on
the storage instead of the comparatively slow load().

The ZEO server has an optional timeout feature that will abort a
connection that does not commit within a certain amount of time.  The
timeout works by closing the socket the client is using, causing both
client and server to abort the transaction and continue.  This is a
drastic step, but can be useful to prevent a hung client or other bug
from blocking a server indefinitely.

A bug was fixed in the ZEO protocol that allowed clients to read stale
cache data while cache verification was being performed.  The fixed
version prevents the client from using the storage until after
verification completes.

The ZEO server has an experimental monitoring interface that reports
usage statistics for the storage server including number of connected
clients and number of transactions active and committed.  It can be
enabled by passing the -m flag to runsvr.py.

The ZEO ClientStorage no longer supports the environment variables
CLIENT_HOME, INSTANCE_HOME, or ZEO_CLIENT.

The ZEO1 package is still included with this release, but there is no
longer an option to install it.

BTrees
------

The BTrees package now has a check module that inspects a BTree to
check internal invariants.  Bugs in older versions of the code code
leave a BTree in an inconsistent state.  Calling BTrees.check.check()
on a BTree object should verify its consistency.  (See the NEWS
section for 3.1 beta 1 below to for the old BTrees bugs.)

Fixed a rare conflict resolution problem in the BTrees that could
cause an segfault when the conflict resolution resulted in any
empty bucket.

Installation
------------

The distutils setup now installs several Python scripts.  The
runzeo.py and zdrun.py scripts mentioned above and several fsXXX.py
scripts from the Tools directory.

The test.py script does not run all the ZEO tests by default, because
the ZEO tests take a long time to run.  Use --all to run all the
tests.  Otherwise a subset of the tests, mostly using MappingStorage,
are run.

Storages
--------

There are two new storages based on Sleepycat's BerkeleyDB in the
BDBStorage package.  Barry will have to write more here, because I
don't know how different they are from the old bsddb3Storage
storages.  See Doc/BDBStorage.txt for more information.

It now takes less time to open an existing FileStorage.  The
FileStorage uses a BTree-based index that is faster to pickle and
unpickle.  It also saves the index periodically so that subsequent
opens will go fast even if the storage was not closed cleanly.

Misc
----

The new ZConfig package, which will be used by Zope and ZODB, is
included.  ZConfig provides a configuration syntax, similar to
Apache's syntax.  The package can be used to configure the ZEO server
and ZODB databases.  See the module ZODB.config for functions to open
the database from configuration.  See ZConfig/doc for more info.

The zLOG package now uses the logging package by Vinay Sajip, which
will be included in Python 2.3.

The Sync extension was removed from ExtensionClass, because it was not
used by ZODB.

What's new in ZODB3 3.1.4?
==========================
Release date: 11-Sep-2003

A new feature to allow removal of connection pools for versions was
ported from Zope 2.6.  This feature is needed by Zope to avoid denial
of service attacks that allow a client to create an arbitrary number
of version pools.

A pair of new scripts from Jim Fulton can be used to synthesize
workloads and measure ZEO performance:  see zodbload.py and
zeoserverlog.py in the Tools directory.  Note that these require
Zope.

Tools/checkbtrees.py was strengthened in two ways:

- In addition to running the _check() method on each BTree B found,
  BTrees.check.check(B) is also run.  The check() function was written
  after checkbtrees.py, and identifies kinds of damage B._check()
  cannot find.

- Cycles in the object graph no longer lead to unbounded output.
  Note that preventing this requires remembering the oid of each
  persistent object found, which increases the memory needed by the
  script.

What's new in ZODB3 3.1.3?
==========================
Release date: 18-Aug-2003

Fixed several critical ZEO bugs.

- If a storage server fails or times out between the vote and the
  finish, the ZEO cache could get populated with objects that didn't
  make it to the storage server.

- If a client loses its connection to the server near the end of a
  transaction, it is now guaranteed to get a ClientDisconnected error
  even if it reconnects before the transaction finishes.  This is
  necessary because the server will always abort the transaction.
  In some cases, the client would never see an error for the aborted
  transaction.

- In tpc_finish(), reordered the calls so that the server's tpc_finish()
  is called (and must succeed) before we update the ZEO client cache.

- The storage name is now prepended to the sort key, to ensure a
  unique global sort order if storages are named uniquely.  This
  can prevent deadlock in some unusual cases.

A variety of fixes and improvements to Berkeley storage (aka BDBStorage)
were back-ported from ZODB 4.  This release now contains the most
current version of the Berkeley storage code.  Many tests have been
back-ported, but not all.

Modified the Windows tests to wait longer at the end of ZEO tests for
the server to shut down.  Before Python 2.3, there is no waitpid() on
Windows, and, thus, no way to know if the server has shut down.  The
change makes the Windows ZEO tests much less likely to fail or hang,
at the cost of increasing the time needed to run the tests.

Fixed a bug in ExtensionClass when comparing ExtensionClass instances.
The code could raise RuntimeWarning under Python 2.3, and produce
incorrect results on 64-bit platforms.

Fixed bugs in Tools/repozo.py, including a timing-dependent one that
could cause the following invocation of repozo to do a full backup when
an incremental backup would have sufficed.

Added Tools/README.txt that explains what each of the scripts in the
Tools directory does.

There were many small changes and improvements to the test suite.

What's new in ZODB3 3.1.2 final?
================================

Fixed bug in FileStorage pack that caused it to fail if it encountered
an old undo record (status "u").

Fixed several bugs in FileStorage pack that could cause OverflowErrors
for storages > 2 GB.

Fixed memory leak in TimeStamp.laterThan() that only occurred when it
had to create a new TimeStamp.

Fixed two BTree bugs that were fixed on the head a while ago:

   - bug in fsBTree that would cause byValue searches to end early.
     (fsBTrees are never used this way, but it was still a bug.)

   -  bug that lead to segfault if BTree was mutated via deletion
      while it was being iterated over.

What's new in ZODB3 3.1.2 beta 2?
=================================

Fixed critical race conditions in ZEO's cache consistency code that
could cause invalidations to be lost or stale data to be written to
the cache.  These bugs can lead to data loss or data corruption.
These bugs are relatively unlikely to be provoked in sites with few
conflicts, but the possibility of failure existed any time an object
was loaded and stored concurrently.

Fixed a bug in conflict resolution that failed to ghostify an object
if it was involved in a conflict.  (This code may be redundant, but it
has been fixed regardless.)

The ZEO server was fixed so that it does not perform any I/O until all
of a transactions' invalidations are queued.  If it performs I/O in the
middle of sending invalidations, it would be possible to overlap a
load from a client with the invalidation being sent to it.

The ZEO cache now handles invalidations atomically.  This is the same
sort of bug that is described in the 3.1.2b1 section below, but it
affects the ZEO cache.

Fixed several serious bugs in fsrecover that caused it to fail
catastrophically in certain cases because it thought it had found a
checkpoint (status "c") record when it was in the middle of the file.

What's new in ZODB3 3.1.2 beta 1?
=================================

ZODB
----

Invalidations are now processed atomically.  Each transaction will see
all the changes caused by an earlier transaction or none of them.
Before this patch, it was possible for a transaction to see invalid
data because it saw only a subset of the invalidations.  This is the
most likely cause of reported BTrees corruption, where keys were
stored in the wrong bucket.  When a BTree bucket splits, the bucket
and the bucket's parent are both modified.  If a transaction sees the
invalidation for the bucket but not the parent, the BTree in memory
will be internally inconsistent and keys can be put in the wrong
bucket.  The atomic invalidation fix prevents this problem.

A number of minor reference count fixes in the object cache were
fixed.  That's the cPickleCache.c file.

It was possible for a transaction that failed in tpc_finish() to lose
the traceback that caused the failure.  The transaction code was fixed
to report the original error as well as any errors that occur while
trying to recover from the original error.

ZEO
---

A ZEO client will not read from its cache during cache verification.
This fix was necessary to prevent the client from reading inconsistent
data.

The isReadOnly() method of a ZEO client was fixed to return the false
when the client is connected to a read-only fallback server.

The sync() method of ClientStorage and the pending() method of a zrpc
connection now do both input and output.

The short_repr() function used to generate log messages was fixed so
that it does not blow up creating a repr of very long tuples.

Storages
--------

FileStorage has a new pack() implementation that fixes several
reported problems that could lead to data loss.

Two small bugs were fixed in DemoStorage.  undoLog() did not handle
its arguments correctly and pack() could accidentally delete objects
created in versions.

Fixed trivial bug in fsrecover that prevented it from working at all.

FileStorage will use fsync() on Windows starting with Python 2.2.3.

FileStorage's commit version was fixed.  It used to stop after the
first object, leaving all the other objects in the version.

BTrees
------

Trying to store an object of a non-integer type into an IIBTree
or OIBTree could leave the bucket in a variety of insane states.  For
example, trying

    b[obj] = "I'm a string, not an integer"

where b is an OIBTree.  This manifested as a refcount leak in the test
suite, but could have been much worse (most likely in real life is that
a seemingly arbitrary existing key would "go missing").

When deleting the first child of a BTree node with more than one
child, a reference to the second child leaked.  This could cause
the entire bucket chain to leak (not be collected as garbage
despite not being referenced anymore).

Other minor BTree leak scenarios were also fixed.

Other
-----

Comparing a Missing.Value object to a C type that provide its own
comparison operation could lead to a segfault when the Missing.Value
was on the right-hand side of the comparison operator.  The Missing
class was fixed so that its coercion and comparison operations are
safe.

Tools
-----

Four tools are now installed by setup.py: fsdump.py, fstest.py,
repozo.py, and zeopack.py.

What's new in ZODB3 3.1.1 final?
================================
Release date: 11-Feb-2003

Tools
-----

Updated repozo.py tool

What's new in ZODB3 3.1.1 beta 2?
=================================
Release date: 03-Feb-2003

The Transaction "hosed" feature is disabled in this release.  If a
transaction fails during the tpc_finish() it is not possible, in
general, to know whether the storage is in a consistent state.  For
example, a ZEO server may commit the data and then fail before sending
confirmation of the commit to the client.  If multiple storages are
involved in a transaction, the problem is exacerbated: One storage may
commit the data while another fails to commit.  In previous versions
of ZODB, the database would set a global "hosed" flag that prevented
any other transaction from committing until an administrator could
check the status of the various failed storages and ensure that the
database is in a consistent state.  This approach favors data
consistency over availability.  The new approach is to log a panic but
continue.  In practice, availability seems to be more important than
consistency.  The failure mode is exceedingly rare in either case.

The BTrees-based fsIndex for FileStorage is enabled.  This version of
the index is faster to load and store via pickle and uses less memory
to store keys.  We had intended to enable this feature in an earlier
release, but failed to actually do it; thus, it's getting enabled as a
bug fix now.

Two rare bugs were fixed in BTrees conflict resolution.  The most
probable symptom of the bug would have been a segfault.  The bugs
were found via synthetic stress tests rather than bug reports.

A value-based consistency checker for BTrees was added.  See the
module BTrees.check for the checker and other utilities for working
with BTrees.

A new script called repozo.py was added.  This script, originally
written by Anthony Baxter, provides an incremental backup scheme for
FileStorage based storages.

zeopack.py has been fixed to use a read-only connection.

Various small autopack-related race conditions have been fixed in the
Berkeley storage implementations.  There have been some table changes
to the Berkeley storages so any storage you created in 3.1.1b1 may not
work.  Part of these changes was to add a storage version number to
the schema so these types of incompatible changes can be avoided in
the future.

Removed the chance of bogus warnings in the FileStorage iterator.

ZEO
---

The ZEO version number was bumped to 2.0.2 on account of the following
minor feature additions.

The performance of full cache verification has improved dramatically.
Measurements from Jim were somewhere in 2x-5x.  The
implementation was fixed to use the very-fast getSerial() method on
the storage instead of the comparatively slow load().

The ZEO server has an optional timeout feature that will abort a
connection that does not commit within a certain amount of time.  The
timeout works by closing the socket the client is using, causing both
client and server to abort the transaction and continue.  This is a
drastic step, but can be useful to prevent a hung client or other bug
from blocking a server indefinitely.

If a client was disconnected during a transaction, the tpc_abort()
call did not properly reset the internal state about the transaction.
The bug caused the next transaction to fail in its tpc_finish().
Also, any ClientDisconnected exceptions raised during tpc_abort() are
ignored.

ZEO logging has been improved by adding more logging for important
events, and changing the logging level for existing messages to a more
appropriate level (usually lower).

What's new in ZODB3 3.1.1 beta 1?
=================================
Release date: 10-Dev-2002

It was possible for earlier versions of ZODB to deadlock when using
multiple storages.  If multiple transactions committed concurrently
and both transactions involved two or more shared storages, deadlock
was possible.  This problem has been fixed by introducing a sortKey()
method to the transaction and storage APIs that is used to define an
ordering on transaction participants.  This solution will prevent
deadlocks provided that all transaction participants that use locks
define a valid sortKey() method.  A warning is raised if a participant
does not define sortKey().  For backwards compatibility, BaseStorage
provides a sortKey() that uses __name__.

Added code to ThreadedAsync/LoopCallback.py to work around a bug in
asyncore.py: a handled signal can cause unwanted reads to happen.

A bug in FileStorage related to object uncreation was fixed.  If an
a transaction that created an object was undone, FileStorage could
write a bogus data record header that could lead to strange errors if
the object was loaded.  An attempt to load an uncreated object now
raises KeyError, as expected.

The restore() implementation in FileStorage wrote incorrect
backpointers for a few corner cases involving versions and undo.  It
also failed if the backpointer pointed to a record that was before the
pack time.  These specific bugs have been fixed and new test cases
were added to cover them.

A bug was fixed in conflict resolution that raised a NameError when a
class involved in a conflict could not be loaded.  The bug did not
affect correctness, but prevent ZODB from caching the fact that the
class was unloadable.  A related bug prevented spurious
AttributeErrors when a class could not be loaded.  It was also fixed.

The script Tools/zeopack.py was fixed to work with ZEO 2.  It was
untested and had two silly bugs.

Some C extensions included standard header files before including
Python.h, which is not allowed.  They now include Python.h first,
which eliminates compiler warnings in certain configurations.

The BerkeleyDB based storages have been merged from the trunk,
providing a much more robust version of the storages.  They are not
backwards compatible with the old storages, but the decision was made
to update them in this micro release because the old storages did not
work for all practical purposes.  For details, see Doc/BDBStorage.txt.

What's new in ZODB3 3.1 final?
===============================
Release date: 28-Oct-2002

If an error occurs during conflict resolution, the store will silently
catch the error, log it, and continue as if the conflict was
unresolvable.  ZODB used to behave this way, and the change to catch
only ConflictError was causing problems in deployed systems.  There
are a lot of legitimate errors that should be caught, but it's too
close to the final release to make the substantial changes needed to
correct this.

What's new in ZODB3 3.1 beta 3?
===============================
Release date: 21-Oct-2002

A small extension was made to the iterator protocol.  The Record
objects, which are returned by the per-transaction iterators, contain
a new `data_txn` attribute.  It is None, unless the data contained in
the record is a logical copy of an earlier transaction's data.  For
example, when transactional undo modifies an object, it creates a
logical copy of the earlier transaction's data.  Note that this
provide a stronger statement about consistency than whether the data
in two records is the same; it's possible for two different updates to
an object to coincidentally have the same data.

The restore() method was extended to take the data_txn attribute
mentioned above as an argument.  FileStorage uses the new argument to
write a backpointer if possible.

A few bugs were fixed.

The setattr slot of the cPersistence C API was being initialized to
NULL.  The proper initialization was restored, preventing crashes in
some applications with C extensions that used persistence.

The return value of TimeStamp's __cmp__ method was clipped to return
only 1, 0, -1.

The restore() method was fixed to write a valid backpointer if the
update being restored is in a version.

Several bugs and improvements were made to zdaemon, which can be used
to run the ZEO server.  The parent now forwards signals to the child
as intended.  Pidfile handling was improved and the trailing newline
was omitted.

What's new in ZODB3 3.1 beta 2?
===============================
Release date: 4-Oct-2002

A few bugs have been fixed, some that were found with the help of
Neal Norwitz's PyChecker.

The zeoup.py tool has been fixed to allow connecting to a read-only
storage, when the --nowrite option is given.

Casey Duncan fixed a few bugs in the recent changes to undoLog().

The fstest.py script no longer checks that each object modified in a
transaction has a serial number that matches the transaction id.
This invariant is no longer maintained; several new features in the
3.1 release depend on it.

The ZopeUndo package was added.  If ZODB3 is being used to run a ZEO
server that will be used with Zope, it is usually best if the server
and the Zope client don't share any software.  The Zope undo
framework, however, requires that a Prefix object be passed between
client and server.  To support this use, ZopeUndo was created to hold
the Prefix object.

Many bugs were fixed in ZEO, and a couple of features added.  See
`ZEO-NEWS.txt` for details.

The ZODB guide included in the Doc directory has been updated.  It is
still incomplete, but most of the references to old ZODB packages have
been removed.  There is a new section that briefly explains how to use
BTrees.

The zeoup.py tool connects using a read-only connection when --nowrite
is specifified.  This feature is useful for checking on read-only ZEO
servers.

What's new in ZODB3 3.1 beta 1?
===============================
Release date: 12-Sep-2002

We've changed the name and version number of the project, but it's
still the same old ZODB.  There have been a lot of changes since the
last release.

New ZODB cache
--------------

Toby Dickenson implemented a new Connection cache for ZODB.  The cache
is responsible for pointer swizzling (translating between oids and
Python objects) and for keeping recently used objects in memory.  The
new cache is a big improvement over the old cache.  It strictly honors
its size limit, where size is specified in number of objects, and it
evicts objects in least recently used (LRU) order.

Users should take care when setting the cache size, which has a
default value of 400 objects.  The old version of the cache often held
many more objects than its specified size.  An application may not
perform as well with a small cache size, because the cache no longer
exceeds the limit.

Storages
--------

The index used by FileStorage was reimplemented using a custom BTrees
object.  The index maps oids to file offsets, and is kept in memory at
all times.  The new index uses about 1/4 the memory of the old,
dictionary-based index.  See the module ZODB.fsIndex for details.

A security flaw was corrected in transactionalUndo().  The transaction
ids returned by undoLog() and used for transactionalUndo() contained a
file offset.  An attacker could construct a pickle with a bogus
transaction record in its binary data, deduce the position of the
pickle in the file from the undo log, then submit an undo with a bogus
file position that caused the pickle to get written as a regular data
record.  The implementation was fixed so that file offsets are not
included in the transaction ids.

Several storages now have an explicit read-only mode.  For example,
passing the keyword argument read_only=1 to FileStorage will make it
read-only.  If a write operation is performed on a read-only storage,
a ReadOnlyError will be raised.

The storage API was extended with new methods that support the Zope
Replication Service (ZRS), a proprietary Zope Corp product.  We expect
these methods to be generally useful.  The methods are:

    - restore(oid, serialno, data, version, transaction)

      Perform a store without doing consistency checks.  A client can
      use this method to provide a new current revision of an object.
      The ``serialno`` argument is the new serialno to use for the
      object, not the serialno of the previous revision.

    - lastTransaction()

      Returns the transaction id of the last committed transaction.

    - lastSerial(oid)

      Return the current serialno for ``oid`` or None.

    - iterator(start=None, stop=None)

      The iterator method isn't new, but the optional ``start`` and
      ``stop`` arguments are.  These arguments can be used to specify
      the range of the iterator -- an inclusive range [start, stop].

FileStorage is now more cautious about creating a new file when it
believes a file does not exist.  This change is a workaround for bug
in Python versions upto and including 2.1.3.  If the interpreter was
builtin without large file support but the platform had it,
os.path.exists() would return false for large files.  The fix is to
try to open the file first, and decide whether to create a new file
based on errno.

The undoLog() and undoInfo() methods of FileStorage can run
concurrently with other methods.  The internal storage lock is
released periodically to give other threads a chance to run.  This
should increase responsiveness of ZEO clients when used with ZEO 2.

New serial numbers are assigned consistently for abortVersion() and
commitVersion().  When a version is committed, the non-version data
gets a new serial number.  When a version is aborted, the serial
number for non-version data does not change.  This means that the
abortVersion() transaction record has the unique property that its
transaction id is not the serial number of the data records.


Berkeley Storages
-----------------

Berkeley storage constructors now take an optional `config` argument,
which is an instance whose attributes can be used to configure such
BerkeleyDB policies as an automatic checkpointing interval, lock table
sizing, and the log directory.  See bsddb3Storage/BerkeleyBase.py for
details.

A getSize() method has been added to all Berkeley storages.

Berkeley storages open their environments with the DB_THREAD flag.

Some performance optimizations have been implemented in Full storage,
including the addition of a helper C extension when used with Python
2.2.  More performance improvements will be added for the ZODB 3.1
final release.

A new experimental Autopack storage was added which keeps only a
certain amount of old revision information.  The concepts in this
storage will be folded into Full and Autopack will likely go away in
ZODB 3.1 final.  ZODB 3.1 final will also have much improved Minimal
and Full storages, which eliminate Berkeley lock exhaustion problems,
reduce memory use, and improve performance.

It is recommended that you use BerkeleyDB 4.0.14 and PyBSDDB 3.4.0
with the Berkeley storages.  See bsddb3Storage/README.txt for details.


BTrees
------

BTrees no longer ignore exceptions raised when two keys are compared.

Tim Peters fixed several endcase bugs in the BTrees code.  Most
importantly, after a mix of inserts and deletes in a BTree or TreeSet, it
was possible (but unlikely) for the internal state of the object to become
inconsistent.  Symptoms then varied; most often this manifested as a
mysterious failure to find a key that you knew was present, or that
tree.keys() would yield an object that disgreed with the tree about
how many keys there were.

If you suspect such a problem, BTrees and TreeSets now support a ._check()
method, which does a thorough job of examining the internal tree pointers
for consistency.  It raises AssertionError if it finds any problems, else
returns None.  If ._check() raises an exception, the object is damaged,
and rebuilding the object is the best solution.  All known ways for a
BTree or TreeSet object to become internally inconsistent have been
repaired.

Other fixes include:

- Many fixes for range search endcases, including the "range search bug:"
  If the smallest key S in a bucket in a BTree was deleted, doing a range
  search on the BTree with S on the high end could claim that the range
  was empty even when it wasn't.

- Zope Collector #419:  repaired off-by-1 errors and IndexErrors when
  slicing BTree-based data structures.  For example,
  an_IIBTree.items()[0:0] had length 1 (should be empty) if the tree
  wasn't empty.

- The BTree module functions weightedIntersection() and weightedUnion()
  now treat negative weights as documented.  It's hard to explain what
  their effects were before this fix, as the sign bits were getting
  confused with an internal distinction between whether the result
  should be a set or a mapping.

ZEO
----

For news about ZEO2, see the file ZEO-NEWS.txt.

This version of ZODB ships with two different versions of ZEO.  It
includes ZEO 2.0 beta 1, the recommended new version.  (ZEO 2 will
reach final release before ZODB3.)  The ZEO 2.0 protocol is not
compatible with ZEO 1.0, so we have also included ZEO 1.0 to support
people already using ZEO 1.0.

Other features
--------------

When a ConflictError is raised, the exception object now has a
sensible structure, thanks to a patch from Greg Ward.  The exception
now uses the following standard attributes: oid, class_name, message,
serials.  See the ZODB.POSException.ConflictError doc string for
details.

It is now easier to customize the registration of persistent objects
with a transaction.  The low-level persistence mechanism in
cPersistence.c registers with the object's jar instead of with the
current transaction.  The jar (Connection) then registers with the
transaction.  This redirection would allow specialized Connections to
change the default policy on how the transaction manager is selected
without hacking the Transaction module.

Empty transactions can be committed without interacting with the
storage.  It is possible for registration to occur unintentionally and
for a persistent object to compensate by making itself as unchanged.
When this happens, it's possible to commit a transaction with no
modified objects.  The change allows such transactions to finish even
on a read-only storage.

Two new tools were added to the Tools directory.  The ``analyze.py``
script, based on a tool by Matt Kromer, prints a summary of space
usage in a FileStorage Data.fs.  The ``checkbtrees.py`` script scans a
FileStorage Data.fs.  When it finds a BTrees object, it loads the
object and calls the ``_check`` method.  It prints warning messages
for any corrupt BTrees objects found.

Documentation
-------------

The user's guide included with this release is still woefully out of date.

Other bugs fixed
----------------

If an exception occurs inside an _p_deactivate() method, a traceback
is printed on stderr.  Previous versions of ZODB silently cleared the
exception.

ExtensionClass and ZODB now work correctly with a Python debug build.

All C code has been fixed to use a consistent set of functions from
the Python memory API.  This allows ZODB to be used in conjunction
with pymalloc, the default allocator in Python 2.3.

zdaemon, which can be used to run a ZEO server, more clearly reports
the exit status of its child processes.

The ZEO server will reinitialize zLOG when it receives a SIGHUP.  This
allows log file rotation without restarting the server.

What's new in StandaloneZODB 1.0 final?
=======================================
Release date: 08-Feb-2002

All copyright notices have been updated to reflect the fact that the
ZPL 2.0 covers this release.

Added a cleanroom PersistentList.py implementation, which multiply
inherits from UserDict and Persistent.

Some improvements in setup.py and test.py for sites that don't have
the Berkeley libraries installed.

A new program, zeoup.py was added which simply verifies that a ZEO
server is reachable.  Also, a new program zeopack.py was added which
connects to a ZEO server and packs it.


What's new in StandaloneZODB 1.0 c1?
====================================
Release Date: 25-Jan-2002

This was the first public release of the StandaloneZODB from Zope
Corporation.   Everything's new! :)