1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739
|
New in version 0.11.0:
Library Changes:
* BUG FIX: When using a bzipped /REFERENCE field, mixing gd_nframes()
calls with reads of data from that field no longer results in
subsequent reads to occur in the wrong place. Reported by Matthew
Petroff.
* BUG FIX FOR CVE-2021-20204: The first RAW field in a Dirfile having
the same name as a previously defined field no longer results in a
segmentation fault when trying to access the Dirifle's reference
field.
By default, the parser will return a syntax error (GD_E_FORMAT)
when encountering a duplicate field name, making the DIRFILE
invalid and avoiding the segmentation fault. To trigger the
segmentation fault, the standard GD_E_FORMAT error return
must be avoided, by doing one of the following:
- The GD_IGNORE_DUPS flag must be used when invoking the format
file parser. This stops the parser from generating an error
when discarding out the duplicate field.
- A caller-supplied parser callback must return GD_SYNTAX_IGNORE
when passed the GD_E_FORMAT error generated by the duplicate
field. (This is what happens in the checkdirfile utility,
which is why it is affected by this bug.)
API Changes:
* A new function gd_open_limit() can be used to limit the number of
open file descriptors used by the library to access RAW data in
a given DIRFILE. GetData will try to keep under the limit by
automatically closing RAW fields when necessary. Note: the
library may exceed the limit in certain cases. See the man
page for full details.
* A number of symbols, which were deprecated in GetData-0.8,
have been removed. This removal extends to the corresponding
symbols in the bindings. The removed symbols, and their
replacements, are:
Removed symbol Replacement
------------------- -------------------
GD_FLOAT GD_FLOAT32
GD_DOUBLE GD_FLOAT64
GD_E_BAD_ENDIANNESS GD_E_ARGUMENT
GD_E_BAD_PROTECTION GD_E_ARGUMENT
GD_E_BAD_VERSION GD_E_ARGUMENT
GD_E_FORMAT_NO_PARENT GD_E_FORMAT_NO_FIELD
gd_bit_t int
gd_spf_t unsigned int
NOTE: The Python bindings have a different FLOAT type alias which
corresponds to the native float type in Python. That symbol is
not deprecated and has not been removed.
* BUG FIX: Lingering references to the type gd_shift_t, which was
deprecated in GetData-0.10.0 and replaced by gd_int64_t, have been
removed from the API.
Bindings Changes:
* PYTHON BUG FIX: When first_sample is non-zero, and neither num_frames
nor num_samples is indicated dirfile.getdata() no longer attempts to
read past the end of the data file. Patch from Matthew Petroff.
Miscellaneous
* Running "make clean" no longer accidentally deletes most of the man pages
from the man/ subdirectory.
* New options --with-pcre and --without-pcre have been added to ./configure
to adjust how the PCRE library is (or is not) used.
* Modules now link to the main getdata library to avoid the potential for
symbol look-up errors on load. Reported by Matthew Petroff.
* Fortran bindings can now be built with gfortran version >= 10.
|=========================================================================|
New in version 0.10.0:
Dirfile Changes:
* Dirfile Standards Version 10 has been released. It adds three new
field types: SARRAY, which is an array of STRING scalars (like a
CARRAY, but with STRINGs instead of CONSTs), and INDIR and SINDIR,
which are vector fields provide indexed look-ups from CARRAY (INDIR) or
SARRAY (SINDIR) scalar arrays. It also adds field code namespaces,
which can be specified with the new /NAMESPACE directive, or else as
part of an /INCLUDE directive.
* Some notes about namespaces:
- Namespaces are separated from field names by a dot (.):
namespace.name
and can be nested arbitrarily deep, separating namespaces with
intermediate dots:
namespace.subspace.subsubspace.name
Namespaces were created to provide an alternative to the prefix and
suffix added to the /INCLUDE directive in Standards Version 9, which
have some unfortunate side-effects due to their modifying field names
directly. Because namespaces nest and are syntactically separate
from field names, they do a better job of encapsulation.
- If the namespace of a field is null (""), which is the default, then
the dot separating the namespace from the field name may be omitted.
- An /INCLUDE statement can specify a namespace which becomes the
included fragment's "root namespace". The root namespace of the base
(top-level) format file is always the nullspace (""), and cannot be
changed.
- In addition to root namespaces, there is also a "current namespace".
At the top of a fragment, the current namespace is set to the root
namespace. A /NAMESPACE directive can be used to change the current
namespace to a subspace under the root namespace. That is, if the
root namespace is "root", then the directive:
/NAMESPACE subspace
changes the current namespace to "root.subspace" regardless of what
the current namespace was before. To change the current namespace
back to the root namespace, use the null token ("") with the
/NAMESPACE directive:
/NAMESPACE ""
- If no namespace is specified in an /INCLUDE line, then the current
namespace becomes the root namespace of the included fragment.
- Subnamespaces under the root namespace may also be specified directly
in the name part of a field specification. As a result, it is never
necessary to use the /NAMESPACE directive.
- Every field code and name in a fragment implicitly gains either the
fragment's root namespace or the current namespace. If the field
code starts with an initial dot, then the root namespace is prepended
to it, otherwise the current namespace is prepended to it. Because
the current namespace is always a subspace of the root namespace,
this means that metadata in a given fragment is never able to access
other fields outside its own root namespace.
- The exception to the above is the implicit INDEX vector, which
ignores all namespaces attached to it, either implicitly, through the
current namespace, or explicitly when specified in the metadata.
This contrasts with the behaviour of the INDEX field in the presence
of affixes, where it can appears or disappears based on the effects
of the affixes creating or modifying the specific field name "INDEX".
* Because syntactically the dot (.) now performs two functions, namely
both separating namespaces from each other and field names, but also
separating a field code from a representation suffix, there exists
ambiguity in the syntax. To resolve the ambiguity, a new representa-
tion suffix, .z has been added which does nothing. As an example, the
field code:
name.r
is interpreted as the real part of the field named "name", assuming
such a field exists. To indicate the field named "r" in the namespace
"name", the field code:
name.r.z
must be used. Note that this ambiguity only exists in a Dirfile where
both "name" and "name.r" are valid field names. If the field named
"name" doesn't exist, then the first field code, "name.r" is unambig-
uously interpreted as the field "r" in the namespace "name".
* A note on the SINDIR field: Unlike every other vector field, this
field produces character string data at the sample rate of it's input
vector, which may be surprising, in certain instances.
* GetData has supported FLAC compression since 0.9.0. Standards Version
10 now adds "flac" to the list of pre-defined encodings.
Library Changes:
* The function gd_array_len() no longer silently ignores a representation
suffix in the field_code provided. In most cases, passing a
representation suffix will now cause this function to fail with a
GD_E_BAD_CODE error.
* A couple of unnecessary malloc's have been removed from the field
code search, reducing the chance of encountering a GD_E_ALLOC error.
Notably, the functions gd_array_len(), gd_bof(), gd_entry_type(),
gd_fragment_index(), and gd_spf() will no longer produce this error at
all.
* NOTE: The GD_VECTOR_ENTRIES symbol in gd_entry_list() and gd_nentries()
calls does not match SINDIR entries, only numeric-valued vectors.
This also affects the corresponding special case functions
(gd_nvectors(), gd_vector_list() &c.)
* BUG FIX: When building in ANSI-C mode, the computation of complex-
valued RECIP fields is now correct.
* BUG FIX: The gd_include() family of functions now correctly clean up
after encountering an error. Previously, encountering a syntax error
would result in these functions erroneously adding fields specified
before the syntax error to the DIRFILE, with a bogus fragment index.
Similarly, when these functions return GD_E_REFERENCE, they no longer
add the included fragment (excluding the bad /REFERENCE directive) to
the DIRFILE without telling the caller about it.
* BUG FIX: gd_alter_protection() wasn't marking affected fragments as
modified, meaning the change in protection level would be lost upon
close unless other metadata changes were made to the fragment as well,
or a flush of the fragment's metadata was triggered explicitly with a
call to gd_rewrite_fragment().
* BUG FIX: The metadata update performed by gd_delete() now successfully
updates all fields which used the deleted field as an input.
Previously some fields could be skipped, leading to segfaults later
when these fields were accessed.
* BUG FIX: gd_add_spec() no longer creates an empty data file while
failing with GD_E_PROTECT when operating in a fragment with data
protection turned on.
* BUG FIX: gd_putdata() now refuses to write to complex-valued LINTERP
fields. Previously, the write would succeed as if the imaginary
part of the field were zero.
* BUG FIX: gd_nentries() now correctly rejects invalid values for the
type parameter.
* BUG FIX GetData now properly deals with circular series of aliases,
turning them all into dangling aliases. Previously, the alias
resolution would terminate at some arbitrary point around the loop,
resulting in internal errors arising from attempts to use an alias as
a field.
* BUG FIX: Several bugs in the I/O positioning performed before reads and
writes to compressed data have been fixed. Previously, reads and
writes could occur in the wrong place in the data stream. Reported by
S. J. Benton.
* BUG FIX: Similarly, a number of bugs associated with random-access
writes to compressed data files which were causing data corruption have
been fixed.
* BUG FIX: When reading LZMA-compressed data, gd_getdata() no longer
hangs if liblzma returns only part of a multibyte sample.
* BUG FIX: The FLAC encoding now works correctly with non-native endian
data.
* BUG FIX: Writes to ASCII (text) encoded files weren't properly updating
the file's I/O position, leading to subsequent reads and writes
occurring in the wrong place.
* BUG FIX: Trying to open a non-existent, gzip-encoded data file now
reports the correct error (GD_E_IO; "No such file or directory"),
instead of segfaulting. Reported by Matthew Petroff.
* BUG FIX: A segfault encountered when closing very large compressed
files after writing to them has been fixed.
* BUG FIX: Attempting to open a SIE-encoded data file a second time
after a first attempt failed no longer results in a segfault.
* BUG FIX: The parser no longer assumes a new string buffer returned by
the parser callback (by assigning it to pdata->line) has the size given
by pdata->buflen, which the callback is not required to update, but
instead determines the buffer size directly. Previously, this
assumption could result in a segfault within the parser.
* BUG FIX: gd_add_polynom() and gd_add_cpolynom() no longer reject valid
poly_ord values.
* BUG FIX: The gd_[m]add() family of functions are now better at
rejecting invalid data types.
* BUG FIX: A segfault-on-error has been fixed in gd_[m]alter_spec.
* BUG FIX: The gd_madd() functions longer accept aliases as parent field
names.
* BUG FIX: Setting n_fields (for LINCOMs) or poly_ord (for POLYNOMs) to
an out-of-range value and then calling gd_free_entry_strings() no
longer results in a segfault.
* BUG FIX: A rare segfault has been fixed in gd_carrays() and
gd_strings().
API Changes:
* The new function gd_alloc_funcs() allows callers to change the memory
manager used by GetData to allocate and de-allocate heap buffers that
it returns. The functions gd_entry(), gd_error_string(),
gd_fragment_affixes(), gd_linterp_tablename(), and gd_raw_filename()
will use this memory manager to allocate the buffers they return. The
function gd_free_entry_strings() will use this memory manager to free
strings. Even if an alternate memory manager is specified, GetData
will still use the Standard Library's malloc() and free() for much of
it's internal storage.
* A new function, gd_match_entries(), extends the functionality of
gd_entry_list() and gd_nentries() by additionally allowing both
searches restricted to a particular fragment and also regular
expression matching against entry names.
* A number of functions which used to return -1 on error now instead
return an appropriate error code, previously only available through
gd_error(). These error codes are all negative-valued. Functions
whose error returns have changed and now do this are:
gd_add(), gd_add_alias(), all the gd_add_<entry_type>() functions,
gd_add_spec(), gd_alter_affixes(), all the gd_alter_<entry_type>(),
functions, gd_alter_encoding(), gd_alter_endianness(),
gd_alter_entry(), gd_alter_frameoffset(), gd_alter_protection(),
gd_alter_spec(), gd_array_len(), gd_bof(), gd_delete(), gd_desync(),
gd_dirfile_standards(), gd_discard(), gd_entry(), gd_entry_type(),
gd_eof(), gd_flush(), gd_fragment_affixes(), gd_fragment_index(),
gd_frameoffset(), gd_get_carray(), gd_get_carray_slice(),
gd_get_constant(), gd_hidden(), gd_hide(), gd_include(),
gd_include_affix(), gd_include_ns(), gd_madd(), gd_madd_alias(),
all the gd_madd_<entry_type>() functions, gd_madd_spec(),
gd_metaflush(), gd_move(), gd_nframes(), gd_open(),
gd_parent_fragment(), gd_protection(), gd_put_carray(),
gd_put_carray_slice(), gd_put_const(), gd_raw_close(),
gd_rename(), gd_rewrite_fragment(), gd_seek(), gd_spf(), gd_sync(),
gd_tell(), gd_unhide(), gd_uninclude(), gd_validate(),
gd_verbose_prefix()
* gd_add_indir(), gd_add_sarray(), gd_add_sindir(), gd_madd_indir(),
gd_madd_sarray(), gd_madd_sindir(), gd_alter_indir(),
gd_alter_sarray(), and gd_alter_sindir() have been added to manipulate
metadata of the new field types.
* gd_msarrays(), gd_get_sarray(), gd_get_sarray_slice(), gd_put_sarray(),
gd_put_sarray_slice(), gd_sarrays() have been added to read and write
SARRAY values.
* gd_fragment_namespace() and gd_include_ns() have been added to read and
write the root namespace of a fragment.
* gd_put_string() now returns zero on success and a negative-valued error
code on error, as well, following the lead of gd_put_constant() and
gd_put_sarray(). Also note that the return type is now int, where
previously it was size_t, despite the documentation having always
claimed it returned int.
* A new data type symbol, GD_STRING, exists to represent string data.
The gd_native_type() function now returns GD_STRING for STRING fields
(as well as for the new SARRAY and SINDIR fields).
* The GD_FUNCTION_ALIASES block, and hence the long-deprecated
GetData-0.6 API, has been removed from getdata.h. Anyone still using
the GetData-0.6 API should modernise to avoid known problems with that
API.
* The gd_shift_t type, which was used for PHASE field shifts, has been
deprecated. It has been replaced with gd_int64_t, which is what it
was always typedef'd to.
* BUG FIX: gd_getdata() and gd_putdata() now properly report GD_E_IO when
an I/O error occurs while reading a LINTERP table. Previously GD_E_LUT
would be erroneously returned, as if the table file were empty.
Bindings Changes:
* MATLAB: The FLAGS argument to GD_INCLUDE is now optional and defaults
to zero.
* C++, F77, F95, PERL: The bindings for gd_put_string() have changed to
reflect changes in the C library. Fortran 77 no longer returns a
n_wrote integer; Fortran 95 implements fgd_put_string a subroutine;
C++ and Perl bindings now return integer zero on success, and negative
on error.
* IDL, MATLAB, PHP BUG FIX: Bindings for the C functions gd_raw_filename
and gd_linterp_tablename now no longer leak the string returned by the
C API.
* PYTHON BUG FIX: A UnicodeEncodeError while assigning to
dirfile.verbose_prefix no longer results in that attribute being set
to None. Instead, it retains its former value, which reflects what
actually happens in the underlying C library in this case.
* PYTHON BUG FIX: Objects returned by dirfile.entry() and
dirfile.fragment() weren't being initialised with the correct reference
count, leading to memory leaks when they went out of scope. Reported
by Alexandra Rahlin.
Miscellaneous:
* The --enable-assert configure option, which hasn't done anything for a
long time, has been removed.
* A new configure option, --disable-util, can be used to suppress
building of the executables in the util/ subdirectory.
* In the standard autotools build system, encodings which use external
libraries for compression (gzip, bzip2, flac, lzma, slim, zzip, zzslim)
are now by default built as dynamically loaded modules. To recover the
old build behaviour, which put everything into the core GetData library
binary, pass --disable-modules to ./configure. Using modules intro-
duces a runtime dependency on GNU libltdl. The CMake-based build
system used in the native Microsoft Windows source release retains the
old (monolithic) behaviour.
|=========================================================================|
New in version 0.9.4:
Library Changes:
* BUG FIX: Arbitrarily-long reads of FLAC-encoded files now work.
Previously, each FLAC frame was written to the start of the output
buffer, overwriting the previous frame, and leaving most of the
buffer uninitialised. Reported by S. J. Benton.
* BUG FIX: Data read from FLAC-encoded 1-byte types are now correct.
Previously, although all data requested was read, only the first half
would be returned, encoded as 16-bit data (i.e., with a zero-byte
between every sample).
Bindings Changes:
* PYTHON BUG FIX: An erroneous preprocessor definition which made the
Python bindings unbuildable for Python 3 has been fixed. Reported by
Akito Kusaka.
Miscellaneous:
* A work-around has been implemented in the configure script to avoid a
bug in bash 4.1's parser. Reported by Akito Kusaka.
* The way configure decides the Python module path has changed slightly
to provide a better default under debian-based systems when using
debhelper. Reported by S. J. Benton.
|=========================================================================|
New in version 0.9.3:
Library Changes:
* GetData can now read SIE files containing the optional nine-byte header
(which can be created by daisie). When read by GetData, information in
the header is completely ignored (because the GetData metadata contains
all the necessary information). GetData never writes the header, but
calls to gd_putdata() will preserve an existing header. Calls which
re-code the file (like gd_alter_endianness(), gd_alter_raw(), &c.) will
result in an existing header being deleted.
* BUG FIX: Filenames and line numbers appearing in GD_E_FORMAT error
strings returned by gd_error_string() are correct again. This bug
also affected parser metadata sent to a registered parser callback.
* BUG FIX: On platforms where char is signed, the library no longer
rejects field names containing bytes with the top bit set.
API Changes:
* gd_verbose_prefix() can now be used on invalid dirfiles. Previously,
this function would return GD_E_BAD_DIRFILE when passed an invalid
DIRFILE pointer.
* BUG FIX: When trying to access a LINTERP table file in a non-existent
directory, GetData now reports the correct error (No such file or
directory). Reported by Johanna Nagy.
Bindings Changes:
* PHP: PHP7 support has been added.
* PYTHON: Python3 support has been added based on a patch from Matthew
Petroff. The earliest supported Python3 version is 3.2. Python2 is
still supported from version 2.4, but Unicode support must be enabled
in Python2. Support for Python 2.3 has been dropped.
* PYTHON: Under Python3, the bindings run into the issue of GetData not
knowing the character encoding of Dirfile metadata (the C library
just deals with bytes). As a result, under Python3, by default,
most strings returned by the library are returned as encoded bytes()
objects, instead of native Unicode str() objects. (This is true
under Python2 as well, but less obvious since the native Python2
str() object is encoded.)
To help deal with this, dirfile and entry objects now have a
character_encoding attribute which can be set to inform pygetdata of
the character encoding to use to decode the strings returned by the
C library into Unicode strings. There is also a global
pygetdata.character_encoding object which can be used to set the
default encoding for newly-created pygetdata objects. See the module
documentation for details. The default for character_encoding is None,
implying no decoding should occur. In Python3, paths are handled
separately (since the filesystem encoding may be different than the
GetData metadata encoding). Error strings are decoded if possible, and
then ASCII encoded to ensure they're always available, regardless of
the capabilities of standard error.
* PYTHON: As a side-effect to the above, pygetdata now accepts Unicode
strings (both in Python2 and in Python3). The specified
character_encoding will be used to encode them to C strings before
being passed to the C library. If no character_encoding is specified,
the current locale's default encoding will be used.
* PYTHON: When using Python 2.6 or newer, calling repr() on a
pygetdata.entry object now returns an eval()able string.
* PYTHON: When using Python 2.7 or newer, a very small C API is produced
for pygetdata. It is defined in pygetdata.h installed alongside the
pygetdata module. This C API is needed by daisie, the stand-alone SIE
encoding library.
* C++ BUG FIX: Including another GetData header before getdata/dirfile.h
no longer results in a fatal circular dependency.
* PHP BUG FIX: The numbits parameter to gd_madd_sbit() is now optional
and defaults to 1, as with other similar functions.
* PHP BUG FIX: Fixed a memory leak on error in gd_getdata().
* PYTHON BUG FIX: Calling pygetdata.dirfile.[m]carrays no longer crashes
if return_type=pygetdata.NULL. In this case, None is returned in place
of the data arrays.
* PYTHON BUG FIX: Entry objects returned by dirfile.entry() now properly
represent scalar field codes.
* PYTHON BUG FIX: An out-of-memory condition encountered by the
underlying C library is now reported using the standard MemoryError
exception. Previously, the bindings would attempt to instantiate a new
pygetdata.AllocError exception when this happened, which wouldn't work
if no memory was available. (The CPython interpreter pre-allocates an
instance of MemoryError to deal with this situation.) For backwards
compatibility, pygetdata.AllocError is now an alias for MemoryError.
* PYTHON BUG FIX: a scalar field code string can now be assigned to the
threshold attribute of WINDOW entries.
* PYTHON BUG FIX: Attempting to delete various attributes from
pygetdata objects now either succeeds or else raises an exception,
instead of crashing.
Miscellaneous:
* The default install directory for the IDL, Perl, PHP, and Python
bindings has changed: they are now all installed under ${exec_prefix}
when possible. The install directories can still be overridden by
./configure options.
|=========================================================================|
New in version 0.9.2.1:
Miscellaneous:
* This release fixes one entry in the test suite (alter_entry_scalar3r)
which was broken in the original 0.9.2 release. The library and
bindings are unchanged (and report their version to be simply 0.9.2).
The error in the test suite was reported by Dinar Valeev.
New in version 0.9.2:
Library Changes:
* BUG FIX: When using a FLAC-encoded reference field, gd_nframes() no
longer leaks file descriptors.
* BUG FIX: A segfault in gd_entry_list() introduced in 0.9.1 has been
fixed. Reported by Christian Trippe.
* BUG FIX: A descriptor leak on error has been plugged in gd_desync().
* BUG FIX: A segfault triggered by encountering an I/O error while
writing metadata has been fixed.
* BUG FIX: Memory leaks in the FLAC and ZZIP encodings, plus single-byte
leaks in gd_add_string() and gd_madd_string() have been plugged.
* BUG FIX: gd_alter_entry() and gd_[m]alter_spec() no longer mangle the
values of entry parameters when asked to remove scalar field codes.
Reported by Dan Horák.
Bindings Changes:
* PYTHON BUG FIX: Fixed a potential segfault in dirfile.get_carray() on
platforms where sizeof size_t != sizeof int. Reported by Dan Horák.
* PHP BUG FIX: Calling gd_discard or gd_close on a persistent dirfile now
does nothing, instead of closing the Dirfile and corrupting the
persistent resource.
|=========================================================================|
New in version 0.9.1:
Library Changes:
* Functions returning entry lists (gd_entry_list(), &c.) or bulk scalar
field data (gd_constants(), gd_carrays(), &c.) no longer run through
the entry list twice, resulting in noticeable speed improvements when
operating on large dirfiles.
* BUG FIX: Top-level aliases pointing to metafields, and non-hidden
aliases pointing to hidden entries are now included in the data
returned by gd_nentries() and gd_entry_list() when they should be.
* BUG FIX: Surprising combinations of hidden fields, meta fields, and
aliases no long confuse the functions which return bulk scalar data
(gd_constants, gd_strings, gd_carrays, &c.). Previously, given the
right metadata, these functions would segfault, return incorrect
data, or raise GD_E_INTERNAL_ERROR.
* BUG FIX: The functions gd_delete(), gd_hide(), gd_hidden(), and
gd_unhide() no longer segfault when asked to operate on a field that
doesn't exist; instead, they properly report error (GD_E_BAD_CODE).
* BUG FIX: gd_seek() and gd_tell() now always return -1 on error.
Previously, they would return zero for some errors; they always
still correctly set the dirfile error.
* BUG FIX: Several memory or other resource leaks triggered by error
returns have been fixed.
* BUG FIX: GetData no longer ignores errors reported by zlib when
writing to gzipped data files. In the past, this could, in some
cases, cause GetData to wedge.
API Changes:
* BUG FIX: gd_constants() and gd_mconstants() now return NULL and set the
dirfile error to GD_E_BAD_TYPE if passed GD_NULL as the return_type.
Previously these functions would return NULL without setting an error
when passed GD_NULL.
* BUG FIX: gd_getdata() and gd_putdata() now truncate their operation to
avoid overflow. If size_t is N-bytes wide, and GetData is operating on
an M-byte wide data type, then truncation occurs at 2**(8N-M)-1
samples. Note, however, that truncation can also be triggered by
intermediate data products, which may have wider data types (larger M)
than the type specified by the caller. If truncation occurs, a short
read/write will occur, without raising an error. Truncation occurs
even when the data type is GD_NULL (i.e. when these functions have
nothing to do). For GD_NULL, M is taken to be 1. Callers should
check the return value of these functions, and make another call to
handle the truncation, if necessary.
* BUG FIX: Instead of seeking to a random place, gd_seek() now returns
GD_E_RANGE when the specified offset, or the resultant position would
overflow a 64-bit signed integer. Note, however, that, in most cases,
the file size limit imposed by your filesystem or OS will be
encountered first (in which case GD_E_IO will be returned). For
comparison, the Windows NTFS driver limits files to 2**44 bytes,
as does the ext4 filesystem.
* BUG FIX: Similarly, gd_getdata() and gd_putdata() return GD_E_RANGE if
asked to operate on data outside the addressable range (more than 2**63
samples past the start of the dirfile).
Bindings Changes:
* PYTHON: The return_type parameter to dirfile.get_constant() and
dirfile.get_carray() is now optional, and will default to the native
type of the data returned, as happens with dirfile.getdata().
* IDL BUG FIX: Using !GD.NULL with GD_GETDATA(), GD_GET_CONSTANT() and
GD_GET_CARRAY() now works. In this case GD_GETDATA() returns the
number of samples read (as in the C API), and the other functions
return zero on success.
* MATLAB BUG FIX: Using GD.NULL with gd_getdata(), gd_get_constant(),
gd_get_carray(), and gd_get_carray_slice(), now works. In this case,
gd_getdata() returns the number of samples read (as in the C API), and
the other functions return zero on success.
* PERL BUG FIX: Using $GetData::NULL with getdata(), get_constant(),
get_carray(), and get_carray_slice() now works. In scalar context,
getdata() returns the number of samples read, while the others return
undef. In list context, they all return an empty array in this case.
* PHP BUG FIX: Passing data with a GD_NULL data type now behaves as
expected (i.e. the passed data is ignored).
* PHP BUG FIX: Using GD_NULL with gd_getdata(), gd_get_constant(),
gd_get_carray() now works. In this case, gd_getdata() returns the
number of samples read (as in the C API), and the other functions
return TRUE on success.
* PYTHON BUG FIX: Using return_type=pygetdata.NULL with
dirfile.getdata(), dirfile.get_constant() or dirfile.get_carray() now
works. dirfile.getdata() will return the number of samples read (as in
the C API), ignoring as_list. The other functions return None on
success.
* PYTHON BUG FIX: Reflecting the merge of GD_E_BAD_REPR into
GD_E_BAD_CODE in the C API in 0.9.0, the BadRepr exception is now an
alias of BadCode.
Miscellaneous:
* The build system now uses ExtUtils::MakeMaker instead of Module::Build
to build the perl bindings. Module::Build was removed from the Perl 5
core in Perl 5.22. ExtUtils::MakeMaker is still a core module. The
bindings are unchanged; the change in build prerequisites is the only
difference users should notice due to this switch.
* A new configure option, --disable-large-tests, will cause the test
suite to skip running tests that write large amounts (>~10MB) of data.
|=========================================================================|
New in version 0.9.0:
Library Changes:
* Literals in format metadata may now have complex form (i.e. include a
semicolon) when the parameter is purely real. However, a non-zero
imaginary part is still an error.
* gd_free_entry_strings() now NULLs pointers after freeing them.
* gd_entry() now returns entry metadata when they contain scalar field
codes which do not exist. In this case the GD_EN_CALC flag in the
object will not be set. Previously, on such entries, this function
would fail with the error GD_E_BAD_SCALAR, and return nothing.
* gd_rename() now by default updates the target of ALIASes pointing to a
renamed field to point to the new field instead of leaving them dangle.
(But see GD_REN_DANGLE in the API section below).
* CARRAYs are no longer truncated to GD_MAX_CARRAY_LENGTH elements.
Flushing metadata to disk will now fail if writing a CARRAY would
overflow a format file line. (It's platform specific, but format file
lines are typically permitted to be at least 2**31 bytes long, so such
an error usually indicates something pathological happening.) The
GD_MAX_CARRAY_LENGTH symbol has been removed from the GetData header
file.
* Write support for bzip2-encoded and lzma-encoded data has been added.
LZMA write support is only available for .xz files, not the obsolete
.lzma format. The write support occurs out-of-place, just like how
writing gzip-encoded data works. See the gzip discussion in the 0.8.0
section below for important notes.
* A new encoding scheme using the Free Lossless Audio Codec (FLAC) to
compress data has been implemented. For some datasets, it provides a
good trade-off between speed and compression. Like gzip, bzip2, and
lzma, is also uses out-of-place writes (see previous point).
* A newly-created dirfile is now always opened in read-write mode, ignor-
ing the access mode specified in the call. Previously, specifying both
GD_RDONLY and GD_CREAT in open calls would result in an access mode
(GD_E_ACCMODE) error if the dirfile didn't already exist.
* Many functions which used to silently ignore representation suffixes in
field codes passed to them no longer do that. Most of these will
report an error (GD_E_BAD_CODE) if passed a representation suffix. The
affected functions are: gd_bof, gd_entry, gd_entry_type, gd_eof,
gd_flush, gd_linterp_tablename, gd_put_carray, gd_put_carray_slice,
gd_putdata, gd_raw_close, gd_raw_filename, gd_seek, gd_spf, gd_sync,
gd_tell.
* The error code GD_E_BAD_REPR has been merged into GD_E_BAD_CODE. The
symbol GD_E_BAD_REPR remains as an alias for GD_E_BAD_CODE, but is
deprecated.
* Attempts to seek past the end-of-field with gd_seek() now always
succeed, although the resultant position is encoding specific.
Previously, attempting to seek past the end-of-field on some encodings
would return an error.
* BUG FIX: The library now properly recovers from an I/O error while
trying to open an unencoded datafile. Previously, such an error would
poison the library's bookkeeping data, preventing all subsequent
attempts to open that file unless the Dirfile was re-opened. Reported
by Alexandra Rahlin.
* BUG FIX: GetData no longer segfaults when trying to do a large forward
seek before a write to a gzipped file. Reported by Joy Didier.
* BUG FIX: gd_putdata() no longer ignores I/O errors while seeking to the
first sample of a write.
* BUG FIX: If the reference field is being written to, gd_nframes() now
flushes it first before calculating the size of the dirfile.
Previously a short count could result for some encodings in this case.
* BUG FIX: Calling gd_putdata() to write gzip data with a non-zero
starting offset equal to the field's current I/O position, no longer
result in the call hanging.
* BUG FIX: In addition to the addition of write support mentioned above,
a number of problems with reading LZMA files has been fixed, which
should result in fewer segmentation faults.
* BUG FIX: The parser no longer silently appends a closing > to scalar
field codes that contain an unmatched opening < (e.g. "scalar<3").
This is now interpreted as a simple field code (which may be rejected
later due to the presence of the invalid '<' character).
* BUG FIX: The parser no longer interprets various numbers as field codes
when it shouldn't (e.g. when specifying a PHASE shift as "1." instead
of "1").
* BUG FIX: When writing scalar field codes to disk which could be inter-
preted as a number (e.g. the field code "1"), the library now forces
the interpretation of these field codes as codes rather than numbers by
appending a scalar index (making, e.g., "1<0>"), which is harmless.
Previously, these were written as-is, resulting in misinterpretation
the next time the Dirfile was opened. This only happens with Standards
Version 8 or later, see the following for earlier versions.
* BUG FIX: If the current Standards Version in effect is 7 or earlier,
ambiguous field codes (e.g., "1"), are now rejected by gd_[m]add() and
gd_alter_entry() with the error GD_E_BAD_CODE, since they can't be
represented in the metadata on disk. For the behaviour with later
Versions, and in permissive mode, see the previous.
* BUG FIX: If performing a metadata update due to renaming fields
(perhaps by passing GD_REN_UPDB to gd_rename()) results in an invalid
field code due to affix restrictions, the update now fails (but see
GD_REN_FORCE). Previously the invalid field code would be stored,
leading to errors when flushing the modified metadata to disk.
* BUG FIX: When performing a metadata update due to a renamed field, the
field codes containing subfields of the renamed field are now also
updated, including field codes specifying meta subfields which do not
exist.
* BUG FIX: reading a LINTERP table with fewer than two lines no longer
results in a segfault on close/discard.
* BUG FIX: gd_alter_raw() and similar no longer fail when asked to re-
encode the data file of a RAW field which has not been previously
accessed.
* BUG FIX: A previously-read LINTERP table is now always discarded when
changing table paths with gd_alter_linterp() or similar. Previously
these obsolete, cached LUTs would sometimes linger, causing incorrect
LINTERP computation.
* BUG FIX: The I/O position reported by gd_tell and gd_seek for slim,
zzip, and zzslim encoded data is now correct.
API Changes:
* CLARIFICATION: The macro GD_SIZE() declared in getdata.h is indeed part
of the public API. It returns the size in bytes of a sample of data of
a given type (e.g. GD_SIZE(GD_COMPLEX64) returns 8). It has been
around since GetData-0.3.0, but has only been documented since
GetData-0.8.3.
* The comp_scal member of the gd_entry_t object has been replaced with a
flags member, containing a flag (GD_EN_COMPSCAL) with the meaning of
the former comp_scal member. There are also flags for hiddenness
(GD_EN_HIDDEN) and whether the scalar entry codes in the field defi-
nition have been dereferenced (GD_EN_CALC).
* gd_[m]add() and gd_alter_entry() can now be used to set or change the
hiddenness of a field by setting or clearing the GD_EN_HIDDEN bit in
the supplied gd_entry_t object.
* Two new rename flags have been added:
- GD_REN_DANGLE which indicates the library shouldn't update ALIASes
whose target has been renamed (instead it will turn them into
dangling aliases)
- GD_REN_FORCE which causes the library to skip updating field codes
which would be invalid due to affixes instead of failing.
* The move_data argument of gd_move() has been replaced with a flags
argument which accepts the GD_REN_* flags, which have the same meaning
as they do with gd_rename().
* gd_move_alias() and gd_delete_alias() have been deleted: their
functions are now performed by gd_move() and gd_delete(), which now
operate on the alias itself when given the field code to an alias,
rather than the field the alias points to.
* A number of different error codes which indicated the same problem (an
I/O error returned by the operating system) have been merged into one.
The error codes GD_E_OPEN, GD_E_TRUNC, GD_E_RAW_IO, GD_E_OPEN_FRAGMENT,
GD_E_FLUSH are replaced by the new error GD_E_IO. The old symbols
remain as aliases but are deprecated. The corresponding error strings
also now include information from the underlying encoding library,
where possible. There is one exception to this merge: attempts to
flush metadata lines which are too long are now reported using
GD_E_LINE_TOO_LONG. Previously, these errors used GD_E_FLUSH.
* The error code GD_E_OPEN_LINFILE has also been removed. It has been
split into two parts:
- I/O errors resulting from reading the LINTERP table file are now
reported using GD_E_IO;
- Syntax errors in the table are reported using the new GD_E_LUT error
code. GD_E_OPEN_LINFILE remains as a deprecated alias for GD_E_LUT.
* gd_encoding_support() has been added to permit run-time determination
of supported encodings.
* gd_array_len() is the new name for gd_carray_len(). It now also
handles STRINGs (which have a length of one). The gd_carray_len() name
remains in the library, but has been marked deprecated.
* BUG FIX: If the dirfile path provided cannot be resolved (due to, for
instance, a symbolic link pointing to a non-existent path), gd_open()
and friends now return the correct error code (GD_E_IO).
* BUG FIX: gd_naliases() now returns an unsigned int, and zero on error,
as documented.
* BUG FIX: The API on 32-bit systems, which was broken in 0.8.7 and only
partially fixed in 0.8.8, should now work as expected again.
Bindings Changes:
* PHP bindings have been added.
* C++: There is no longer a default value for the "index" argument for
Entry methods (including subclasses) which accept it (viz. Input,
Scalar, ScalarIndex, Scale, CScale, Offset COffset, Coefficient,
CCoefficient). The exception to this is with Entry subclasses for
which zero is the only allowed value for the parameter.
* F77 and F95: The bindings no longer raise SIGABRT when the dirfile
space is exhausted. Instead they simply return a dirfile unit number
referencing a static, invalid dirfile.
* F77: Functions to add fields with named scalar parameters have been
added (GDASBT GDASCL GDASCP GDASCR GDASLC GDASMX GDASPH GDASPN GDASRC
GDASRW GDASSB GDASWD), but only for those field types which permit
named scalars. Similarly, functions for altering field metadata with
named scalars are also present (GDLSBT GDLSCL GDLSCP GDLSCR GDLSLC
GDLSMX GDLSPH GDLSPN GDLSRC GDLSRW GDLSSB GDLSWD). These are provided
as an alternative to using GDASCA after the fact.
* IDL: The entry structure parser has been rewritten. It no longer
requires members which it doesn't need, and is also a lot more lax
about numerical data types. Notably, it now ignores a supplied
COMP_SCAL member. Floating point parameters can be specified in either
the base name (M, B, A, DIVIDEND) or else the member prefixed with 'C'
(CM, CB, CA, CDIVIDEND), whatever numerical type. The bindings will
ingest them appropriately. Also, N_FIELDS and POLY_ORD, may be
omitted, and will be calculated from the supplied data. A scalar
IN_FIELDS is treated like an single element array.
* IDL: GD_REFERENCE is now a function, instead of a procedure, as the
documentation has always claimed it was. It returns the current
reference field (or the empty string, if there is none). The second
parameter, the new reference field, is optional. (Previously the
second parameter was required.)
* PERL: The entry hash parser has been rewritten. It no longer requires
keys which it doesn't need.
* PERL: alter_entry() now only updates defined elements in the passed
entry hash.
* PYTHON: Building the python bindings now requires NumPy. Previously,
NumPy support was optional.
* PYTHON: for backwards compatibility, exceptions now exist for
deprecated error codes (such as OpenError). These deprecated
exceptions are simply aliases for the current ones and are never
returned by the bindings.
* C++ BUG FIX: The Entry methods Input, Scalar, and ScalarIndex
(including subclasses) now return zero or NULL when passed an out-of-
range index value. Previously they would return, variously, zero,
NULL, another value for some other, valid index value, or segfault.
* C++ BUG FIX: The flags parameter to Dirfile::Delete() is now unsigned,
as it is in the C API.
* F95 BUG FIX: fgd_add and fgd_alter_entry no longer ignore named scalar
parameters provided in supplied entry structures.
* PYTHON BUG FIX: Several memory leaks have been plugged. Patch from
Matthew Petroff.
Miscellaneous:
* The minimum autotools versions have been bumped. Autoconf-2.65 or
newer, automake-1.13 or newer, and libtool-2.2.7b or newer are now
required to rebuild the configure script and associated build environ-
ment. NOTE: In general, most people building GetData from a source
release don't need the tools to build GetData; the autotools are only
needed if changes need to be made to the configure script or Makefile
input files provided in the release or if building from the repository.
|=========================================================================|
New in version 0.8.9:
Library Changes:
* BUG FIX: The metadata writer now correctly stores bytes in the range
0x01 through 0x1F to the format files (encoded as hex escape sequences:
\x##).
* BUG FIX: A number of memory leaks associated with error returns from
library functions have been fixed.
* BUG FIX: Attempting to create a new field or alias with the name of
an existing dangling alias now fails with error GD_E_DUPLICATE, as it
should. Reported by Alexandra Rahlin.
* BUG FIX: Random-access reads on a RAW field with a frame offset no
longer result in mispositioning of the field's I/O pointer, which
previously would result in returning data from the wrong part of a
field. Reported by S. J. Benton.
* BUG FIX: The return value of gd_nframes() is now correct when using a
sample-index encoded field as the reference field.
* BUG FIX: The sample-index encoding no longer creates sequential records
with the same value.
Bindings Changes:
* F77 and F95 BUG FIX: Passing zero as the field code length (F77) to
GDFLSH GDSYNC GDRCLO, or, equivalently, an empty string as the field
code to fgd_flush, fgd_sync, fgd_raw_close now causes operation on all
fields, as with passing NULL to the corresponding C API functions does.
This is what the documentation said these functions did all along.
* PYTHON BUG FIX: Accessing the prefix or suffix member of fragment
objects no longer leaks memory if the other affix is non-NULL.
|=========================================================================|
New in version 0.8.8:
Library Changes:
* BUG FIX: The incorrect handling of the explicit 64-bit API declarations
in getdata.h (those associated with the GD_64BIT_API symbol), which
prevented GetData-0.8.7 from compiling on 32-bit systems, has been
fixed.
GetData-0.8.8 is ABI compatible with GetData-0.8.7.
|=========================================================================|
New in version 0.8.7:
Library Changes:
* BUG FIX: Opening a Dirfile read-write (GD_RDWR) no longer fails if
/INCLUDEd fragments are read-only. Reported by Alexandra Rahlin.
* BUG FIX: Several fixes have been made to the sample index encoding
(SIE) engine, which should now produce properly encoded data when
performing random writes.
Bindings Changes:
* F95 BUG FIX: Passing the empty string to fgd_reference() now returns
the current reference field without modifying it, instead of crashing.
* PYTHON BUG FIX: Querying dirfile.reference no longer causes a crash on
an empty dirfile. Reported by Alexandra Rahlin.
* PYTHON BUG FIX: Memory leaks have been fixed in functions returning
lists of field names or other metadata lists. Reported by Alexandra
Rahlin.
* PYTHON BUG FIX: On error, dirfile.get_string() no longer segfaults or
leaks memory.
|=========================================================================|
New in version 0.8.6:
Library Changes:
* BUG FIX: Computation of LINCOMs with complex valued input fields now
correctly happens in the complex plane. As a side effect,
gd_native_type() now also correctly reports such LINCOM fields to be
complex valued.
* BUG FIX: The gd_[m]add() functions now ignore zero-length scalar
strings. Previously they would store these invalid field codes,
causing problems later.
* BUG FIX: Returning complex-valued CARRAYs as purely real now works.
Previously only the first element requested would be returned, the
remaining output buffer containing uninitialised data.
* BUG FIX: Entry members spf, bitnum, numbits, and period are now com-
pletely ignored by gd_[m]add() when corresponding named scalars are
specified. Previously, an invalid value in these members would result
in the entry being rejected, even though the rest of GetData ignored
the invalid, unused value.
* BUG FIX: The parsing of the \x and \u escape sequences is now correct.
* BUG FIX: A scalar field code specified for the last factor in a POLYNOM
entry is no longer ignored by gd_[m]add().
* BUG FIX: gd_[m]add() no longer rejects MPLEX fields with negative
count_val.
* BUG FIX: DIVIDE fields with complex-valued divisors are now properly
computed.
* BUG FIX: Complex-valued POLYNOM and RECIP fields are now computed
properly when the library is built in ANSI C mode.
* BUG FIX: gd_alter_entry() no longer corrupts the DIRFILE when modifying
named scalars of MPLEX fields.
* BUG FIX: Writing complex-valued MPLEX fields no longer corrupts the
stored data.
* BUG FIX: gd_bof() now returns the correct number (i.e.: zero) when
reporting the beginning of field of derived fields shifted to before
the start of the dirfile. Previously, this function incorrectly
returned values ranging from zero to one less than the samples-per-
frame of the requested field.
* BUG FIX: gd_flush(), gd_sync(), gd_raw_close() no longer segfault when
operating on a LINCOM with only one input field.
* BUG FIX: gd_seek() now works correctly on PHASE fields; previously, the
sign of the PHASE shift was flipped.
* BUG FIX: gd_seek() now correctly positions the virtual I/O pointer of
the INDEX field.
* BUG FIX: gd_framenum_subset() now returns the correct value when passed
a field_end which is beyond the end of field, and then is required to
extrapolate outside of the specified frame range.
* BUG FIX: gd_error_string() now produces the correct string when
reporting an out-of-range poly_ord encountered by gd_add_polynom() and
similar.
* BUG FIX: gd_[m]alter_spec() no longer ignore co-efficients specified
for POLYNOM entries.
* BUG FIX: gd_alter_encoding() now deletes the internal cache of RAW
filenames of the affected fragment; previously, these old, cached
filenames could lead to I/O errors when reading and writing the re-
encoded RAW data files.
* BUG FIX: Calling the Legacy API function GetFormat() on a Dirfile with
MPLEX or WINDOW fields no longer results in a segmentation fault.
* BUG FIX: Attempts to read past the EOF of a gzipped field no longer
results in an I/O error, but successfully returns no data.
* BUG FIX: The internal recursion counter wasn't being properly reset on
certain error conditions, leading to spurious GD_E_RECURSE_LEVEL errors
being returned from valid calls.
Bindings Changes:
* F77 and F95 BUG FIX: Named scalar indices are now indexed from one
instead of zero, like all other array indices are in the Fortran
GetData bindings.
* C++ BUG FIX: Fixed segfault in RawEntry destructor. Reported by S. J.
Benton.
* C++ BUG FIX: Entry::ComplexScalars() now returns non-zero for RECIP
entries when appropriate, instead of always returning zero. (The
RecipEntry::ComplexScalars() method always returned the correct value.)
* IDL BUG FIX: The /UPDATEDB flag to gd_rename is no longer ignored;
also, the /MOVE_DATA flag no longer also acts as if /UPDATEDB had been
specified.
* PYTHON BUG FIX: The keyword for the "dirfile" parameter in the
getdata.fragment constructor is now properly spelled.
|=========================================================================|
New in version 0.8.5:
Library Changes:
* BUG FIX: The zzslim encoding framework, which was inadvertently
rendered non-compilable in 0.8.4, should once again work. Reported by
Matthew Hasselfield.
Bindings Changes:
* PYTHON BUG FIX: A spurious debugging message accidentally left in the
0.8.4 release has been expunged.
|=========================================================================|
New in version 0.8.4:
Dirfile Changes:
* CLARIFICATION: Inconsistent behaviour in the library, coupled with
contradictory statements in the Dirfile Standards, has resulted in
confusion over the meaning of the final, optional, parameter in a MPLEX
definition. To be clear: this parameter is simply the expected period
between successive occurrences of the "count" value in the index
vector. (It is used by GetData only to figure out a reasonable length
for the MPLEX lookback.) The following clarifications and corrections
have been made to the definition of the MPLEX field type in
dirfile-format(5):
- The final, optional parameter has been renamed from "max" to
"period".
- This parameter indicates not the maximal range of the index vector,
as previously stated, but the expected number of samples between
successive occurrences of the specified "count" value in the index
vector; as before, it should be regarded as a hint, and does not
place any actual restriction on the contents of the index vector.
- The incorrect requirement that the "count" parameter be non-negative
has been deleted.
- The incorrect requirement that "count" be less than or equal to
"max", if "max" is present and non-zero, has been deleted.
- The meaning of an omitted or zero "period", indicating that the
spacing of "count" in the index vector is unknown or non-uniform, is
now explicitly stated.
- Different MPLEX fields using the same index vector may specify
different periods.
See also the related library bug fixes below.
Library Changes:
* BUG FIX: The library no longer incorrectly rejects negative count_val
MPLEX parameters. Furthermore, a count_val of -1 in gd_alter_entry or
gd_[m]alter_mplex calls is not a special value: it just sets count_val
to -1.
* BUG FIX: The library no longer incorrectly rejects MPLEX fields where
count_val is greater than the period (formerly called count_max).
* BUG FIX: The default MPLEX period, if none is specified, is
2 * count_val + 1, not simply 2 * count_val, when count_val is greater
than 5.
* BUG FIX: Trying to position an I/O pointer to before sample zero with
gd_seek() now properly fails. Previously it would allow it, causing
bizarre things to happen later.
* BUG FIX: Using gd_tell to get the I/O pointer position of a derived
field with exactly two input fields (MULTIPLY, DIVIDE, MPLEX, WINDOW)
now properly reports an error in the "multiposition" case (ie. when the
two inputs are ultimately reading from different locations in the same
RAW field).
* BUG FIX: Trying to read sample zero of a gzipped RAW field the second
time now works as expected. Previously, the request to reposition the
I/O pointer back to zero would be ignored by the gzip framework.
Reported by Alexandra Rahlin.
* BUG FIX: The field code of a reference field declared in a subfragment
included with affixes is no longer corrupted by spurious application of
the affixes when the subfragment's metadata are (re-)written. Reported
by Seth.
API Changes:
* The count_max member of the gd_entry_t object has been renamed to
period. The corresponding dummy argument in various function proto-
types has been similarly renamed.
Bindings Changes:
* C++: The Entry and MplexEntry member functions CountMax and SetCountMax
have been renamed to Period and SetPeriod for consistency with the
changes listed above. CountMax and SetCountMax are still available as
aliases, but are marked deprecated.
* IDL BUG FIX: GD_ENTRY structures representing MPLEX fields are now
properly interpreted. Previously, the underlying gd_entry_t object was
being incorrectly initialised.
* PYTHON BUG FIX: Numpy arrays returned by dirfile.getdata() are now the
correct length. Previously they would always be the length requested
(or the length of the dirfile, if no length was explicitly given) even
if fewer samples were returned. Extra elements would contain unini-
tialised memory.
|=========================================================================|
New in version 0.8.3:
Library Changes:
* BUG FIX: Several bugs on big-ended systems have been fixed. Reported
by Dinar Valeev.
* BUG FIX: Adding an entry via gd_[m]add() with a negative CARRAY scalar
index no longer results in an internal error or worse when the added
field is later read. Instead, a proper error is returned.
* BUG FIX: Changing a CONST field to a real floating point type using
gd_alter_const() (or similar) no longer results in the value of the
field being corrupted.
* BUG FIX: A bug in the parser's tokeniser has been fixed to prevent
spurious "unterminated token" syntax errors. Typically this would only
be seen when using gd_strtok() to tokenise an (apparently innocuous)
user-supplied string, but a carefully crafted format file could also be
made to get gd_open() to produce it.
Bindings Changes:
* Bindings for MATLAB have been added.
* The IDL, Perl, and Python bindings no longer require a C99 compiler to
be built.
|=========================================================================|
New in version 0.8.2:
Library Changes:
* BUG FIX: A trailing symlink (i.e. situations where a symlink is the
last element of a path) no longer confuses GetData when the target of
the symlink: (1) is absolute, or (2) starts with "../". Reported by
S. J. Benton.
* BUG FIX: Trying to read data from the first sample of an MPLEX no
longer results in an internal error if the first sample of the index
doesn't match the target value.
|=========================================================================|
New in version 0.8.1:
Library Changes:
* gd_dirfilename() now returns a fully canonicalised version of the
dirfile path.
* BUG FIX: A segfault when negotiating symbolic links in file paths,
typically manifesting in gd_open calls, has been fixed.
* BUG FIX: gd_strtok now saves a copy of the string passed to it, as the
documentation suggests it should. Previously it cached the pointer
itself.
* BUG FIX: A number of minor memory leaks, mostly occurring when the
library encounters an error, have been fixed.
* BUG FIX: /HIDDEN directives weren't being written for aliases.
* BUG FIX: The parser now fails properly on /INCLUDEs which specify paths
which don't point to regular files.
* BUG FIX: Fixed a bug which would result in the parser getting confused
over the current Standards Version, potentially leading to rejected
valid dirfiles. Reported by Daniel Flanigan.
* BUG FIX: gd_alter_entry() and gd_madd_alias() weren't clearing the
Dirfile error before operation, resulting in them failing erroneously
in certain situations.
* BUG FIX: When including an existing fragment which itself has subfrag-
ments, gd_include() no longer returns the wrong fragment index.
* WIN32 BUG FIX: On Windows, the parser can now properly handle hexade-
cimal floating point.
API Changes:
* How the API deals with field code affixes has changed. Input vector
and scalar fields are reported by gd_entry(), &c. now include their
prefix and suffix, if any, which should remove the need to do manual
affix bookkeeping when reading metadata. The other side to this change
is that when modifying metadata (gd_add(), gd_alter_entry(), &c.),
supplied field codes must also contain the appropriate affixes.
* A new function, gd_linterp_tablename() has been added which returns a
fully canonicalised version of the look-up table pathname for a
LINTERP.
Bindings Changes:
* F77 BUG FIX: A memory leak has been fixed in GDALLC.
|=========================================================================|
New in version 0.8.0:
Dirfile Changes:
* Dirfile Standards Version 9 has been released. It adds two new field
types: MPLEX, which allows multiplexing multiple low-rate channels in
one high-rate one, and WINDOW, which extracts a portion of a field
based on the value of a second field. More changes are listed below.
* Alternate field names may be defined using the /ALIAS directive.
Aliases can't be used as a parent when defining metafields, but are
otherwise indistinguishable from a field's original ("canonical") name.
* The /VERSION directive now has fragment scope, which should alleviate
some of the problems associated with "version leakage" found in
previous versions. Previous versions are unchanged, but a "/VERSION 9"
directive will not propagate back up into a parent fragment, nor will
any /VERSION directive be propagated back up into a Version 9 fragment.
* Fields may be "hidden" using the /HIDDEN directive. The Standards
don't say what a "hidden" field does, but GetData will, by default,
exclude them from the count and list functions.
* Literal integers in the format may now be specified in octal (using
0####) or hexadecimal (using 0x##### or 0X#####) in addition to
decimal. C99-standard hexadecimal floating point literals (0x##.##p##,
&c.) are also accepted.
* The /INCLUDE directive can take two additional, optional parameters
which specify a prefix and/or suffix used to modify the entry names of
the fields defined in the included fragment. (The intent of this is to
permit the inclusion of multiple subdirfiles which define the same set
of fields, by modifying their field names.)
* Three new encoding schemes are available. The first is the Sample-
Index Encoding (SIE), similar to run-length encoding, useful for
compressing data which changes very rarely. Like the ASCII encoding,
GetData implements it internally, so it's always available. The other
two schemes (zzip and zzslim) are based around the ZZip library, an
access library for PKWARE ZIP files. These are unusual in that they
store all raw data in the same ZIP archive.
* The /ENCODING directive takes an optional second token. The zzip and
zzslim encoding schemes optionally use this token to specify the name
of the combined ZIP archive.
* The data type names FLOAT (an alias for FLOAT32) and DOUBLE (an alias
for FLOAT64) have been deprecated.
Library Changes:
* On POSIX systems, GetData now caches the dirfile directory (and any
other necessary subdirectories). This guards against third-parties
renaming the directory while GetData is interested in it.
* GetData is, in general, more careful about pathname management. At
least on POSIX systems, it will now try to canonicalise all paths it
comes across, including resolving symlinks. As a result, GetData no
longer becomes befuddled by a caller who opens a Dirfile with a
relative pathname and then calls chdir(2). Functions like
gd_fragmentname() and gd_raw_filename() now return canonicalised paths.
* The library no longer forces a sync of all raw data on close. The sync
causes a flush of filesystem buffers, which is usually unnecessary
overhead, and degredatious to efficient disk I/O. Use an explicit
gd_sync() or gd_flush() before closing to recover the old behaviour, if
desired.
* The error messages output by GetData with GD_VERBOSE turned on now has
"libgetdata: " prefixed to it. A further prefix may be specified by
the caller.
* When truncating a dirfile, the primary format file is no longer deleted
and then recreated. Instead, it is simply truncated in-place. This
change means that a Dirfile which is undergoing truncation always
appears (to GetData, at least) to be a valid Dirfile, which can be
useful if it is being concurrently read.
* Write support for gzip-encoded data has been added. Writes occur out
of place, which means sufficient space on the filesystem is needed for
a second copy of every field being written. Writing to a gzip
compressed field requires uncompressing the portion of the field before
the write of the existing file, recompressing it to the temporary file,
then writing the new data to the temporary file. A subsequent write at
a later position, will continue to write to the compressed file, but a
subsequent write to an earlier position requires coping all remaining
data from the old file to the temporary file, finalising it, moving it
over top of the old file, and then starting afresh with a new temporary
file. As a result, it's not very fast, and likely should be avoided
for non-sequential writes.
* Raw data files are now only opened for writing if writing is going to
occur. In particular, this means that opening a dirfile with GD_RDWR
will still work even if you don't have write access to it, so long as
you don't try writing to it.
* GetData now tracks a "current location" for all RAW files, and allows
reads and writes to occur at the "current location" (via the special
symbol GD_HERE). For some derived fields the idea of a "current
location" doesn't make sense, and this won't work.
* BUG FIX: In previous versions, renaming a field with gd_rename()
wouldn't update or invalidate the cache of fields which depended on the
old name, leading to surprising results. It will now either invalidate
these fields' metadata caches or else change the name in those
definitions as well.
* BUG FIX: The invalid syntax "/META parent/child grandchild ..." is now
properly rejected by the parser. Previously it half-worked, resulting
in a segfault in gd_close().
* BUG FIX: Several syntax errors encountered in PHASE and RECIP field
specifications weren't being reported by the parser, leading to
corrupted metadata on open.
* BUG FIX: GetData no longer omits the input field to a RECIP when
writing metadata to disk. Nor does it forget to put newlines after
RECIP or DIVIDE field specifications.
* BUG FIX: GetData no longer assumes all LINCOM input fields have the
same number of samples per frame.
* BUG FIX: Writing an empty fragment with GD_PRETTY_PRINT turned on no
longer results in a floating-point exception.
* BUG FIX: Renaming a field now properly flags the fragment which
contains it as dirty, which will ensure the rename is written to disk
when the metadata are next flushed.
* BUG FIX: Whitespace within tokens is now properly escaped when metadata
is flushed to disk.
* BUG FIX: The parser no longer gets tripped up by two character escape
sequences in a row, nor by an escape sequence at the start of a token.
* BUG FIX: "/PROTECT data" directives are now being written when they
should be. Previously, they were dropped completely. Reported by
Alexandra Rahlin.
* BUG FIX: CARRAY indices provided to gd_add() in the entry.scalar_ind
member are no longer ignored. Reported by S. J. Benton.
* BUG FIX: When adding a metafield with gd_(m)add(), GetData no longer
checks the protection of the fragment indexed by entry->fragment_index.
Instead it ignores entry->fragment_index completely (as the
documentation indicates it should) and checks the protection of the
fragment containing the parent field.
* BUG FIX: calling gd_putdata() with num_frames and num_samples both zero
no longer confuses GetData: instead it simply does nothing.
API Changes:
* Functions which add fields (gd_add(), gd_add_<type>(), &c.) can now be
used to add metafields by using their full "parent/name" field code.
The old, explicit metafield adding routines (gd_madd(), &c.) are still
available.
* As with their corresponding names in the Standards, the gd_type_t
symbols GD_FLOAT and GD_DOUBLE are now deprecated. Use GD_FLOAT32 and
GD_FLOAT64 as appropriate.
* A new open flag, GD_TRUNCSUB, will cause GD_TRUNC to descend into
subdirectories of a directory it is truncating. If not specified along
with GD_TRUNC, it does nothing.
* New public functions gd_nentries() and gd_entry_list() provide a
generic interface to the field counting and list functionality. The
other functions (gd_nfields(), gd_nfields_by_type(), &c. and
gd_field_list(), gd_field_list_by_type(), &c.) are now simply special
cases of these two functions.
* The "hiddenness" of a field name may be queried with gd_hidden(), set
with gd_hide() and cleared with gd_unhide().
* /INCLUDE affixes may be queried with gd_fragment_affixes() and added,
modified or deleted with gd_alter_affixes(). A new fragment can be
added to the dirfile with affixes using gd_include_affix().
* New aliases may be added with gd_add_alias() or gd_madd_alias. The
target of an alias is returned by gd_alias_target(); a list of aliases
of a field, or their number, is returned by gd_aliases() and
gd_naliases().
* Because GetData considers aliases and canonical names to be
indistinguishable, passing an alias name to most functions will result
in the function operating on target field. To manipulate aliases
themselves, the API has the new interfaces gd_move_alias() and
gd_delete_alias().
* In addition to the special symbol GD_HERE which may be passed to
gd_getdata() and gd_putdata(), the "current location" of fields may be
queried with gd_tell() and moved with gd_seek().
* The typedefs gd_bit_t and gd_spf_t have been deprecated. They are
replaced with 'int' and 'unsigned int' respectively. For backwards
compatibility, the typedefs (with their new types) are still declared
in getdata.h, but the API no longer makes use of them.
* GetData's tokeniser has been exposed to the public API via the
gd_strtok() function, which works similar to strtok(3), and returns
successive tokens of a string given it.
* A new function gd_error_count() returns the number of GetData errors
encountered on a Dirfile since it was last called (or since the Dirfile
was opened). This is useful for programs that care whether *something*
failed, but don't really care what it was.
* Reading MPLEX fields can result in inefficiencies due to GetData having
to look backwards for the first value of the part of the derived field
requested. A new function, gd_mplex_lookback(), can be used to alter
how much lookback is done, if any.
* The "move_data" flag in gd_rename() has been replaced with a generic
"flags" parameter. Two rename flags are defined: GD_REN_DATA, which
recovers the old behaviour of move_data, and GD_REN_UBDB which will
cause references to the renamed field to be renamed also in derived
fields which refer to it. Without GD_REN_UBDB, these definitions are
left unchanged.
* Flags which affect the long-term operation of GetData may be modified
after open using gd_flags(). Currently these flags are just GD_VERBOSE
and GD_PRETTY_PRINT.
* The gd_flush() function, which both syncs and closes RAW files, has
been broken up into its two parts with two new functions: gd_sync(),
which just syncs and gd_raw_close(), which just closes. gd_flush()
still does both.
* Using the new gd_desync() function will cause GetData to determine
whether the Dirfile metadata on disk has changed since the Dirfile was
opened.
* A prefix to the automatic error messages printed by GetData when using
the GD_VERBOSE flag may be specified with gd_verbose_prefix().
* When requesting the exclusive creation of a new dirfile with (GD_CREAT
| GD_EXCL) GetData will now return the new error GD_E_EXISTS if the
dirfile already exists. Previously it would return GD_E_CREAT in this
situation.
* The error codes GD_E_BAD_ENDIANNESS, GD_E_BAD_PROTECTION, and
GD_E_BAD_VERSION have been combined into a more generic error code:
GD_E_ARGUMENT. The old symbols are retained for backwards
compatibility but have been marked deprecated.
* CONST and CARRAY handling has been combined somewhat. When passed to
{get,put}_carray[_slice]() or gd_carray_len(), CONSTs behave like
length-one CARRAYs. Similarly, {get,put}_constant() will operate on
the first element of a CARRAY, as if it were a CONST field.
* C89 API: Passing NULL to gd_alter_crecip() for cdividend is now treated
as if it were passed zero (ie. it indicates no change for cdividend).
* BUG FIX: gd_spf() now returns GD_E_DIMENSION when passed a scalar field
code, as the documentation says it should. Previously it returned
GD_E_BAD_FIELD_TYPE.
Legacy API Changes:
* The never-used n_mplex/mplexEntries part of the FormatType now has a
use: both MPLEX and WINDOW entries are reported there.
Bindings Changes:
* Bindings for Perl5 have been added. Perl is not well suited to
numerical analysis, and the bindings may be inefficient.
* F77: Due to small namespace issues, the encoding parameters
(corresponding to the GD_...ENCODED symbols in the C API) have all been
renamed. They are now of the form GDE_xx.
* C++: The bindings now make use of the explicitly-64-bit offset type
gd_off64_t defined in getdata.h. Previously, the bindings tried to
force off_t to be 64-bit through preprocessor shenanigans, which led,
in certain cases, to linking problems down the road when used by third
parties.
* Python: The return_type, num_fields, and num_samples parameters to
dirfile.getdata() are now optional. The return_type defaults to
pygetdata.FLOAT, and if neither num_fields nor num_samples are given,
all frames (ie. the value of dirfile.nframes) are returned.
* IDL: A number of undocumented, unnecessary function aliases have been
removed. They should be replaced with their canonical names, without
change in use:
Alias Replacement
---------------------- ----------------
GD_ADD_CLINCOM GD_ADD_LINCOM
GD_ADD_CPOLYNOM GD_ADD_POLYNOM
GD_ADD_CRECIP GD_ADD_RECIP
GD_ALTER_CLINCOM GD_ALTER_LINCOM
GD_ALTER_CPOLYNOM GD_ALTER_POLYNOM
GD_ALTER_CRECIP GD_ALTER_RECIP
GD_FIELD_LIST_BY_TYPE GD_ENTRY_LIST
GD_FRAMENUM_SUBSET GD_FRAMENUM
GD_MADD GD_ADD
GD_MADD_BIT GD_ADD_BIT
GD_MADD_CARRAY GD_ADD_CARRAY
GD_MADD_CLINCOM GD_ADD_LINCOM
GD_MADD_CPOLYNOM GD_ADD_POLYNOM
GD_MADD_CRECIP GD_ADD_RECIP
GD_MADD_DIVIDE GD_ADD_DIVIDE
GD_MADD_LINCOM GD_ADD_LINCOM
GD_MADD_LINTERP GD_ADD_LINTERP
GD_MADD_MULTIPLY GD_ADD_MULTIPLY
GD_MADD_PHASE GD_ADD_PHASE
GD_MADD_POLYNOM GD_ADD_POLYNOM
GD_MADD_RECIP GD_ADD_RECIP
GD_MADD_SBIT GD_ADD_SBIT
GD_MADD_STRING GD_ADD_STRING
GD_MALTER_SPEC GD_ALTER_SPEC
GD_MCONSTANTS GD_CONSTANTS
GD_MFIELD_LIST GD_ENTRY_LIST
GD_MFIELD_LIST_BY_TYPE GD_ENTRY_LIST
GD_MSTRINGS GD_STRINGS
GD_MVECTOR_LIST GD_VECTOR_LIST
GD_NFIELDS_BY_TYPE GD_NENTRIES
GD_NMFIELDS GD_NENTRIES
GD_NMFIELDS_BY_TYPE GD_NENTRIES
GD_NMVECTORS GD_NVECTORS
* BUG FIX: All bindings now provide a named constant corresponding to the
C API error GD_E_UNKNOWN_ENCODING, which was overlooked in previous
releases.
* IDL BUG FIX: The missing /IGNORE_REFS and /PRETTY_PRINT are now
available in gd_open.
|=========================================================================|
New in version 0.7.3:
Library Changes
* BUG FIX: Passing a zero dividend to alter_crecip() and alter_recip()
now results in the dividend not changing, per documentation.
* BUG FIX: GetData wasn't properly computing the module directory,
causing encodings using external modules to fail with GD_E_UNSUPPORTED.
* BUG FIX: Metafield records are now properly stored in the format file
for Standards Version 8.
* BUG FIX: gd_raw_filename() wasn't clearing the Dirfile error before
operation, resulting in it failing erroneously in certain situations.
Utilities Changes:
* BUG FIX: A potential segmentation fault has been patched in
dirfile2ascii.
|=========================================================================|
New in version 0.7.2:
Library Changes
* If built with modules, the plugin shared objects will now be installed
in a separate directory (by default "${libdir}/getdata", but see the
--with-module-dir option in ./configure).
* BUG FIX: When using modules, a missing plugin no longer results in a
lock up the second time an attempt is made to find it.
* BUG FIX: GetData now properly processes /INCLUDE directives with
absolute paths, as the documentation insists it should.
* BUG FIX: gd_add()ing the first raw field in a dirfile to a subfragment
no longer results in memory corruption when calling gd_close().
API Changes
* If called with GD_CREAT but no encoding specified, gd_include() now
will duplicate the encoding of the parent fragment (if any).
Miscellaneous
* The python module install dir can now be changed by passing
--with-python-module-dir to configure.
|=========================================================================|
New in version 0.7.1:
Library Changes
* BUG FIX: When the parser fails in a gd_include() call, the internal
list of conformant Standards Versions for the loaded dirfile is no
longer corrupted.
* BUG FIX Performing two consecutive reads of the same field in an ASCII
encoded datasources no longer confuses GetData.
* BUG FIX: Numerous memory leaks have been patched (mostly pin-holes,
although there are a few, rare but egregious ones).
API Changes:
* The value(s) of a CONST or CARRAY field added with gd_[m]add() is now
guaranteed to be zero; previously, it was unspecified.
* BUG FIX: Specifying both GD_VERBOSE and GD_IGNORE_DUPS no longer
results in spurious "Field code already defined" messages when dupli-
cate fields are encountered.
* BUG FIX: Calling gd_delete on a metafield no longer results in a
segfault.
Bindings Changes:
* F77 BUG FIX: A memory leak in GDASCA has been fixed.
* Python BUG FIX: The first element of a python list (instead of a NumPy
array) is no longer dropped when passed to pygetdata on 64-bit systems.
Utilities Changes:
* dirfile2ascii now pads reads past the end-of-field. The value used to
pad may be specified with '-z', and defaults to 0/NaN. Previously, the
value of uninitialized memory was output.
* BUG FIX: dirfile2ascii now does what the user intends when presented
with cart-before-horse arguments, ie. "dirfile2ascii -e field DIRFILE".
Previously this resulted in confusion.
* BUG FIX: dirfile2ascii now accepts field conversion '-u', which was
documented but overlooked in the argument parser.
|=========================================================================|
New in version 0.7.0:
Dirfile Changes
* Dirfile Standards Version 8 has been released. It adds three new field
types: DIVIDE, which computes X/Y for vectors X and Y, RECIP, which
computes A/X for vector X and scalar A, and CARRAY, which is an array
of CONST scalars. A few other changes are listed below.
* Like CONST fields, CARRAY elements can be used as parameters in the
definition of other fields. Syntax is 'field<n>', where 'field' is the
field name and 'n' the element index in the CARRAY (and the angle
brackets are literal angle brackets). If the '<n>' is omitted, the
first element (numbered 0) is assumed.
* Slashes on reserved words are now mandatory, allowing fields with
reserved words as names.
* The long-deprecated single character data type codes have been removed
from Standards Version 8. (They are still accepted by GetData, which
handles all previous Standards Versions as well.)
* The /ENDIAN directive now takes an optional second parameter. The only
allowed value for this second token is "arm", which indicates that
double precision floating point data (including double precision
complex data) are stored in the ARM middle-endian format. Without
this, floating point data are assumed to have the same byte sex as
integer data.
Library Changes
* GetData, including its bindings, is now supported under MacOS X,
Cygwin, and Win32 using MinGW. In the case of MinGW, this includes
linking against the Microsoft C Runtime, which isn't POSIX compliant.
Users should exercise caution when writing to dirfiles in this case.
* GetData can now be used on ARM processors. As a side-effect, GetData
can now read and write the middle-ended floating point format used by
older ARM processors (on any processor, via endianness conversion).
* Some changes have been made to the parser. It now operates in one of
two modes: pedantic mode, which adheres strictly to one particular
Dirfile Standards Version, and permissive mode (the default) which is
more lenient, as the old parser was. Encountering a /VERSION directive
will switch the parser from permissive to pedantic mode unless told not
to (see GD_PERMISSIVE below), and will change the particular Standards
Version to which pedantic mode is adhering.
* As a result of the previous change, the Standards compliance of the
parser in strict mode has been greatly increased. This affects allowed
characters in field names, and field name lengths. The existence or
lack thereof of the implicit FILEFRAM field (an alias for INDEX) is
also affected.
* Since Standards Versions 4 and earlier were never codified, we've made
some assumptions as to their behaviour in certain undocumented situta-
tions. Most noticeably, they still prohibit ASCII control codes (bytes
0x01 through 0x1F) and '/' in field names. We also assume these
dirfiles have native byte sex, even though some early GetData behaviour
implied only little-ended data was acceptable.
* Every Dirfile object now records its current Standards Version, which
affects functions such as gd_add_spec(). The Version will start out as
the value of the last /VERSION directive encountered by the parser, or
else the latest compliant Standards Version. It can be changed (see
below).
* When writing metadata to disk, fragments will be written according to
the current Standards Version of the dirfile object, rather than always
using the latest version, as before.
* The C library now sports an "ANSI C" mode, which allows compilation of
GetData without a C99-compliant compiler. Some functionality may be
lost, most prominently, but unsurprisingly, the default (C99) API.
* The artificial maximum line length of 4096 bytes for format metadata
has been removed. The maximum line length is now 2**31 bytes on 32-bit
systems, and more on 64-bit systems. This also has the side-effect of
removing the same length limit which had been applied to field names.
* LINTERP tables are no longer required to be sorted. The library will
sort them, if needed. As a side-effect of this, tables are no longer
read twice when loading, leading to potential speed-up.
* BUG FIX: The library wasn't properly checking that the second and
higher input fields in LINCOM and MULTIPLY fields were, in fact,
vectors, leading to possible segfaults.
* BUG FIX: A memory leak associated with modifying LINCOM metadata has
been fixed.
* BUG FIX: Using a complex valued vector as the second input field in a
MULTIPLY wouldn't result in the field being internally flagged as
complex, leading to loss of the imaginary part in certain situations.
* BUG FIX: Trying to write to a LINTERP via a table which isn't monotonic
now results in error. Previously, this was allowed, with arbitrary
data stored to disk.
* BUG FIX: On encountering a line longer than it was prepared to deal
with, the parser used to get very confused, potentially resulting in
bizarre behaviour. On the off chance such a line is encountered,
GetData will now raise GD_E_LINE_TOO_LONG in these cases. (And, since
the maximum line length has been increased to at least 2**31 charac-
ters, this should only occur in pathological situations.)
* BUG FIX: Deleting a metafield was not being communicated to its parent,
resulting in differing opinions within the library as to the deleted
field's existence.
API Changes
* Due to namespace conflicts with the MacOS X System Library (which
contains the C Standard Library), a wholesale renaming of public
symbols has been performed. All public functions, and data types now
begin with 'gd_'. All public preprocessor macros now begin with 'GD_'.
Symbol renaming has occurred in the following manner:
- functions which started with 'put' simply prefix 'gd_' to their name.
So put_constant() becomes gd_put_constant().
- functions which started with 'get' replace 'get' with 'gd', except
those functions with a corresponding 'put' function; these simply
prefix 'gd_'. So, get_constants() becomes gd_constants(), but
get_constant() becomes gd_get_constant().
- functions which started with 'dirfile' replace 'dirfile' with 'gd',
except for dirfilename(), which becomes gd_dirfilename(). So,
dirfile_alter_encoding() becomes gd_alter_encoding().
- macros which started with 'GETDATA_' now start with 'GD_'. Those
that started with 'NO_GETDATA_' now start with 'GD_NO_'. So
GETDATA_LEGACY_API becomes GD_LEGACY_API and NO_GETDATA_LEGACY_API
becomes GD_NO_LEGACY_API.
* The above renaming rules result in one gd_ namespace clash. As a
result, get_reference() has been removed completely. Calls will have
to be changed to use gd_reference(), formerly dirfile_reference().
* Furthermore, the function dirfile_protect() is now
gd_alter_protection() to match the naming of other fragment metadata
altering functions.
* To ease transition to the new namespace, defining GD_FUNCTION_ALIASES
before including getdata.h will create a bunch of preprocessor macros
which map the old names to the new.
* The meaning of the GD_PEDANTIC flag to gd_[cb]open() has changed. It
now forces the parser to start off in pedantic mode (see above),
following the latest Standards Version, rather than the default
permissive mode. This is a subtle change, and in most cases the
effects of GD_PEDANTIC will be the same as they were before.
* A new dirfile flag, GD_PERMISSIVE, has been added. Passing this to
gd_[cb]open() will prohibit the parser from switching into pedantic
mode when it encounters a /VERSION directive. This flag is needed to
read dirfiles which claim to adhere to the Standards via a /VERSION
directive, but contain invalid syntax nonetheless.
* The current Standards Version of an open dirfile can be set or queried
by calling the new function gd_dirfile_standards(). NB: this only
affects the open dirfile, not the metadata stored on disk.
* A new function, gd_rewrite_fragment() will force GetData to write the
specified fragment to disk, even if no metadata in the fragment has
changed.
* Two new functions, gd_bof() and gd_eof() can be used to find the sample
number of the beginning- or end-of-field for a given field. Notably,
these can be used to calculate the amount of raw data on disk for a
given field.
* The new gd_invalid_dirfile() function will return a newly created,
invalid dirfile. Primarily useful to callers as a "null dirfile"
placeholder.
* GD_E_OPEN_INCLUDE has been renamed to GD_E_OPEN_FRAGMENT, but the
former symbol remains as an alias.
* Two new bitflags to gd_cbopen and gd_open: GD_ARM_ENDIAN and
GD_NOT_ARM_ENDIAN allow specifying the default ordering of double pre-
cision data. These flags may also be used with gd_alter_endianness()
and will be returned by gd_endianness(). On all platforms except for
middle-endian ARMs, GD_NOT_ARM_ENDIAN equals zero and may be omitted.
* Where gd_flush() and gd_metaflush() in the past raised
GD_E_OPEN_INCLUDE on I/O error, they now raise the new GD_E_FLUSH.
* All functions now raise GD_E_DIMENSION when encountering a scalar field
when expecting a vector. In the past, the error value returned was
different for different functions.
* getdata.h now wraps itself in an extern "C" block if included by a C++
compiler.
* If passed a NULL pointer, gd_error_string() will return a newly
malloc'd buffer long enough to hold the entire string. It should be
free'd by the caller.
* The parser data passed to the callback function now includes the length
of the buffer. The parser also permits the callback to assign a new
pointer to the 'line' element of the parser data if the buffer passed
in is too small.
* gd_metaflush() now fails and raises GD_E_ACCMODE if called on a
read-only dirfile. (Previously it would successfully do nothing.)
Legacy API Changes
* In the legacy format struct, DIVIDE fields are listed as MULTIPLYs and
RECIP fields are listed as LINCOMs, with the dividend stored in m[0].
* BUG FIX: In previous versions, attempting to access an invalid dirfile
(anything that would cause gd_open to fail) resulted in leaked memory.
This leak has been plugged.
* BUG FIX: A segfault involving the reporting of POLYNOM fields has been
fixed in GetFormat().
Bindings
* Bindings for the functionality introduced into 0.7.0 have been added.
* F77: GDPROT has been renamed GDAPRT to mirror the renaming of
gd_alter_protection().
* F77: GDGTCO and GDPTCO no longer return an indicator of success.
* F77: The signatures of GDGSCA and GDASCA have changed to return/take
the scalar index as well.
* F95: For consistency sake, the Fortran 95 bindings have undergone the
same renaming so they have the same name as their C counterparts. All
functions are now preceded by 'fgd_'.
* F95: The fgd_get_constant_* and fgd_put_constant_* procedures are now
subroutines, not functions.
* C++: Const member functions are now flagged as such.
* C++: The len parameter to Dirfile::ErrorString is now ignored.
* C++: dirfiles are now opened read-only by default.
* IDL: The IDL bindings have also followed suit in the great renaming.
The exception to this is GETDATA_CONSTANTS(), which is not renamed.
* IDL: Various constants defined in getdata.h but not used by the IDL
bindings have been removed from the structure returned by
GETDATA_CONSTANTS().
* Python: dirfile.getdata() now raises ValueError if neither num_frames
nor num_samples are specified. In the past, it would happily return
nothing in this case.
* Python: dirfiles are now opened read-only by default.
* C++ BUG FIX: Entry::Scalar() no longer rejects valid values of index.
* IDL BUG FIX: A memory impropriety involving keyword arguments. This
bug manifested by rejecting valid keyword arguments on 64-bit systems,
but may have had other, subtle effects on 32-bit systems as well.
* Python BUG FIX: A non-literal phase shift is now properly reported.
* Python BUG FIX: Calling Dirfile.discard() or Dirfile.close() no longer
results in a segfault.
Miscellaneous
* The --disable-checkdirfile option has been removed from ./configure.
It is now always installed.
* A utility, dirfile2ascii, is now installed alongside checkdirfile.
This utility converts all or part of a dirfile into ASCII text.
dirfile2ascii was written by Matthew Truch.
* The thoroughness of the checks done by checkdirfile have increased. It
now reports format syntax errors, Dirfile Standards compliance,
problems with field definitions (such as missing input fields), and the
size of the dirfile.
|=========================================================================|
New in version 0.6.3:
Bindings
* A memory leak has been plugged in the Python bindings manifesting in
calls to dirfile.getdata() returning a NumPy array. In previous
versions, the memory used by the returned array to hold the data was
not reclaimed by GC.
|=========================================================================|
New in version 0.6.2:
Library Changes
* Attempting to get the number of frames from an ASCII file no longer
fails the second time.
|=========================================================================|
New in version 0.6.1:
API Changes
* The getdata.h header no longer includes complex.h explicitly.
Legacy API Changes
* A potential segfault relating to error reporting in the legacy API has
been fixed.
Bindings
* C++ BUG FIX: Several bugs preventing compilation of the C++ test-suite
under Fedora 13 have been fixed.
* IDL BUG FIX: Several segfaults in the IDL bindings have been
eradicated.
Miscellaneous
* Libtool has been upgraded to 2.2.6b, which provides proper support for
Fortran-9x. The outdated, internal libltdl has been removed. Building
modules now requires an installed libltdl.
|=========================================================================|
New in version 0.6.0:
License
* The GetData library and its bindings are now distributed under the GNU
Lesser Public License, version 2.1 or later.
Dirfile Changes
* GetData now support Dirfile Standards Version 7. Standards Version 7
adds complex data types, a signed bitfield (SBIT), a polynomial derived
field (POLYNOM), and other features detailed below.
* Metafields can now be specified using a normal format specification
line and their full (slashed) field code.
* Field codes can now contain a "representation suffix", which allows
specifying a complex norm used to convert complex valued data into
purely real data.
* The n_fields parameter in a LINCOM specification is now optional.
* A new encoding scheme which supports LZMA compressed data using the .xz
container format has been added.
* Look-up tables may contain a complex y-value. This prohibits writing
to the associated LINTERP field.
Library Changes
* The field parameters spf (RAW), bitnum and numbits (BIT and SBIT) are
now restricted to 16-bit numbers, which should be more than enough.
The public API uses gd_spf_t and gd_bit_t for these quantities.
* The shift field parameter is now a 64-bit integer type, called
gd_shift_t.
* The parser callback has received a new, API breaking signature, which
should hopefully reduce the amount of API breakage in the future. It
is also now possible for the caller to pass data through GetData to the
callback, if desired.
* Computation of derived fields can now handle complex valued inputs, and
computation will occur in the complex plane when required.
* The computation of LINCOM fields has been re-written and is now ~20%
faster for LINCOMs with 2 or 3 input fields. (No change for LINCOMs of
1 field.)
* A new open flag, GD_PRETTY_PRINT, tells the library to attempt to write
a nicer looking format file. What this specifically means, isn't part
of the public API and shouldn't be relied on in portable code.
* BUG FIX: dirfile_madd_spec() and dirfile_add_spec() didn't properly
flag the fragments they modified as dirty, leading to possible loss of
metadata.
* BUG FIX: calling get_nframes on a bzip2 encoded dirfile would report
the wrong dirfile length.
* BUG FIX: LINTERP tables are now closed after loading, resolving a file
descriptor leak of great age (ie. pre-0.3.0).
* BUG FIX: When re-writing a format file fragment containing fields with
CONST scalar field parameters, GetData no longer replaces the CONST
field codes in the field specification lines with the value of the
CONST field.
* BUG FIX: when moving a field with dirfile_move(), the fragment index in
the entry objects of metafields defined for the field were not updated.
(The metafields were placed in the correct fragment when the metadata
was flushed to disk, despite this.)
* BUG FIX: Calling getdata() or putdata() on 64-bit architectures in
certain situations involving a non-zero frameoffset no longer results
in bizarrity.
* BUG FIX: A memory leak associated with RAW field metadata has been
fixed.
* BUG FIX: 64-bit integer literals in the format file are no longer
truncated to double precision when read.
API Changes
* This release breaks ABI compatibility and, to a lesser extent, API
compatibility. The SOVERSION of the library has been incremented
accordingly.
* The dirfile flags, including encoding and endianness symbols are now
explicitly long ints. Previously these quantities mixed long int with
int arbitrarily. This affects the public API.
* A C-89 API will be declared by getdata.h if GETDATA_C89_API is defined
before inclusion. If this symbol is not defined, the C API defined
will include C-99 conforming complex types, and getdata.h will include
complex.h if `complex' has not been defined before inclusion.
* Due to the new, simple way of specifying META fields in Standards
Version 7, dirfile_add_spec() and dirfile_alter_spec() can be used to
modify metafields. dirfile_madd_spec() and dirfile_malter_spec() remain
as well.
* An array containing the CONST scalar field codes used when non-literal
parameters are used in the specification of various different fields
has been added to the gd_entry_t object. As a side-effect of this,
dirfile_add(), dirfile_alter_entry(), and dirfile_madd() can be used to
manipulate non-literal field parameters.
* A function, get_native_type(), now exists which returns the underlying
data type of a specified field code.
* A function, get_framenum(), is now available to perform a "reverse
look-up" on a field: given a data value, it reports the frame number
where the field has that value. The field must be monotonic.
* A function, dirfile_validate(), now exists which can be used to check
the validity of a field code before use.
* Two new error codes have been added: GD_E_DOMAIN, potentially returned
by get_framenum(), and GD_E_BAD_REPR, indicating an unrecognised
representation suffix, or an attempt to write to a representation.
* A new syntax error suberror code has been added: GD_E_FORMAT_LITERAL,
which indicates a syntax error in a complex literal number.
* BUG FIX: On success, dirfilename(), dirfile_reference(), and
get_reference() didn't change the error code to GD_E_OK, resulting in
spurious error codes from these functions.
* BUG FIX: dirfile_madd_string() and dirfile_madd_const() now don't fail
erroneously by reporting GD_E_BAD_CODE.
* BUG FIX: error strings relating to GD_E_DELETE error codes now report
the proper error information.
* BUG FIX: the list returned by get_mstrings() is now properly NULL
terminated.
* BUG FIX: the entry->table member now reports the proper path to the
LUT. (That is, the path stored in the format file.) Previously it
incorrectly reported a path relative to the base dirfile directory.
Legacy API Changes
* Because they would otherwise be unreported, in the structure returned
by GetFormat(), POLYNOM entries are reported as LINCOM entries, by
discarding higher order terms. Similarly, SBIT entries are reported as
BIT entries.
Bindings
* Bindings for Python and the Interactive Data Language (IDL) have been
added.
* Bindings for the functionality introduced into 0.6.0 have been added.
* C++: The missing bindings for dirfilename(), get_nvectors(), and
get_fragment_index() have been added.
* C++: Empty entry class constructors have been added which allow delayed
specification of field metadata.
* C++: The dirfile.h header now includes all other C++ headers, and is
now the only header required to be included in applications. (The
other headers can still be included, with no ill effects.)
* All bindings have a much expanded test suite, which has resulted in the
discovery of many bugs (subsequently fixed).
* F77: The callback function is now dirfile-specific, as it is in the C
API.
* C++ BUG FIX: Entry class constructors now properly set the field name
for metafields.
* C++ BUG FIX: Fragment::SetEncoding() now actually does what it adver-
tises, instead of doing nothing.
* C++ BUG FIX: The recode argument to various functions is now optional
and defaults to zero, as the documentation already indicated.
* C++ BUG FIX: dirfile_close() was not being called by the Dirifle
destructor.
* F77 BUG FIX: a NULL C string returned from the C API now returns an
empty string, rather than causing a segfault.
* F77 BUG FIX: attempting to call another F77 function before calling
GDCOPN or GDOPEN no longer results in a segfault.
* F77 BUG FIX: passing an out-of-range dirfile unit no longer results in
undefined behaviour.
* F77 BUG FIX: the last character of a string returned by the bindings is
no longer deleted.
* F77 BUG FIX: string lengths now report the correct length.
* F77 BUG FIX: passing dirfile unit 0 to GDCLOS or GDDSCD no longer
corrupts the bindings.
* F77 BUG FIX: GDFLDN no longer accepts indices less than one.
* F77 BUG FIX: GDESTR now space pads its output string, rather than
filling the remainder of the string with garbage.
* F77 BUG FIX: The get_entry() bindings, GDGExx no longer segfault on
dirfile error.
* F77 BUG FIX: GDGTST and GDPTST now report the FORTRAN string length,
which is one less than the C API would report, due to the lack of a
terminating null.
* F95 BUG FIX: Attempting to retrieve a PHASE entry with fget_entry() now
works.
* F95 BUG FIX: Several routines which were named fdrifile_<foo> are now
properly spelled.
Miscellaneous
* All bindings can now be disabled by passing --disable-bindings to
./configure.
|=========================================================================|
New in version 0.5.0:
Dirfile Changes
* Support for two new encoding schemes has been added which handle bzip2
and gzip compression. Like the slim encoding, the bzip and gzip encod-
ing schemes currently do not support writing, but do allow reading
dirfile data compressed with the standard gzip and bzip2 utilities.
Reading compressed data is unsurprisingly slow.
Library Changes
* Encoding schemes relying on external libraries (slim, gzip, bzip2) may
now be built into stand-alone library modules which will be loaded, as
needed, at runtime. GetData will fail gracefully if modules are
missing. See the README for full details.
* BUG FIX: Writing metadata to disc now preserves the permissions of the
format file fragments which are changed.
* BUG FIX: Format files without a line feed character at the end of the
file no longer cause the parser to segfault.
* BUG FIX: putdata() now reports the correct number of samples written
for ASCII encoded files.
* BUG FIX: GetData no longer inserts unnecessary "/./" elements into the
paths it reports.
API Changes
* The caller can now register a callback function with a DIRFILE by
opening the dirfile with dirfile_cbopen instead of dirfile_open. This
callback function will be called by GetData whenever the format file
parser encounters a syntax error. The callback function can be used by
the caller to either correct the syntax error, or else tell GetData
whether to ignore the line or not. The callback function may be later
modified by calling dirfile_parser_callback.
* A fragment may be removed from a dirfile by calling dirfile_uninclude.
* The pathname of the dirfile may be retrieved by calling dirfilename.
* The reference field for the dirfile may be set or retrieved by calling
dirfile_reference.
* Fragment metadata may be queried or modified by calling:
- get_encoding, dirfile_alter_encoding
- get_endianness, dirfile_alter_endianness
- get_frameoffset, dirfile_alter_frameoffset
- get_protection, dirfile_protect
- get_parent_fragment
* Various functions now exist to modify field metadata:
- dirfile_alter_entry, dirfile_alter_<field-type>,
dirfile_alter_spec, dirfile_malter_spec
* A field may be moved to a different format file fragment using
dirfile_move.
* A field may be renamed by using dirfile_rename.
* A field may be removed from the dirfile by calling dirfile_delete.
* The type or fragment index of a field may be retrieved by calling
get_entry_type or get_fragment_index, respectively.
* The pathname of a raw field may be obtained from a call to
get_raw_filename.
* A DIRFILE may be deallocated without saving modified metadata by
calling dirfile_discard.
* A new flag, GD_IGNORE_DUPS, may be passed to dirfile_open to tell the
parser to ignore duplicate field specifications in the format files.
This is really only useful identically duplicate specifications, since
there is no indication of which of the duplicates is honoured.
Explicit control can be obtained by handling this inside a caller
supplied callback function.
Bindings:
* All functions in the Fortran 77 bindings have been completely renamed
from GDFxxx to GDxxxx to provide a larger namespace for our use.
* Fortran 77, Fortran 95, and C++ bindings have been updated for the
latest API changes.
* BUG FIX: Fortran 77 functions which return character strings no longer
corrupt memory as a side-effect of operation. This also affected the
Fortran 95 bindings.
Miscellaneous:
* BUG FIX: The dirfile_madd_bit(3) manual page has been corrected to show
the correct order or parameters for all the dirfile_madd_<field_type>
functions.
|=========================================================================|
New in version 0.4.2:
Library Changes
* BUG FIX: The default protection level of format file fragments is now
properly set to "none". Reported by Brendan Crill.
* BUG FIX: getdata() now reports the correct number of samples read for
slim and ASCII encoded files. Reported by Adam Hincks.
* BUG FIX: A segmentation fault only encountered when reading slim
encoded data has been fixed in dirfile_flush().
Legacy API Changes
* Several fixes have been made to better re-create the legacy API:
- the return value of GetFormat is no longer marked const (although
it should be considered so).
- a symbolic link getdata_struct.h -> getdata.h is created when the
headers are installed, but only if the legacy API is present in the
library.
- when included with a C++ compiler, the legacy function prototypes
are wrapped in an extern "C" block. (The new API functions are
not.)
|=========================================================================|
New in version 0.4.1:
Library Changes
* BUG FIX: Adding an ASCII encoded RAW field no longer creates an empty,
unencoded binary file.
* BUG FIX: A few minor memory leaks have been patched. Most of these
were the result of insufficient clean up in dirfile_close().
* As an exception to the Standards, the full stop character '.' is once
again permitted in field names. This is required by BLAST. Instead,
field names are prohibited from ending with the known encoding exten-
sions. The Standards still prohibit the full stop character in field
names, and the GD_PEDANTIC flag will disable this exception. Further-
more, fields may not be added via GetData which contain a full stop
character.
* Some minor speed improvements have been made to the format file parser
resulting in a ~30% decrease in parsing time.
|=========================================================================|
New in version 0.4.0:
Dirfile Changes
* GetData now supports Dirfile Standards Version 6. Standards Version 6
adds numerical (CONST) and string (STRING) scalar fields to the
dirfile, as well as other features outlined below.
* CONST names can now appear as parameters in field specifications in
certain places in lieu of a literal number.
* The tokeniser has been re-written. Any sequence of non-NUL bytes can
now form a valid token. Field codes may not have ASCII control charac-
ters or the "reserved characters" &, /, ;, <, >, |, . but may contain
any other characters (including whitespace).
* Subordinate fields may now be attached to fields. These subfields are
defined with the META directive, and referenced in most cases as
"<parent-field>/<subfield>".
* Binary files in dirfiles may now be "encoded". In addition to raw
(unencoded) files, two encodings are currently supported:
- ASCII encoding (a simple proof-of-functionality encoding), and
- Slimlib encoding (a compression library used by ACT).
Encoding schemes are fragment-local, although they are inherited from
parent fragments.
* Endianness is now also fragment-local, meaning different endiannesses
can co-exist in the same dirfile (so long as the fields are defined in
different format file fragments), and GetData will do the Right Thing.
* "FILEFRAM" is no-longer supported as an alias for "INDEX". This
affects GetData's implementation of all Standards Versions. Code that
made use of FILEFRAM will have to be updated to use INDEX instead.
* The reference field, previously the first RAW field in the format file,
may now be specified using the REFERENCE directive. The reference
field is the field that is looked at when get_nframes() is called.
This is important to programs like kst which support streaming
dirfiles.
* Portions (or all) of a dirfile's data and metadata can be protected
from change by specifying the PROTECT directive. This protection is
"advisory", ie. while GetData will respect it in all cases, don't count
on the PROTECT directive to ensure data integrity.
Library Changes
* INDEX is now a normal field. It appears in the count/list functions.
Attempts to write to INDEX will fail.
* The GD_E_EMPTY error code has been removed: it is no longer an error to
query a dirfile containing no RAW fields.
* The presence of two fields in the dirfile with the same name is now
detected, and will cause a syntax error when parsing the format file.
Previously, such things were allowed, but only one of the synonym
fields could ever be queried. (Which field was returned was arbitrary.)
* Field code look-ups for input fields are now cached, which should
result in slightly better performance from the library.
API Changes
* Fields may now be added to dirfiles in a number of different ways:
- by passing an gd_entry_t to dirfile_add()
- by passing a field specification line to dirfile_add_spec()
- by passing field parameters to one of the dirfile_add_<foo>()
functions.
* Whole other format file fragments may also be added to the dirfile by
calling dirfile_include().
* A function is now present (dirfile_metaflush()) to flush metadata
changes to disk (by re-writing format file fragments). dirfile_flush()
and dirfile_close() will also flush metadata, if needed.
* STRING and CONST values can be retrieved/set by calling
(get,put)_string() and (get,put)_constant(). (get,put)data() are only
for vector type fields.
* META fields can be queried/set like normal fields using the get/put
functions and their full (slashed) field code.
* In addition to get_nfields() and get_field_list(), there are now
corresponding functions that provide lists/counts of vectors
(get_vector_list()/get_nvectors()), particular field types
(get_field_list_by_type()/get_nfields_by_type()), as well as functions
that provide lists of string values (get_string_values()) and constant
values (get_const_values()).
* Analogous functions for the add, list, and counting functions exist for
META fields, as well.
* Extra flags have been added to dirfile_open() to permit indicating the
encoding type, if not specified in the dirfile itself.
* DIRFILE struct members and gd_entry_t private members are now com-
pletely hidden from the public interface. Where previously callers
could query dirfile->error to check for an error, they must now call
get_error(dirfile). This change was made to reduce unintentional ABI
breakage when modifying internal library properties.
* GD_VERBOSE has been added to the list of available dirfile_open()
flags. If this flag is specified, GetData will write errors to stderr
as encountered.
Legacy API Changes
* Error codes which cannot be returned by the legacy API might not have a
corresponding string in GD_ERROR_CODES[]. Instead, these entries will
simply be the NULL pointer.
* No facilities exist in the legacy API to set or query CONST or STRING
fields. However, META fields of vector type can be queried/set using
GetData() and PutData(), as in the new API.
|=========================================================================|
New in version 0.3.1:
Legacy API Changes
* BUG FIX: Dirfiles are now opened in read-only mode, unless instantiated
via PutData(), allowing GetData() calls on read-only dirfiles. If
PutData() is called on a dirfile previously opened read-only, it will
be re-opened in read-write mode.
Bindings
* The C++ bindings, formerly called libdirfile, are now called
libgetdata++ to be more explicit about what this library is.
Miscellaneous
* The package now includes pkg-config support for libgetdata.
|=========================================================================|
New in version 0.3.0:
Dirfile Changes
* GetData now supports Dirfile Standards Version 5 which includes support
for signed 8-bit, and signed and unsigned 64-bit integer types.
* As part of Standards Version 5, the restriction on field name length
has been removed. (The filesystem will impose an effective limit on
RAW fields of a few hundred characters, and format file lines are
limited to several thousand, limiting derived field names.)
* The library can now convert between big and little endiannesses. An
optional directive, ENDIAN, added in Standards Version 5, is available
to specify the byte-sex of a dirfile.
API Changes
* There is a new interface which fixes issues with thread safety and
largefile support in the old interface. The old interface (referred to
as the "legacy API") is still supported, but doesn't fully implement
Dirfile Standards Version 5. See the README for full details.
* putdata now respects FRAMEOFFSET.
* putdata can now write to PHASE fields, and multi-bit BIT fields.
* Some error codes have been renamed, and others removed or added, in
order to regularise error codes between getdata and putdata in the
sundry versions. Furthermore, the underlying values of some of these
codes have changed (notably excluding GD_E_OK, which is guaranteed to
be zero). Changes include:
- GD_E_OPEN_FORMAT is now called GD_E_OPEN
- GD_E_BAD_RETURN_TYPE is now called GD_E_BAD_TYPE
- GD_E_NO_RAW_FIELDS is now called GD_E_EMPTY
- PD_E_MULT_LINCOM is now called GD_E_BAD_PUT_FIELD
- GD_E_OPEN_RAWFIELD and PD_E_OPEN RAWFIELD are now both represented
by GD_E_RAW_IO
- PD_E_BAD_CODE is now handled by GD_E_BAD_CODE
- GD_E_FIELD, GD_E_SIZE_MISMATCH, ENDIAN_ERROR, CLOSE_ERROR are no
longer applicable and have been removed
- PD_E_CLOSE_RDONLY, PD_E_WRITE_LOCK, PD_E_FLOCK_ALLOC, which were
were defined in the header but never used, have been removed
- GD_E_TRUNC, GD_E_CREAT, GD_E_BAD_DIRFILE, GD_E_RANGE and
GD_E_ACCMODE are new
This affects the legacy API.
Legacy API Changes
* The legacy API has been marked deprecated.
* Error codes have changed per the description above.
* Only the public members of FormatType are now initialised by GetFormat.
Bindings
* Added bindings for C++ (libdirfile), Fortran 77 (libfgetdata), and
Fortran 95 (libf95getdata)
Miscellaneous
* A rudimentary, but thorough, test-suite has been made.
|