1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 2886 2887 2888 2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 2929 2930 2931 2932 2933 2934 2935 2936 2937 2938 2939 2940 2941 2942 2943 2944 2945 2946 2947 2948 2949 2950 2951 2952 2953 2954 2955 2956 2957 2958 2959 2960 2961 2962 2963 2964 2965 2966 2967 2968 2969 2970 2971 2972 2973 2974 2975 2976 2977 2978 2979 2980 2981 2982 2983 2984 2985 2986 2987 2988 2989 2990 2991 2992 2993 2994 2995 2996 2997 2998 2999 3000 3001 3002 3003 3004 3005 3006 3007 3008 3009 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 3020 3021 3022 3023 3024 3025 3026 3027 3028 3029 3030 3031 3032 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 3052 3053 3054 3055 3056 3057 3058 3059 3060 3061 3062 3063 3064 3065 3066 3067 3068 3069 3070 3071 3072 3073 3074 3075 3076 3077 3078 3079 3080 3081 3082 3083 3084 3085 3086 3087 3088 3089 3090 3091 3092 3093 3094 3095 3096 3097 3098 3099 3100 3101 3102 3103 3104 3105 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 3117 3118 3119 3120 3121 3122 3123 3124 3125 3126 3127 3128 3129 3130 3131 3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 3143 3144 3145 3146 3147 3148 3149 3150 3151 3152 3153 3154 3155 3156 3157 3158 3159 3160 3161 3162 3163 3164 3165 3166 3167 3168 3169 3170 3171 3172 3173 3174 3175 3176 3177 3178 3179 3180 3181 3182 3183 3184 3185 3186 3187 3188 3189 3190 3191 3192 3193 3194 3195 3196 3197 3198 3199 3200 3201 3202 3203 3204 3205 3206 3207 3208 3209 3210 3211 3212 3213 3214 3215 3216 3217 3218 3219 3220 3221 3222 3223 3224 3225 3226 3227 3228 3229 3230 3231 3232 3233 3234 3235 3236 3237 3238 3239 3240 3241 3242 3243 3244 3245 3246 3247 3248 3249 3250 3251 3252 3253 3254 3255 3256 3257 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 3273 3274 3275 3276 3277 3278 3279 3280 3281 3282 3283 3284 3285 3286 3287 3288 3289 3290 3291 3292 3293 3294 3295 3296 3297 3298 3299 3300 3301 3302 3303 3304 3305 3306 3307 3308 3309 3310 3311 3312 3313 3314 3315 3316 3317 3318 3319 3320 3321 3322 3323 3324 3325 3326 3327 3328 3329 3330 3331 3332 3333 3334 3335 3336 3337 3338 3339 3340 3341 3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 3355 3356 3357 3358 3359 3360 3361 3362 3363 3364 3365 3366 3367 3368 3369 3370 3371 3372 3373 3374 3375 3376 3377 3378 3379 3380 3381 3382 3383 3384 3385 3386 3387 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 3407 3408 3409 3410 3411 3412 3413 3414 3415 3416 3417 3418 3419 3420 3421 3422 3423 3424 3425 3426 3427 3428 3429 3430 3431 3432 3433 3434 3435 3436 3437 3438 3439 3440 3441 3442 3443 3444 3445 3446 3447 3448 3449 3450 3451 3452 3453 3454 3455 3456 3457 3458 3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 3470 3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 3489 3490 3491 3492 3493 3494 3495 3496 3497 3498 3499 3500 3501 3502 3503 3504 3505 3506 3507 3508 3509 3510 3511 3512 3513 3514 3515 3516 3517 3518 3519 3520 3521 3522 3523 3524 3525 3526 3527 3528 3529 3530 3531 3532 3533 3534 3535 3536 3537 3538 3539 3540 3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 3552 3553 3554 3555 3556 3557 3558 3559 3560 3561 3562 3563 3564 3565 3566 3567 3568 3569 3570 3571 3572 3573 3574 3575 3576 3577 3578 3579 3580 3581 3582 3583 3584 3585 3586 3587 3588 3589 3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 3637 3638 3639 3640 3641 3642 3643 3644 3645 3646 3647 3648 3649 3650 3651 3652 3653 3654 3655 3656 3657 3658 3659 3660 3661 3662 3663 3664 3665 3666 3667 3668 3669 3670 3671 3672 3673 3674 3675 3676 3677 3678 3679 3680 3681 3682 3683 3684 3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 3696 3697 3698 3699 3700 3701 3702 3703 3704 3705 3706 3707 3708 3709 3710 3711 3712 3713 3714 3715 3716 3717 3718 3719 3720 3721 3722 3723 3724 3725 3726 3727 3728 3729 3730 3731 3732 3733 3734 3735 3736 3737 3738 3739 3740 3741 3742 3743 3744 3745 3746 3747 3748 3749 3750 3751 3752 3753 3754 3755 3756 3757 3758 3759 3760 3761 3762 3763 3764 3765 3766 3767 3768 3769 3770 3771 3772 3773 3774 3775 3776 3777 3778 3779 3780 3781 3782 3783 3784 3785 3786 3787 3788 3789 3790 3791 3792 3793 3794 3795 3796 3797 3798 3799 3800 3801 3802 3803 3804 3805 3806 3807 3808 3809 3810 3811 3812 3813 3814 3815 3816 3817 3818 3819 3820 3821 3822 3823 3824 3825 3826 3827 3828 3829 3830 3831 3832 3833 3834 3835 3836 3837 3838 3839 3840 3841 3842 3843 3844 3845 3846 3847 3848 3849 3850 3851 3852 3853 3854 3855 3856 3857 3858 3859 3860 3861 3862 3863 3864 3865 3866 3867 3868 3869 3870 3871 3872 3873 3874 3875 3876 3877 3878 3879 3880 3881 3882 3883 3884 3885 3886 3887 3888 3889 3890 3891 3892 3893 3894 3895 3896 3897 3898 3899 3900 3901 3902 3903 3904 3905 3906 3907 3908 3909 3910 3911 3912 3913 3914 3915 3916 3917 3918 3919 3920 3921 3922 3923 3924 3925 3926 3927 3928 3929 3930 3931 3932 3933 3934 3935 3936 3937 3938 3939 3940 3941 3942 3943 3944 3945 3946 3947 3948 3949 3950 3951 3952 3953 3954 3955 3956 3957 3958 3959 3960 3961 3962 3963 3964 3965 3966 3967 3968 3969 3970 3971 3972 3973 3974 3975 3976 3977 3978 3979 3980 3981 3982 3983 3984 3985 3986 3987 3988 3989 3990 3991 3992 3993 3994 3995 3996 3997 3998 3999 4000 4001 4002 4003 4004 4005 4006 4007 4008 4009 4010 4011 4012 4013 4014 4015 4016 4017 4018 4019 4020 4021 4022 4023 4024 4025 4026 4027 4028 4029 4030 4031 4032 4033 4034 4035 4036 4037 4038 4039 4040 4041 4042 4043 4044 4045 4046 4047 4048 4049 4050 4051 4052 4053 4054 4055 4056 4057 4058 4059 4060 4061 4062 4063 4064 4065 4066 4067 4068 4069 4070 4071 4072 4073 4074 4075 4076 4077 4078 4079 4080 4081 4082 4083 4084 4085 4086 4087 4088 4089 4090 4091 4092 4093 4094 4095 4096 4097 4098 4099 4100 4101 4102 4103 4104 4105 4106 4107 4108 4109 4110 4111 4112 4113 4114 4115 4116 4117 4118 4119 4120 4121 4122 4123 4124 4125 4126 4127 4128 4129 4130 4131 4132 4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 4157 4158 4159 4160 4161 4162 4163 4164 4165 4166 4167 4168 4169 4170 4171 4172 4173 4174 4175 4176 4177 4178 4179 4180 4181 4182 4183 4184 4185 4186 4187 4188 4189 4190 4191 4192 4193 4194 4195 4196 4197 4198 4199 4200 4201 4202 4203 4204 4205 4206 4207 4208 4209 4210 4211 4212 4213 4214 4215 4216 4217 4218 4219 4220 4221 4222 4223 4224 4225 4226 4227 4228 4229 4230 4231 4232 4233 4234 4235 4236 4237 4238 4239 4240 4241 4242 4243 4244 4245 4246 4247 4248 4249 4250 4251 4252 4253 4254 4255 4256 4257 4258 4259 4260 4261 4262 4263 4264 4265 4266 4267 4268 4269 4270 4271 4272 4273 4274 4275 4276 4277 4278 4279 4280 4281 4282 4283 4284 4285 4286 4287 4288 4289 4290 4291 4292 4293 4294 4295 4296 4297 4298 4299 4300 4301 4302 4303 4304 4305 4306 4307 4308 4309 4310 4311 4312 4313 4314 4315 4316 4317 4318 4319 4320 4321 4322 4323 4324 4325 4326 4327 4328 4329 4330 4331 4332 4333 4334 4335 4336 4337 4338 4339 4340 4341 4342 4343 4344 4345 4346 4347 4348 4349 4350 4351 4352 4353 4354 4355 4356 4357 4358 4359 4360 4361 4362 4363 4364 4365 4366 4367 4368 4369 4370 4371 4372 4373 4374 4375 4376 4377 4378 4379 4380 4381 4382 4383 4384 4385 4386 4387 4388 4389 4390 4391 4392 4393 4394 4395 4396 4397 4398 4399 4400 4401 4402 4403 4404 4405 4406 4407 4408 4409 4410 4411 4412 4413 4414 4415 4416 4417 4418 4419 4420 4421 4422 4423 4424 4425 4426 4427 4428 4429 4430 4431 4432 4433 4434 4435 4436 4437 4438 4439 4440 4441 4442 4443 4444 4445 4446 4447 4448 4449 4450 4451 4452 4453 4454 4455 4456 4457 4458 4459 4460 4461 4462 4463 4464 4465 4466 4467 4468 4469 4470 4471 4472 4473 4474 4475 4476 4477 4478 4479 4480 4481 4482 4483 4484 4485 4486 4487 4488 4489 4490 4491 4492 4493 4494 4495 4496 4497 4498 4499 4500 4501 4502 4503 4504 4505 4506 4507 4508 4509 4510 4511 4512 4513 4514 4515 4516 4517 4518 4519 4520 4521 4522 4523 4524 4525 4526 4527 4528 4529 4530 4531 4532 4533 4534 4535 4536 4537 4538 4539 4540 4541 4542 4543 4544 4545 4546 4547 4548 4549 4550 4551 4552 4553 4554 4555 4556 4557 4558 4559 4560 4561 4562 4563 4564 4565 4566 4567 4568 4569 4570 4571 4572 4573 4574 4575 4576 4577 4578 4579 4580 4581 4582 4583 4584 4585 4586 4587 4588 4589 4590 4591 4592 4593 4594 4595 4596 4597 4598 4599 4600 4601 4602 4603 4604 4605 4606 4607 4608 4609 4610 4611 4612 4613 4614 4615 4616 4617 4618 4619 4620 4621 4622 4623 4624 4625 4626 4627 4628 4629 4630 4631 4632 4633 4634 4635 4636 4637 4638 4639 4640 4641 4642 4643 4644 4645 4646 4647 4648 4649 4650 4651 4652 4653 4654 4655 4656 4657 4658 4659 4660 4661 4662 4663 4664 4665 4666 4667 4668 4669 4670 4671 4672 4673 4674 4675 4676 4677 4678 4679 4680 4681 4682 4683 4684 4685 4686 4687 4688 4689 4690 4691 4692 4693 4694 4695 4696 4697 4698 4699 4700 4701 4702 4703 4704 4705 4706 4707 4708 4709 4710 4711 4712 4713 4714 4715 4716 4717 4718 4719 4720 4721 4722 4723 4724 4725 4726 4727 4728 4729 4730 4731 4732 4733 4734 4735 4736 4737 4738 4739 4740 4741 4742 4743 4744 4745 4746 4747 4748 4749 4750 4751 4752 4753 4754 4755 4756 4757 4758 4759 4760 4761 4762 4763 4764 4765 4766 4767 4768 4769 4770 4771 4772 4773 4774 4775 4776 4777 4778 4779 4780 4781 4782 4783 4784 4785 4786 4787 4788 4789 4790 4791 4792 4793 4794 4795 4796 4797 4798 4799 4800 4801 4802 4803 4804 4805 4806 4807 4808 4809 4810 4811 4812 4813 4814 4815 4816 4817 4818 4819 4820 4821 4822 4823 4824 4825 4826 4827 4828 4829 4830 4831 4832 4833 4834 4835 4836 4837 4838 4839 4840 4841 4842 4843 4844 4845 4846 4847 4848 4849 4850 4851 4852 4853 4854 4855 4856 4857 4858 4859 4860 4861 4862 4863 4864 4865 4866 4867 4868 4869 4870 4871 4872 4873 4874 4875 4876 4877 4878 4879 4880 4881 4882 4883 4884 4885 4886 4887 4888 4889 4890 4891 4892 4893 4894 4895 4896 4897 4898 4899 4900 4901 4902 4903 4904 4905 4906 4907 4908 4909 4910 4911 4912 4913 4914 4915 4916 4917 4918 4919 4920 4921 4922 4923 4924 4925 4926 4927 4928 4929 4930 4931 4932 4933 4934 4935 4936 4937 4938 4939 4940 4941 4942 4943 4944 4945 4946 4947 4948 4949 4950 4951 4952 4953 4954 4955 4956 4957 4958 4959 4960 4961 4962 4963 4964 4965 4966 4967 4968 4969 4970 4971 4972 4973 4974 4975 4976 4977 4978 4979 4980 4981 4982 4983 4984 4985 4986 4987 4988 4989 4990 4991 4992 4993 4994 4995 4996 4997 4998 4999 5000 5001 5002 5003 5004 5005 5006 5007 5008 5009 5010 5011 5012 5013 5014 5015 5016 5017 5018 5019 5020 5021 5022 5023 5024 5025 5026 5027 5028 5029 5030 5031 5032 5033 5034 5035 5036 5037 5038 5039 5040 5041 5042 5043 5044 5045 5046 5047 5048 5049 5050 5051 5052 5053 5054 5055 5056 5057 5058 5059 5060 5061 5062 5063 5064 5065 5066 5067 5068 5069 5070 5071 5072 5073 5074 5075 5076 5077 5078 5079 5080 5081 5082 5083 5084 5085 5086 5087 5088 5089 5090 5091 5092 5093 5094 5095 5096 5097 5098 5099 5100 5101 5102 5103 5104 5105 5106 5107 5108 5109 5110 5111 5112 5113 5114 5115 5116 5117 5118 5119 5120 5121 5122 5123 5124 5125 5126 5127 5128 5129 5130 5131 5132 5133 5134 5135 5136 5137 5138 5139 5140 5141 5142 5143 5144 5145 5146 5147 5148 5149 5150 5151 5152 5153 5154 5155 5156 5157 5158 5159 5160 5161 5162 5163 5164 5165 5166 5167 5168 5169 5170 5171 5172 5173 5174 5175 5176 5177 5178 5179 5180 5181 5182 5183 5184 5185 5186 5187 5188 5189 5190 5191 5192 5193 5194 5195 5196 5197 5198 5199 5200 5201 5202 5203 5204 5205 5206 5207 5208 5209 5210 5211 5212 5213 5214 5215 5216 5217 5218 5219 5220 5221 5222 5223 5224 5225 5226 5227 5228 5229 5230 5231 5232 5233 5234 5235 5236 5237 5238 5239 5240 5241 5242 5243 5244 5245 5246 5247 5248 5249 5250 5251 5252 5253 5254 5255 5256 5257 5258 5259 5260 5261 5262 5263 5264 5265 5266 5267 5268 5269 5270 5271 5272 5273 5274 5275 5276 5277 5278 5279 5280 5281 5282 5283 5284 5285 5286 5287 5288 5289 5290 5291 5292 5293 5294 5295 5296 5297 5298 5299 5300 5301 5302 5303 5304 5305 5306 5307 5308 5309 5310 5311 5312 5313 5314 5315 5316 5317 5318 5319 5320 5321 5322 5323 5324 5325 5326 5327 5328 5329 5330 5331 5332 5333 5334 5335 5336 5337 5338 5339 5340 5341 5342 5343 5344 5345 5346 5347 5348 5349 5350 5351 5352 5353 5354 5355 5356 5357 5358 5359 5360 5361 5362 5363 5364 5365 5366 5367 5368 5369 5370 5371 5372 5373 5374 5375 5376 5377 5378 5379 5380 5381 5382 5383 5384 5385 5386 5387 5388 5389 5390 5391 5392 5393 5394 5395 5396 5397 5398 5399 5400 5401 5402 5403 5404 5405 5406 5407 5408 5409 5410 5411 5412 5413 5414 5415 5416 5417 5418 5419 5420 5421 5422 5423 5424 5425 5426 5427 5428 5429 5430 5431 5432 5433 5434 5435 5436 5437 5438 5439 5440 5441 5442 5443 5444 5445 5446 5447 5448 5449 5450 5451 5452 5453 5454 5455 5456 5457 5458 5459 5460 5461 5462 5463 5464 5465 5466 5467 5468 5469 5470 5471 5472 5473 5474 5475 5476 5477 5478 5479 5480 5481 5482 5483 5484 5485 5486 5487 5488 5489 5490 5491 5492 5493 5494 5495 5496 5497 5498 5499 5500 5501 5502 5503 5504 5505 5506 5507 5508 5509 5510 5511 5512 5513 5514 5515 5516 5517 5518 5519 5520 5521 5522 5523 5524 5525 5526 5527 5528 5529 5530 5531 5532 5533 5534 5535 5536 5537 5538 5539 5540 5541 5542 5543 5544 5545 5546 5547 5548 5549 5550 5551 5552 5553 5554 5555 5556 5557 5558 5559 5560 5561 5562 5563 5564 5565 5566 5567 5568 5569 5570 5571 5572 5573 5574 5575 5576 5577 5578 5579 5580 5581 5582 5583 5584 5585 5586 5587 5588 5589 5590 5591 5592 5593 5594 5595 5596 5597 5598 5599 5600 5601 5602 5603 5604 5605 5606 5607 5608 5609 5610 5611 5612 5613 5614 5615 5616 5617 5618 5619 5620 5621 5622 5623 5624 5625 5626 5627 5628 5629 5630 5631 5632 5633 5634 5635 5636 5637 5638 5639 5640 5641 5642 5643 5644 5645 5646 5647 5648 5649 5650 5651 5652 5653 5654 5655 5656 5657 5658 5659 5660 5661 5662 5663 5664 5665 5666 5667 5668 5669 5670 5671 5672 5673 5674 5675 5676 5677 5678 5679 5680 5681 5682 5683 5684 5685 5686 5687 5688 5689 5690 5691 5692 5693 5694 5695 5696 5697 5698 5699 5700 5701 5702 5703 5704 5705 5706 5707 5708 5709 5710 5711 5712 5713 5714 5715 5716 5717 5718 5719 5720 5721 5722 5723 5724 5725 5726 5727 5728 5729 5730 5731 5732 5733 5734 5735 5736 5737 5738 5739 5740 5741 5742 5743 5744 5745 5746 5747 5748 5749 5750 5751 5752 5753 5754 5755 5756 5757 5758 5759 5760 5761 5762 5763 5764 5765 5766 5767 5768 5769 5770 5771 5772 5773 5774 5775 5776 5777 5778 5779 5780 5781 5782 5783 5784 5785 5786 5787 5788 5789 5790 5791 5792 5793 5794 5795 5796 5797 5798 5799 5800 5801 5802 5803 5804 5805 5806 5807 5808 5809 5810 5811 5812 5813 5814 5815 5816 5817 5818 5819 5820 5821 5822 5823 5824 5825 5826 5827 5828 5829 5830 5831 5832 5833 5834 5835 5836 5837 5838 5839 5840 5841 5842 5843 5844 5845 5846 5847 5848 5849 5850 5851 5852 5853 5854 5855 5856 5857 5858 5859 5860 5861 5862 5863 5864 5865 5866 5867 5868 5869 5870 5871 5872 5873 5874 5875 5876 5877 5878 5879 5880 5881 5882 5883 5884 5885 5886 5887 5888 5889 5890 5891 5892 5893 5894 5895 5896 5897 5898 5899 5900 5901 5902 5903 5904 5905 5906 5907 5908 5909 5910 5911 5912 5913 5914 5915 5916 5917 5918 5919 5920 5921 5922 5923 5924 5925 5926 5927 5928 5929 5930 5931 5932 5933 5934 5935 5936 5937 5938 5939 5940 5941 5942 5943 5944 5945 5946 5947 5948 5949 5950 5951 5952 5953 5954 5955 5956 5957 5958 5959 5960 5961 5962 5963 5964 5965 5966 5967 5968 5969 5970 5971 5972 5973 5974 5975 5976 5977 5978 5979 5980 5981 5982 5983 5984 5985 5986 5987 5988 5989 5990 5991 5992 5993 5994 5995 5996 5997 5998 5999 6000 6001 6002 6003 6004 6005 6006 6007 6008 6009 6010 6011 6012 6013 6014 6015 6016 6017 6018 6019 6020 6021 6022 6023 6024 6025 6026 6027 6028 6029 6030 6031 6032 6033 6034 6035 6036 6037 6038 6039 6040 6041 6042 6043 6044 6045 6046 6047 6048 6049 6050 6051 6052 6053 6054 6055 6056 6057 6058 6059 6060 6061 6062 6063 6064 6065 6066 6067 6068 6069 6070 6071 6072 6073 6074 6075 6076 6077 6078 6079 6080 6081 6082 6083 6084 6085 6086 6087 6088 6089 6090 6091 6092 6093 6094 6095 6096 6097 6098 6099 6100 6101 6102 6103 6104 6105 6106 6107 6108 6109 6110 6111 6112 6113 6114 6115 6116 6117 6118 6119 6120 6121 6122 6123 6124 6125 6126 6127 6128 6129 6130 6131 6132 6133 6134 6135 6136 6137 6138 6139 6140 6141 6142 6143 6144 6145 6146 6147 6148 6149 6150 6151 6152 6153 6154 6155 6156 6157 6158 6159 6160 6161 6162 6163 6164 6165 6166 6167 6168 6169 6170 6171 6172 6173 6174 6175 6176 6177 6178 6179 6180 6181 6182 6183 6184 6185 6186 6187 6188 6189 6190 6191 6192 6193 6194 6195 6196 6197 6198 6199 6200 6201 6202 6203 6204 6205 6206 6207 6208 6209 6210 6211 6212 6213 6214 6215 6216 6217 6218 6219 6220 6221 6222 6223 6224 6225 6226 6227 6228 6229 6230 6231 6232 6233 6234 6235 6236 6237 6238 6239 6240 6241 6242 6243 6244 6245 6246 6247 6248 6249 6250 6251 6252 6253 6254 6255 6256 6257 6258 6259 6260 6261 6262 6263 6264 6265 6266 6267 6268 6269 6270 6271 6272 6273 6274 6275 6276 6277 6278 6279 6280 6281 6282 6283 6284 6285 6286 6287 6288 6289 6290 6291 6292 6293 6294 6295 6296 6297 6298 6299 6300 6301 6302 6303 6304 6305 6306 6307 6308 6309 6310 6311 6312 6313 6314 6315 6316 6317 6318 6319 6320 6321 6322 6323 6324 6325 6326 6327 6328 6329 6330 6331 6332 6333 6334 6335 6336 6337 6338 6339 6340 6341 6342 6343 6344 6345 6346 6347 6348 6349 6350 6351 6352 6353 6354 6355 6356 6357 6358 6359 6360 6361 6362 6363 6364 6365 6366 6367 6368 6369 6370 6371 6372 6373 6374 6375 6376 6377 6378 6379 6380 6381 6382 6383 6384 6385 6386 6387 6388 6389 6390 6391 6392 6393 6394 6395 6396 6397 6398 6399 6400 6401 6402 6403 6404 6405 6406 6407 6408 6409 6410 6411 6412 6413 6414 6415 6416 6417 6418 6419 6420 6421 6422 6423 6424 6425 6426 6427 6428 6429 6430 6431 6432 6433 6434 6435 6436 6437 6438 6439 6440 6441 6442 6443 6444 6445 6446 6447 6448 6449 6450 6451 6452 6453 6454 6455 6456 6457 6458 6459 6460 6461 6462 6463 6464 6465 6466 6467 6468 6469 6470 6471 6472 6473 6474 6475 6476 6477 6478 6479 6480 6481 6482 6483 6484 6485 6486 6487 6488 6489 6490 6491 6492 6493 6494 6495 6496 6497 6498 6499 6500 6501 6502 6503 6504 6505 6506 6507 6508 6509 6510 6511 6512 6513 6514 6515 6516 6517 6518 6519 6520 6521 6522 6523 6524 6525 6526 6527 6528 6529 6530 6531 6532 6533 6534 6535 6536 6537 6538 6539 6540 6541 6542 6543 6544 6545 6546 6547 6548 6549 6550 6551 6552 6553 6554 6555 6556 6557 6558 6559 6560 6561 6562 6563 6564 6565 6566 6567 6568 6569 6570 6571 6572 6573 6574 6575 6576 6577 6578 6579 6580 6581 6582 6583 6584 6585 6586 6587 6588 6589 6590 6591 6592 6593 6594 6595 6596 6597 6598 6599 6600 6601 6602 6603 6604 6605 6606 6607 6608 6609 6610 6611 6612 6613 6614 6615 6616 6617 6618 6619 6620 6621 6622 6623 6624 6625 6626 6627 6628 6629 6630 6631 6632 6633 6634 6635 6636 6637 6638 6639 6640 6641 6642 6643 6644 6645 6646 6647 6648 6649 6650 6651 6652 6653 6654 6655 6656 6657 6658 6659 6660 6661 6662 6663 6664 6665 6666 6667 6668 6669 6670 6671 6672 6673 6674 6675 6676 6677 6678 6679 6680 6681 6682 6683 6684 6685 6686 6687 6688 6689 6690 6691 6692 6693 6694 6695 6696 6697 6698 6699 6700 6701 6702 6703 6704 6705 6706 6707 6708 6709 6710 6711 6712 6713 6714 6715 6716 6717 6718 6719 6720 6721 6722 6723 6724 6725 6726 6727 6728 6729 6730 6731 6732 6733 6734 6735 6736 6737 6738 6739 6740 6741 6742 6743 6744 6745 6746 6747 6748 6749 6750 6751 6752 6753 6754 6755 6756 6757 6758 6759 6760 6761 6762 6763 6764 6765 6766 6767 6768 6769 6770 6771 6772 6773 6774 6775 6776 6777 6778 6779 6780 6781 6782 6783 6784 6785 6786 6787 6788 6789 6790 6791 6792 6793 6794 6795 6796 6797 6798 6799 6800 6801 6802 6803 6804 6805 6806 6807 6808 6809 6810 6811 6812 6813 6814 6815 6816 6817 6818 6819 6820 6821 6822 6823 6824 6825 6826 6827 6828 6829 6830 6831 6832 6833 6834 6835 6836 6837 6838 6839 6840 6841 6842 6843 6844 6845 6846 6847 6848 6849 6850 6851 6852 6853 6854 6855 6856 6857 6858 6859 6860 6861 6862 6863 6864 6865 6866 6867 6868 6869 6870 6871 6872 6873 6874 6875 6876 6877 6878 6879 6880 6881 6882 6883 6884 6885 6886 6887 6888 6889 6890 6891 6892 6893 6894 6895 6896 6897 6898 6899 6900 6901 6902 6903 6904 6905 6906 6907 6908 6909 6910 6911 6912 6913 6914 6915 6916 6917 6918 6919 6920 6921 6922 6923 6924 6925 6926 6927 6928 6929 6930 6931 6932 6933 6934 6935 6936 6937 6938 6939 6940 6941 6942 6943 6944 6945 6946 6947 6948 6949 6950 6951 6952 6953 6954 6955 6956 6957 6958 6959 6960 6961 6962 6963 6964 6965 6966 6967 6968 6969 6970 6971 6972 6973 6974 6975 6976 6977 6978 6979 6980 6981 6982 6983 6984 6985 6986 6987 6988 6989 6990 6991 6992 6993 6994 6995 6996 6997 6998 6999 7000 7001 7002 7003 7004 7005 7006 7007 7008 7009 7010 7011 7012 7013 7014 7015 7016 7017 7018 7019 7020 7021 7022 7023 7024 7025 7026 7027 7028 7029 7030 7031 7032 7033 7034 7035 7036 7037 7038 7039 7040 7041 7042 7043 7044 7045 7046 7047 7048 7049 7050 7051 7052 7053 7054 7055 7056 7057 7058 7059 7060 7061 7062 7063 7064 7065 7066 7067 7068 7069 7070 7071 7072 7073 7074 7075 7076 7077 7078 7079 7080 7081 7082 7083 7084 7085 7086 7087 7088 7089 7090 7091 7092 7093 7094 7095 7096 7097 7098 7099 7100 7101 7102 7103 7104 7105 7106 7107 7108 7109 7110 7111 7112 7113 7114 7115 7116 7117 7118 7119 7120 7121 7122 7123 7124 7125 7126 7127 7128 7129 7130 7131 7132 7133 7134 7135 7136 7137 7138 7139 7140 7141 7142 7143 7144 7145 7146 7147 7148 7149 7150 7151 7152 7153 7154 7155 7156 7157 7158 7159 7160 7161 7162 7163 7164 7165 7166 7167 7168 7169 7170 7171 7172 7173 7174 7175 7176 7177 7178 7179 7180 7181 7182 7183 7184 7185 7186 7187 7188 7189 7190 7191 7192 7193 7194 7195 7196 7197 7198 7199 7200 7201 7202 7203 7204 7205 7206 7207 7208 7209 7210 7211 7212 7213 7214 7215
|
2025 04 20 - v25.0
This major release fix several issues reported since last release and adds some
new features and improvements.
* Add multiple assessment report format output at once. Thanks to jcarnu for
the patch.
* Support more Oracle to PostgrSQL exception mapping.
* Allow overriding of PG_SUPPORTS_* settings as they are set in the
configuration file. PG_VERSION will have no effect in this case.
Thanks to Pavel Stehule for the feature request.
* Add multiple report format output at once. Thanks to Jean-Christophe Arnu
for the patch.
* Add parsing of ALTER statement from file for the QUERY action.
* Add support for oracle_fdw COPY using CSV format
* Add parallel export of each partition. Until now all partitions of a
partitioned table was exported serialy in a single process.
* Allow online data migration to continue if the destination table does not
exist when ON_ERROR_STOP is disabled. Thanks to chetan2211 for the feature
request.
* Tables data export is now done using the current SCN to have the same
snapshot of data between multiprocess export.
* Add replacement of USERENV call with MODULE by current_setting with
application_name.
* Add support for data movement using psql copy selecting over oracle_fdw
- Type INSERT (when FDW_SERVER is set) preserves prior behaviour of both
INSERT and COPY
- Type COPY (when FDW_SERVER is set) uses a new mode of using the `psql`
with `\copy` with `TO PROGRAM...FROM STDIN BINARY...BINARY`
Thanks to Martin Nash for the patch.
* Add support for oracle_fdw in combination with psql "\copy" and server-side
COPY using BINARY stream for data movement. Brings both local and server-side
oracle_fdw binary copy into one branch. Control over which mode is used is
provided via ORACLE_FDW_COPY_MODE configuration, which defaults to "local".
Thanks to Martin Nash for the patch.
* Convert DBMS_SQL cursor/parsing/execute simple form to PostgreSQL dynamic
query execution. Thanks to tanguydelignieresaccenture for the patch with
some regex improvement by me to handle comments.
* Add replacement of the MOD operator. Thanks to ec-digit-dbecoe for the report.
* Add some additional objects/difficulties assessment migration scores:
- WHEN OTHER THEN exception clause that hide the real problem during
migration.
- 'SSSSS' and 'J' for to_char() that could need rewrites.
* Add SCRIPT action to process sqlplus scripts as a whole, not line per line.
* Change default settings in config file generated by the --init_project option with:
- PG_NUMERIC_TYPE 0
- NULL_EQUAL_EMPTY 1
New options and configuration directives:
* Add --no_clean_comment option to not remove comments in source file before
parsing. With huge DDL file with comments, it could take a very long time.
* Add -O | --options used to override any configuration parameter, it can
be used multiple time. Syntax:
-O "PARAM1_NAME=value" -O "PARAM2_NAME=value"
or
-O "PARAM1_NAME=value|PARAM2_NAME=value"
* Add option --no_start_scn to force Ora2Pg to not use a SCN to export data
unless --snc is used. By default the current SCN is used to export data
from all tables.
* Add ORACLE_FDW_COPY_MODE configuration directive.
When using Ora2Pg COPY with oracle_fdw it is possible to use two different
modes: 1) "local", which uses psql on the host running Ora2Pg for the "TO"
binary stream; 2) "server", which uses PostgreSQL server-side COPY for the "TO"
binary stream. Both modes use psql for the "FROM STDIN BINARY". However,
"local" runs the psql "FROM STDIN BINARY" on host Ora2Pg is run from, whereas
"server" runs the psql "FROM STDIN BINARY" on the PostgreSQL server. "local"
mode should work on any PostgreSQL-based system, including managed offerings,
which are not expected to support use of "server" mode due to permissions. The
default is "local" as this is compatible with more configurations.
* Add ORACLE_FDW_COPY_FORMAT configuration directive.
When using Ora2Pg COPY with oracle_fdw it is possible to use either BINARY or
CSV data format. BINARY provides better performance, however, requires exact
data type matching between the FDW and destination table. CSV provides greater
flexibiliity with respect to data type matching: if the FDW and destination
data types are functionally-compatible the columns can be copied. The default
is "binary".
Bug fixes:
- Fix always returns row count from stats, even if --count_rows flag is used.
Thanks to alavrent2022 for the report.
- Fix multiprocess with TEST_DATA action. Thanks to tanguydelignieresaccenture
for the report.
- Fix another if condition issue. Thanks to skm9380 for the report.
- Fix assignment sign in UPDATE statements. Thanks to ec-digit-dbecoe for the
report.
- Fix mysql # comments. Thanks to ec-digit-dbecoe fir the report.
- Fix enum column with value "inf" that is expanded to the word "Infinity".
Thanks to nbromage for the report.
- Fix MySQL labels not converted to PostgreSQL syntax. Thanks to
ec-digit-dbecoex for the report.
- Fix MySQL DECLARE declaration after BEGIN clause. Thanks to ec-digit-dbecoe
for the report.
- Fix error Invalid Identifier error for DBMS_LOB.GETLENGTH() by checking to
user privilege. Thanks to tanguydelignieresaccenture for the patch.
- Fix TO_DATE translation when a NLS_setting is present. Thanks to
anguydelignieresaccenture for the patch.
Fix export type TEST reporting 0 for PostgreSQL functions count if using
default PACKAGE_AS_SCHEMA. Thanks to tanguydelignieresaccenture for the patch.
- Use double quote instead of single quote in ora2pg command to fix an issue
on windows with ora2pg_scanner. Thanks to ec-digit-dbecoe for the patch.
- Fix TEST_DATA foreign table ordering consistency. Thanks to
tanguydelignieresaccenture.
- Fix ordering of Oracle data for TEST_DATA export. Thanks to
tanguydelignieresaccenture for the patch.
- Handling Group By and Having Clause inside subqueries (HAVING Clause
misplacement in Ora2Pg). Thanks to newtora2pg for the patch.
- Remove import of tmpdir function of Perl File::Spec module, it fails on
Windows.
- Limit search for spatial type in column to the first line. Thanks to Amit
Kumar for the report.
- Fix case where rewrite of listagg to string_agg is not done. Thanks to
bbellsct for the report.
- Apply reserved word quoting in PG query to for data validation. Thanks to
mmay9CO for the report.
- Prevent full scan when looking for spatial srid and dim. Thanks to Amit
Kumar for the report.
- Fix output format detection condition
- Fix double declaration of variable
- Append CREATE SCHEMA only if CREATE_SCHEMA is enabled for SEQUENCE export.
Thanks to Brian Hayden for the report.
- Fix schema name for procedures/functions when PG_SCHEMA is set. Thanks to
mgole001 for the report.
- Fix data export filename for partition when RENAME_PARTITION is enabled.
Thanks to Simon Pane for the report.
- Fix replacement of SYSTIMESTAMP by statement_timestamp() instead of
CURRENT_TIMESTAMP. Thanks to dstinit for the report.
- Fix methode import when using File::Spec. Thanks to niteshn24 for the
report.
- Apply MOFIDY_TYPE rewriting after REPLACE_AS_BOOLEAN. Thanks to jstmx for
the report.
- Fix inner query IN keyword missing after converting. Thanks to rajatjha28
for the report.
- Fix export of procedure with out parameter when PG_version < 11. Thanks to
Pavel Stehule for the report.
- Remove rewrite of numeric operation with TRUNC() to use interval, too
much false positive. Thanks to Pavel Stehule for the report.
- Force binmode when output is compressed with bzip2. Thanks to Vladimir
Roganov for the report.
- Fix eval() error detection. Thanks to Vladimir Roganov for the report.
- Fix incorrect translation from OUT to INOUT param
- Handling Multiline COMMENT in Ora2Pg.
- Handling Referencing in Triggers.
- Handling Timestamp Datatype.
- Handling XMLELEMENT Conversion.
- [MSSQL] Add parsing of a DDL file to migrate tablesi with indexes and
constraints. The GO keyword that ends a statements must be replaced by
a semi-colon (;) and all brackets must be removed from the source file
before.
- Fix parsing of foreign keys with input file. Thanks to raulcejas04 for
the report.
- Fix schema for trigger function when read from file.
- Remove double quote on custom type when readind table definition from a file.
- Fix use of REPLACE_TABLES and REPLACE_COLS with input files.
- Fix export of procedure to function when pg_supports_procedure is off by
removing the unwanted extra_param parameter. Force pg_supports_outparams to
off when pg_supports_procedure is off. Thanks to Pavel Stehule for the report.
- Remove goldengate suplemental table logging
- Fix pg_supports_ifexists with change in previous commit
- Keep schema in function/procedure name when it is read from file an we are
not in PACKAGE action.
- Fix infinit loop in NLS_SORT replacement.
- Avoid duplicate name in foreign keys constraints
- Fix override of any Ora2Pg configuration directive from command line for
complex values like REPLACE_TABLES.
- Remove sinh, cosh and tanh from unsupported oracle function list
- Fix parsing of stored procedure from file.
- Fix if condition in export_schema.ps1. Thanks to Robin Pringle for the patch.
- Fix replacement by PERFORM
- Add "Schema Export Complete" and "Ora2Pg ending" message to provide enhancement
documented in Issue #1806. Thanks to Simon Pane for the patch.
- Quote PG_USER in ORACLE_FDW user mapping. Thanks to Simon Pane for the patch.
- Remove PASSWORD from keywork list
- Make sure that the column alias for trim() is not prefixed by the schema.
- Fix TRUNCATE TABLE command when the table name needed to be quoted
- Fix column list in COPY statements, columns was doubled.
- Fix port setting for FDW server. Thanks to Aymen Zaiter for the report.
- Added ORACLE_FDW_COPY_FORMAT: binary or csv
- [mysql] Replace json_extrat() with json_extrat_path. Thanks to mgole001 for
the report.
- Remove table alias from column alias after TRIM function. Thanks to Simon
Pane for the patch.
- Fix validation of nullable boolean columns. Thanks to Eckhart Worner for
the patch.
- Fix regular expression for function-based indexes in validation. Thanks to
Eckhart Worner for the patch.
- Fix regression in NULL_EQUAL_EMPTY feature. Thanks to Eric Delanoe for the
report.
- Fix export of foreign keys for PG <= 10
- Fix rename of partitions when REPLACE_TABLES is used on the parent table
and when PREFIX_PARTITION is enabled.
- Set environment variable PGPASSWORD for ORACLE_FDW_COPY_FORMAT and remove
PGPASSWORD from OFBC psql commands. Thanks to Martin Nash for the patch.
Use of PGPASSWORD results in the password being exposed to anyone with
host access. Switching to relying on .pgpass, which can be located in a
custom location for Ora2Pg by setting PGPASSFILE in the session running
Ora2Pg, avoids password exposure.
- Update to use explicit column list for OFBC. Thanks to Martin Nash for the
patch.
- Fix conversion of data format for TO_TIMESTAMP function. Thanks to Priyanshi
Gupta for the report.
- Second fix for bug migrating data in json column. Thanks to mat-efluid for
the report.
- Fix migration of data from CLOB to jsonb isung COPY mode to preserve json
escaping. Thanks to Thomas Herzog for the report.
- Fix quoting for index creation on reserved keyword. Thanks to
moonbeamglitterblossom for the report.
- Fix case where values was transformed as boolean when when a table has the
same name as a data type. Thanks to twiti7 for the report.
- Fix use of TRANSFORM_VALUE when a function is used. Thanks to Thomas Herzog
to the report
- Fix for subquery where clause issue
- Fix incorrect "unsupported partition type" warning for oracle_fdw copy.
Thanks to Martin Nash for the patch.
2024 03 29 - v24.3
This release fix several issues reported since last release and adds some
new features and improvements.
* Add option control the "prefetch" used by oracle_fdw COPY/INSERT
Prior to this change Ora2Pg uses the default "prefetch" of oracle_fdw,
which at the time of writing is 50. Allowing this to be controlled by an
Ora2Pg configuration/option gives the option of increased performance at
the cost of some additional memory on the PostgreSQL side. Thanks to Martin
Nash for the patch.
* Modify the behavior of triggers export with EXPORT_INVALID. It used to
apply to ENABLED or DISABLED triggers instead of real VALID or INVALID
triggers. Export of INVALID triggers will be controlled by EXPORT_INVALID
like others objects like functions, packages, etc. This mean that disabled
triggers that are valid will be exported by default now, this was not
the case before. Thanks to dcgadmin for the feature request.
* Add new configuration directive PGTT_NOSUPERUSER. By default the pgtt
extension is loaded using the superuser privilege when EXPORT_GTT is
activated. Enabled it if you run the SQL scripts generated using a non
superuser user. It will use:
LOAD '$libdir/plugins/pgtt';
instead of default:
LOAD 'pgtt';
Thanks to Simon Martin for the feature request.
Here is the full list of changes and acknowledgements:
- Fix Inf replacement that must only be done with numeric datatype. Thanks to
gael-efluid for the report.
- Fix some replacement of OUTER JOIN (+). Thanks to Carens Kurniawan Wijaya
for the report.
- Fix schema filter with test function count. Thanks to dcgadmin for the
report.
- Fix TEST count objects when a table name is modified. Thanks to korolan
for the report.
- Fix issue with multi style comments. Thanks to newtglobal.com for the patch.
- Fix documentation about EXPORT_INVALID to precise that it also concern
disabled triggers. Thanks to dcgadmin for the report.
- Fix missing FOR EACH clause in trigger export after a regression introduced
by commit fb6b0ad. Thanks to Carens Kurniawan Wijaya for the report.
- Fix data export for table with a geometry column. Thanks to ruralqiu for the
report.
- Revert changes introduced by commit fc7008c, for some obscurs Oracle reasons
the DBA_SDO_GEOM_METADATA doesn't always exist.
Always uses ALL_SDO_GEOM_METADATA instead. Thanks to ruralqiu and
Pierre3939 for the report.
2024 03 07 - v24.2
This release fix several issues reported since last release and adds some
new features and improvements.
* Allow DATA_EXPORT_ORDER to take a filename at value to be able to
give a custom table order. The file must contain the ordered list
of the tables. One table per line in upper case for Oracle. Thanks
to DataCloudGaze for the feature request.
* Add progress bar when --oracle_speed is used to avoid waiting for
the whole data export ends.
* Add replacement of the BITAND function by the & operator
* Add option -f, --format to set the output format for the reports.
It can be html or json. Default to html. Thanks to mgole001 for the
feature request.
* Add automatic addition of the partition key to the primary key. Prefix
all columns with the alias in the query to get data from Oracle.
* Add information about MSSQL masked columns in SHOW_COLUMN.
* Add information about columnstore and compression on MSCSQL tables with
action SHOW_TABLE.
* Add new configuration directive PARTITION_BY_REFERENCE to defined
how to export Oracle partition by reference. Possible values are none,
duplicate or the number of hash partitions to create.
Value 'none' mean no translation and export of partition by reference
like before. Value 'duplicate' will duplicate the referenced column
in the partitioned table and apply the same partitioning from the
referenced table to the partitioned table. If the value is a number,
the table will be partitioned with the HASH method using the value
as the modulo. For example if you set it to 4 it will create 4 HASH
partitions.
Default is none to not export the partitions by reference definition.
Here is the full list of changes and acknowledgements:
- Fix default values in MSSQL function declaration and missing END keyword.
- Fix parsing of MSSQL function with a single query. Thanks to Saravanan Newt
for the report.
- Fix negative Oracle decimal when comparing data. Thanks to es99-24 for the
report.
- Fix typos in documentation. Thanks to Simon Martin for the report.
- Avoid doubling the NAME keyword in the XMLELEMENT() function.
- Remove clause "REFERENCING OLD AS OLD NEW AS NEW" that is useless and throw
an error in PostgreSQL
- Fix DSN for MSSQL in autogenerated configuration file using --init_project.
- Fix parsing of REFERENCING clause in trigger. Thanks to Carens Kurniawan
Wijaya for the report.
- Fix case where ALL_TAB_COLUMNS and ALL_SDO_GEOM_METADATA was used instead
of USER_TAB_COLUMNS. Thanks to rvanouter for the report.
- Fix double quoting in partition by reference where clause.
- Disallow setting of PARTITION_BY_REFERENCE to duplicate when FDW_SERVER
is set.
- Fix trigger export with duplicate FOR EACH clause when a REFERENCING clause
is present. Thanks to Carens Kurniawan Wijaya for the report.
- Fix typo in variable name in ora2pg_scanner
- Fix export of default partition for MySQL. Thanks to Priyanshi Gupta for
the report.
- Fix TEST_DATA action with issues on foreign server and import foreign schema
declaration. Thanks to Florent Jardin for the report.
- Fix export of NOT NULL constraint with column replacement. Thanks to Florent
Jardin for the report.
- Prevent reading file ora2pg_stdout_locker when it does not exist. Thanks to
Florent Jardin for the report.
- Fix translation of timestamp with precision < 6. Thanks to Andrei Briukhov
for the report.
- Fix json_arrayagg with returning clause, type was not translated.
- Fix data export for table partitioned by reference with duplicate method.
The query to extract data on Oracle side add the duplicated column and
perform the join with the referenced table using the FK definition.
- Fix partition by LIST export with a useless cast to text. Thanks to
Priyanshi Gupta for the report.
- Added closing curly bracket for object details. Thanks to andreas42 for
the patch.
- Added quotes around value of "human days cost" and "migration level".
Thanks to andreas42 for the patch.
- Format object type detail output as JSON array. Thanks to andreas42 for
the patch.
- Fix variable declaration in previous commit
- Fix MSSQL table export with nonexistent column auto_created in version
bellow 2017. Thanks to Florent Jardin for the report.
- Use ADD CONSTRAINT syntax to specify name of primary key. Thanks to Martin
Karlgren for the patch.
- Skip unwanted work on PG database when option --oracle_speed is enabled,
especially drop of constraints. Thanks to John Tian for the report.
- Remove trailing ); from primary key statements when reading from file.
Thanks to Martin Karlgren for the patch.
- Fix option in CREATE USER MAPPING for export of DBLINK.
- Fix translation of MSSQL floating point datatype to use float(n) notation.
- Exclude from assessment objects in Oracle bin.
- Add Oracle package HTP and HTF to migration assessment.
- Fix MSSQL foreign key export with multiple columns referenced.
- Fix MSSQL export of unique constraints that was merging columns of several
unique constraints for the same table into a single constraint.
- Fix case of MSSQL datetime default value 0 that must be converted to
'1900-01-01 00:00:00'
- Add rewrite of MSSQL getutcdate() function.
- Fix MSSQL index type and add compression + columnstore information
- Add clause IF NOT EXIST to all CREATE EXTENSION calls
- Fix MSSQL bit data migration through tds_fdw, it is already exported
as boolean by the fdw.
- Fix duplicated indexes in MSSQL export.
- Add export of MSSQL indexes with columns included (CREATE INDEX+INCLUDE).
2023 09 08 - v24.1
This release fix several issues reported since last release and adds some
new features and improvements.
- Replace "set feedback off" by "\set QUIET on;" and "set pagesize 0" with
"\pset pager off". Thanks to Martin Gerhardy for the suggestion.
- Always add package name into search_path of packages functions. Thanks to
janopha for the report.
- Allow to specify a password file to set PG_PWD at PostgreSQL database
connection. If the specified file exists on the system, Ora2Pg will read
the first line to get the password at each call of send_to_pgdb(). It can
be useful in some situation where the password change during data migration.
Thanks to Marius Hope for the feature request.
- Added option --dump_as_json and fixed some json output errors. Thanks to
Martin Gerhardy for the patch.
Here is the full list of changes and acknowledgements:
- Fix schema prefixing of type created in packages stored procedures.
Thanks to janopha for the report.
- Fix perl function get_schema_condition() to use quote_ident() in generated
SQL filters. Thanks to franxav06 for the report.
- Attempt to better test the not null constraint count in Oracle.
- Fix regression with PSQL_RELATIVE_PATH. Thanks to Ryan Taylor for the
report.
- Do not add the partition key to PK if DISABLE_PARTITION is enabled. Thanks
to pavel-moskotin-db for the report.
- Filter list of indexes on name instead of the GENERATED column because
we are missing the ones that have been created automatically by the
Automatic Indexing feature of Oracle 19c. Thanks to Franck Pachot for
the report.
- Fix incompleteness in GRANT action, add grant usage on schema to owner and
users. Thanks to elexus for the report.
- Apply missing --blob_to_lo to import_all.sh script.
- Fix regression in MySQL hash partitions export. Thanks to Shubham Dabriwala
for the report.
- Fix Oracle INTERVAL data export with negative value. Thanks to shubham-yb
for the report.
- Fix double count of not null constraints for TEST action. Thanks to Simon
Pane for the patch.
- Fix replacement of Oracle sys_refcursor in function return type.
- Fix replacement of SQL script setting from Oracle.
- Make scripts executable. Thanks to Martin Gerhardy for the patch.
- Exclude data pump SYS_EXPORT_SCHEMA_.* tables from Oracle export.
- ora2pg: use env based shebang for perl. Thanks to Martin Gerhardy for
the patch.
- Add unsupported clause message for PRAGMA AUTONOMOUS_TRANSACTION when it is
not rewritten by Ora2Pg. Thanks to Martin Gerhardy for the patch.
- Allow schema specific definitions of partitioning columns. Thanks to Martin
Gerhardy for the patch.
- Fixed invalid variable name in read_grant_from_file. Thanks to Martin
Gerhardy for the patch.
- Fix not double quoted column in alter set not null column statement.
Thanks to leonteq-reisg for the patch.
- Bug fixes and special treatment for Types with body. Thanks to Martin
Gerhardy for the patch.
- Avoid redundant definition of the version. Thanks to Martin Gerhardy for
the patch.
- Fix a MySQL exception when the column type is ENUM for version < 5.7.
Thanks to Code-UV for the report.
2023 07 05 - v24.0
This major release adds support to migration of SQL Server database to
PostgreSQL. It also fixes several issues reported since past height months
and adds some new features and improvements.
* Enable the use of ALLOW/EXCLUDE directive with SHOW_* reports and throw
a fatal error if global filters in ALLOW/EXCLUDE are set.
* Add replacement of DBMS_LOCK.SLEEP with pg_sleep
* Split estimate cost details per function/procedure/and package function.
* Add cmin, cmax, ctid to reserved keywords list.
* Add cost for presence of ADD CONSTRAINT in PLSQL code. It needs constraint
name stability.
* Add CLOB_AS_BLOB configuration directive to treat CLOB as BLOB when
exporting data. When enabled Ora2Pg will apply same behavior on CLOB
than BLOB with BLOB_LIMIT setting. This could be useful if you have
large CLOB data. Enabled by default. Thanks to Omar Mebarki for the patch.
* Allow COPY and TABLE type to use the NULLIF construct. Thanks to Luke Davies
for the patch.
* Add new SEQUENCE_VALUES export type to export DDL to set the last values
of sequences from current Oracle database last values like the following
statements: ALTER SEQUENCE departments_seq START WITH 290;
Thanks to sergey grinko for the feature request.
* Add replacement of Oracle variable : varname into PG :'varname'.
* Add SQL Server migration to Ora2Pg. Most of the SQL Server objects are
supported as well as data export. Translation of the TSQL stored
procedures to plpgsql is complicated because of the lack of statement
separator in TSQL but as usual Ora2Pg is doing is best to do as much
work as possible. Migration assessment is also possible with SQL Server
database. There is some dedicated configuration directives added to
ora2Pg.conf.
* Add support to MySQL PARTITION BY KEY() with a translation to HASH
partitioned table using the PK/UK definition of the table or the
columns specified in the KEY() clause. Thanks to Shubham Dabriwala
for the report.
* Make EXPORT_INVALID configuration directive works with TRIGGER export.
Until now disabled triggers were not exported, setting EXPORT_INVALID
to 1 will force the export of disabled triggers. Thanks to chetank-yb
for the report.
* Add support of MySQL generated default value on update. For example:
CREATE TABLE t1 (
dt DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);
Ora2Pg will translate this syntax into a trigger on the table to force
the value of the column on an update event.
Thanks to heysky for the report.
* Add translation of ST_GEOMETRY data type to PostGis geometry datatype.
* Replace ROWNUM in target list with a "row_number() over ()" clause. Thanks
to Rui Pereira for the report.
New configuration directives:
* Add configuration directive ST_GEOMETRYTYPE_FUNCTION to be able to set the
function to use to extract the geometry type from a ST_Geometry column.
Default: ST_GeometryType, example it should be set to sde.ST_GeometryType
for ArcSDE. Thanks to Albert88373 for the report.
* Add four new configuration directive to be able to change or prefix the
functions used to extract information from ST_Geometry object and values.
- ST_SRID_FUNCTION: Oracle function to use to extract the srid from
ST_Geometry meta information. Default: ST_SRID, for example it should be
set to sde.st_srid for ArcSDE.
- ST_DIMENSION_FUNCTION: Oracle function to use to extract the dimension
from ST_Geometry meta information. Default: ST_DIMENSION, for example it
should be set to sde.st_dimention for ArcSDE.
- ST_ASBINARY_FUNCTION: Oracle function to used to convert an ST_Geometry
value into WKB format. Default: ST_ASBINARY, for example it should be set
to sde.st_asbinary for ArcSDE.
- ST_ASTEXT_FUNCTION: Oracle function to used to convert an ST_Geometry
value into WKT format. Default: ST_ASTEXT, for example it should be set
to sde.st_astext for ArcSDE.
Thanks to Albert88373 for the report.
* Add INSERT_ON_CONFLICT configuration directive. When enabled this instruct
Ora2Pg to add an ON CONFLICT DO NOTHING clause to all INSERT statements
generated for this type of data export. Thanks to Clemens Rieder for the
feature request.
Backward compatibility:
* Change the behavior of CASE_INSENSITIVE_SEARCH to allow the use of a
collation instead of the citext extension. To disable the feature the
value none can be used. If the migration is not MSSQL this feature is
disabled.
* Remove PREFIX_PARTITION configuration directive, it is now replaced by
the RENAME_PARTITION directive. Previous behavior was to construct the
partition name from the table name, the partition name and the sub
partition name if any. The problem is that we often reach the max length
for an object name and this leads to duplicate partition name. Now, when
RENAME_PARTITION is enabled the partition tables will be renamed
following rules:
<tablename>_part<pos>
where "pos" is the partition number. For subpartition this is:
<tablename>_part<pos>_subpart<pos>
If this is partition/subpartition default:
<tablename>_part_default
<tablename>_part<pos>_subpart_default
This change will break backward comaptibilty, if PREFIX_PARTITION is
still set, it will simply enable RENAME_PARTITION.
* Set START value to MINVALUE when a sequence is cycled and that the START
value is upper that MAXVALUE. Thanks to Shane Borden for the report.
Here is the full list of changes and acknowledgements:
- Fix MODIFY_STRUCT that was not working with MySQL. Thanks to Code-UV for
the report.
- Fix license string in Makefile.PL. Thanks to RodRaen for the report.
- Do not remove non alphanumeric character in index name. Thanks to gwidt
for the report.
- Reorder trigger event when the update of column is not the last one. Thanks
to tayalarun1 for the report.
- Fix export of MySQL function containing special characters and white spaces
in names. Thanks to Shubham Dabriwala for the report.
- Fix grant export for partitions. Thanks to elexus for the report.
- Add some other transformation for sqlplus/psql scripts.
- Remove comma as possible separator for values in DEFINED_PK, it was
preventing the use of a function with multiple parameters.
- Fix export of geometry tables when PG_SCHEMA is set.
- Add rewriting of some sqlplus settings to psql settings.
- Fix TABLESPACE export for partitioned tables. Thanks to elexus for the
report.
- Fix for Issue #1637. Thanks to Simon Pane for the patch.
- Fix typo in --init_project directories tree generation for sequences
values.
- Fix alias in view target list for function call without alias defined in
MySQL export. Thanks to Shubham Dabriwala for the report.
- Fix Mysql procedure export when a datatype with precision is used in
parameter list. Thanks to Shubham Dabriwala for the report.
- Fix collation on string default values. Thanks to Shubham Dabriwalafor
the report.
- Exclude recycle bin object from ALL_TAB_COLUMNS lookup. Thanks to Dave
Betterton for the report.
- Fix data types translation (TINYINT|SMALLINT|MEDIUMINT|INTEGER|BIGINT|INT)
for MySQL table export. Thanks to Shubham Dabriwala for the report.
- Do not export synonym destination table with table_owner when EXPORT_SCHEMA
is disabled. Thanks to Priyanshi Gupta for the report.
- Fully qualify calls to get_sequence_last_values() when PG_SCHEMA is set.
Thanks to Marius Hope for the report.
- Fix regression on exporting view as table when VIEW_AS_TABLE contains
regexp. Thanks to Neil Bower for the report.
- Fix missing execution of initial command statements at start of TEST_DATA
action and on both side, those applying to source and destination. Thanks
to Petter Jacobsen for the report.
- Fix script to get sequence last value with TEST action. Thanks to franxav06
for the patch.
- Prepend PERFORM before call to DBMS_OUTPUT.* when USE_ORAFCE is enabled.
- Disable USE_ORAFCE when export type is SHOW_REPORT.
- Extending the enhancement in Pull Request #1621 to the Oracle_FDW user
mapping. Thanks to Simon Pane for the patch.
- Changed prefix string to "DIFF:" in test report. Thanks to Simon Pane for
the patch.
- Fix cases where %ROWCOUNT was not correctly replaced. Thanks to Rui Pereira
for the report.
- Fix parsing of ORACLE_DSN when creating foreign server in COPY mode. Thanks
to Luke Davies for the report.
- Fix for Issue #1622, #1627. Thanks to Simon Pane for the patch.
- Fix index creation with DESC order in COPY action when DROP_INDEXES is
enabled. Thanks to Luke Davies for the report.
- Fix for Issue #1610, #1612, #1617 and #1381. Thanks to Simon Pane for the
patch.
- Fix typo in sqlnet.ora name (was sqlnet.or). Thanks to Martin Nash for the
patch.
- Fix data export, REPLACE_QUERY was not applied. Thanks to Bachev Constantin
for the report.
- Fix call to replace_sys_context().
- Fix timestamp(n) data type translation.
- Remove use of column GENERATION_EXPRESSION for MySQL version < 5.7.0. Thanks
to Hans Choi for the report.
- Fix conversion of DATE datatype to timestamp(0) instead of timestamp. Thanks
to Akhil Reddy for the report.
- Add NVARCHAR/NCHAR defaut convertion data types to DATA_TYPE configuration
directive in ora2pg.conf comments. Thanks to Akhil Reddy for the report.
- Rename method _get_partitions_type function into _get_partitions_list.
- Fix synonym export when no schema information is available.
- Fix support of REFERENCING clause in triggers.
- Fix partition output file renaming with new RENAME_PARTITION directive.
Thanks to Rahul Barigidad for the report.
- Fix export of the ROWNUM clause when there is a variable.
- Fix sprintf placeholders in geometry queries.
- Fix some others issues with row count report.
- Fix row count with destination schema and when the PostgreSQL table
doesn't exist.
Thanks to bizen-ya for the report.
- Fix tests comparison with the different settings of EXPORT_SCHEMA,
SCHEMA and PG_SCHEMA. Thanks to Marius Hope and bizen-ya for the
report.
- Fix St_AsText() call for MySQL data extraction.
- Add column count comparison for MySQL
- Export multi column partition by list as an expression with concat
operator. Multi column partition by list is not supported by PostgreSQL.
- Fix creation of non existant indexes on partition. Thanks to Shubham
Dabriwala for the report.
- Fix MySQL function export when there is no BEGIN clause. Thanks to
Shubham Dabriwala for the report.
- Fix MySQL export of unsigned numeric. Thanks to Shubham Dabriwala for
the report.
- Fix MySQL output with wrong synthax for JOIN without ON clause. Thanks
to Shubham Dabriwala for the report.
- Fix virtual column export. Thanks to Rafal Hollins for the report.
- Fix index creation on partition with no columns for MySQL export.
Thanks to Shubham Dabriwala for the report.
- Fix export of MySQL auto_increment when PG_INTEGER_TYPE is disabled.
Thanks to Shubham Dabriwala for the report.
- Fix MySQL subpartition export. Thanks to Sanyam Singhal for the report.
- Move any INTO clause in CONNECT BY query to the final SELECT on the
resulting CTE. Thanks to taptarap for the report.
- Fix translation of MySQL curtime() function in default values. Thanks
to Shubham Dabriwala for the report.
- Fix possible "Nested quantifiers in regex" error when exporting package
with package name containing regex special characters. Thanks to durandm70
for the report.
- Fix documentation about use of unique key for ORACLE_COPY.
- Fix extra comma at end of a CHECK contraint. Thanks to Shubham Dabriwala
for the report.
- Always add DROP TYPE statements with package export even if DROP_IF_EXISTS
is not enabled. Thanks to Rui Pereira for the report.
- Fix default value of simple dot in MySQL export. Thanks to Shubham
Dabriwala for the report.
- Fix regression in data type translation after fix on unsigned numeric type.
Thanks to Shubham Dabriwala for the report.
2022 10 08 - v23.2
This release fix several issues reported since past height months and
adds some new features and improvements.
* Add export of MySQL KEY and LINEAR KEY partitioning, translated as HASH
partitioning.
* Allow export of object with dollar sign is his name.
* Add export of CHECK constraints for MySQL >= 8.0.
* Add Functional/Expression indexes export from MYSQL.
* Add export of MySQL virtual column. Thanks to Shubham Dabriwala for the
* Remove scale and precision of a numeric if the scale is higher than the
precision. PostgreSQL does not support decimal/numeric attributes where
the scale is explicitly higher than the precision.
* Add command line option --drop_if_exists to add statement to drop objects
before creation if it exists. It corresponds to the DROP_IF_EXISTS
* Add option -C | --cdc_file to be able to change the name of the default file
used to store/read SCN per table during export. Default is TABLES_SCN.log in
the current directory. This is the file written by the --cdc_ready option.
* Add multiprocess to count rows in PostgreSQL tables (TEST_COUNT) using -P
command line option.
* Add support to PostgreSQL 14 procedure with out parameters.
* Set default PostgreSQL database target version to 14.
New configuration directives:
* Add configuration directive MVIEW_AS_TABLE and command line option
--mview_as_table to set which materialized view to export as table.
By default none. Value must be a list of materialized view name or
regexp separated by space or comma. If the object name is a materialized
view and the export type is TABLE, the view will be exported as a create
table statement. If export type is COPY or INSERT, the corresponding data
will be exported.
* Add configuration variable FDW_IMPORT_SCHEMA to rename the schema where
foreign tables for data migration will be created. If you use several
instances of ora2pg for data migration through the foreign data wrapper,
you might need to change the name of the schema for each instance.
Default: ora2pg_fdw_import
* Add TRANSFORM_VALUE configuration directive to apply an expression when
retrieving data from Oracle. For example:
TRANSFORM_VALUE CLOB_TABLE[CHARDATA:translate("CHARDATA", chr(0), ' ')]
to replace all Oracle char(0) in a string by a space character.
* Add EXCLUDE_COLUMNS configuration directive. Unlike MODIFY_STRUCT
that is used to redefine a table structure, this directive allow
to specify a list of columns per table that must be excluded from
the export. For example:
EXCLUDE_COLUMNS T1(nocol1,nocol2) T2(nocol1,nocol2)
* Add new configuration directive EXPORT_GTT to export Oracle Global Temporary
Table using syntax recognized by the pgtt extension. For more information see
https://github.com/darold/pgtt Default is to not export global temporary
table as they are not supported natively by PostgreSQL.
* Add new configuration option NO_EXCLUDED_TABLE. By default Ora2Pg exclude
from export some Oracle "garbage" tables that should never be part of an
export. This behavior generates a lot of REGEXP_LIKE expressions which are
slowing down the export when looking at tables. To disable this behavior
enable this directive, you will have to exclude or clean up later by
yourself the unwanted tables. The regexp used to exclude the table are
defined in the array @EXCLUDED_TABLES in lib/Ora2Pg.pm. Note this is behavior
is independent to the EXCLUDE configuration directive.
Backward compatibility:
* Force rewrite of all invalid date starting with zero year 0000 into 1970-01-01
when it is a default value and NULL for data. Old behavior was to only replace
0000-00-00 date.
* Until now there was a lot of untranslated call to TRUNC(date) because
Ora2Pg is unable to detect that the parameter is a date or a number.
The problem is that Oracle has TRUNC(number) too and Ora2Pg try to not
apply the transformation if there is a doubt. In most of the migration
have met very few TRUNC(number) so now all call to TRUNC()
will be converted to date_trunc(). There must be false positive rewrite
but this should be far less work than the actual situation.
Here is the full list of changes and acknowledgements:
- Fixed PostgreSQL "relation not found error" in _dump_fdw_table(), PostgreSQL
search_path was not being used. Thanks to James Schriever for the patch.
- Fix year and month quoting as reserved words when they are used as aliases.
Thanks to duursma for the report.
- Fix conversion of to_number(substr(...)) when PG substr() return empty
string where Oracle return NULL which make the conversion to numeric fail.
The fix using (nullif(substr(...), )::numeric) only concern TABLE export.
Thanks to Menelaos Perdikeas for the report.
- Add export of MySQL KEY and LINEAR KEY partitioning, now exported as HASH
partitioning. Thanks to Sanyam Singhal for the report.
- Remove $ sign from characters that require object name quoting.
- Fix export of objects with the $ sign in the name. Thanks to yano-rxa and
duursma for the report.
- Prevent translation of EXEC when used as alias. Thanks to Rui Pereira for
the report.
- Fix MySQL enum data type export with regression introduced by commit 24a476.
Thanks to Shivansh Gahlot for the report.
- Rename ORACLE_FDW_TRANSFORM to TRANSFORM_VALUE to apply in all case an
expression when retrieving data from Oracle. For example:
TRANSFORM_VALUE CLOB_TABLE[CHARDATA:translate("CHARDATA", chr(0), ' ')]
to replace all Oracle char(0) in a string by a space character.
- Fix add_month() translation failing with some use cases. Thanks to duursma
for the report.
- Add export of CHECK constraints for MySQL >= 8.0. Thanks to Rahul
Barigidad for the report.
- Fix MySQL unsigned numeric data type conversion. Thanks to Rahul Barigidad
for the report.
- Add Functional/Expression indexes fail while exporting from MYSQL. Thanks to
Shubham Dabriwala for the report.
- Fix export of descending indexes for MySQL. Thanks to Shubham Dabriwala for
the report.
- Force MySQL auto increment sequence to start at 1 when value is 0.
Thanks to Rahul Barigidad for the report.
- Fix conversion of MySQL decimal(p,s) to keep the original datatype
instead of use of real or double.
- Add export of MySQL virtual column. Thanks to Shubham Dabriwala for the
report.
- Fix export of MySQL function with return clause only. Thanks to Shubham
Dabriwala for the report.
- Remove scale and precision of a numeric if the scale is higher than the
precision. PostgreSQL does not support decimal/numeric attributes where
the scale is explicitly higher than the precision. Thanks to Rahul Barigidad
for the report.
- Fix export of comment for views. Thanks to gh-k-murata for the report.
- Add command line option --drop_if_exists to add statement to drop objects
before creation if tehy exists. It corresponds to the DROP_IF_EXISTS
configuration directive. Thanks to Yoni Sade for the feature request.
- Add option --mview_as_table to documentation.
- Add configuration directive MVIEW_AS_TABLE and command line option
--mview_as_table to set which materialized view to export as table.
By default none. Value must be a list of materialized view name or
regexp separated by space or comma. If the object name is a materialized
view and the export type is TABLE, the view will be exported as a create
table statement. If export type is COPY or INSERT, the corresponding data
will be exported.
- Disable EXPORT_GTT when export type is not TABLE. Thanks to gh-k-murata for
the report.
- Fix generated external servers wrongly placed in a schema. Thanks to duursma
for the report.
- Add configuration variable FDW_IMPORT_SCHEMA to rename the schema where
foreign tables for data migration will be created. If you use several
instances of ora2pg for data migration through the foreign data wrapper,
you might need to change the name of the schema for each instance. Default
to ora2pg_fdw_import. Thanks to James Schriever for the feature request.
- Fix wrong conversion of rownum clause when a subquery is used. Thanks to
Rui Pereira for the report.
- Escape comma and backslashes in BFILE data export. Thanks to duursma for
the patch.
- Fix possible infinite loop in Oracle outer join parsing. Thanks a lot to
yano-rxa for the report.
- Remove privileges default settings on views exported from SYNONYMs
- Add support for Rectangle geometry type. Thanks to duursma for the patch.
- Fix double replacement of IS NULL/IS NOT NULL when NULL_EQUAL_EMPTY is
enabled.
- Add CHECK not null only constraints to not null constraint count.
- Fix CHECK NOT NULL only constraints that was not exported by generating
NOT NULL constraints instead. They are exclude from the count of CHECK
constraint as suggested by Florent Jardin but a count difference persist
for NOT NULL constraints.
- Fix TYPE export when SCHEMA and PRESERVE_CASE have different values. Thanks
to Florent Jardin for the report.
- Fix custom exception replacement. Thanks to Rui Pereira for the report.
- Fix Collection and Polygon geometry INTERNAL export.
Thanks to duursma for the patch.
- Fix export of efile with parenthesis. Thanks to duursma for the report.
- Fix wrong column indices used in spatial index extraction. Thanks to duursma
for the report.
- Fix call of ST_GeomFromText() with WKT export. Thanks to duursma for the
report.
- Disable USE_LOB_LOCATOR with WKT geometry export type, ST_GeomFromText and
SDO_UTIL.TO_WKTGEOMETRY functions return a CLOB instead of a geometry.
Thanks to duursma for the report.
- Fix INTERNAL conversion uses the srid from the object instead of the
meta-data. Thanks to duursma for the report.
- Fix regression in data export when REPLACE_AS_BOOLEAN is set. Thanks to
Juri Berlanda for the report.
- Fix call to procedure using dblink. Thanks to Rui Pereira for the report.
- Keep untouched call to DBMS_OUTPUT functions if USE_ORAFCE is enabled.
Thanks to Sanyam Singhal for the report.
- Partial fix for MySQL subpartitioning export.
- Fix partitions export for MySQL. Thanks to Sanyam Singhal for the report.
- Fix generation of export_all.sh following the operating system.
- Add information of use of PARALLEL_TABLES with COPY, INSERT and TEST_DATA
actions. It is also useful with TEST, TEST_COUNT, and SHOW_TABLE if
--count_rows is used for real row count.
- Prevent calling real rows count twice with TEST action, and allow it for
the SHOW_TABLE action.
- Handle count errors when single process.
- Move row count wait for all child die to the right place
- Fix rewrite of nested replace() functions in CHECK constraint. Thanks to
Menelaos Perdikeas for the report.
- Fix call of procedures with out parameters when it is not declared in a
package. Thanks to taptarap for the report.
- Some minor code improvement. Thanks to Markus Elfring for the patch.
- Set encoding to read configuration file to utf8.
- Remove useless multiple semi-colon after END of a function.
- Fix conversion of regexp_replace() by always appending the 'g' modifier.
Thanks to Rui Pereira for the report.
- Fix synonym detection to avoid listing public synonym when no schema is
specified. Thanks to Dilan Salinda for the report.
- Fix regexp error with multi-line comment in default value declaration. Thanks
to taptarap for the report.
- Add missing sub-partition key in partitioned table primary key. Thanks to
downvoteit for the report.
- Replace all invalid date starting with zero year 0000- to 1970-01-01 when it
is a default value and NULL for data. Old behavior was to only replace
0000-00-00 date. Thanks to duursma for the report.
- Enclose \i path to data file beween quote to fix import of table with space
in their name.
- Add PARTITION to the list of reserved work and fix custom keywords list from
ORA_RESERVED_WORDS that was not applied. Thanks to markhooper99 for the
report.
- Add LOAD of pgtt extension before creating global temporary table with TABLE
:export. Thanks to duursma for the report.
- Fix case where package names should be lower cased. Thanks to Sergey Petrov
for the patch.
- Cover more case where ALTER ... OWNER TO should not be generated.
- Fix case where ALTER ... OWNER TO should not be generated when a view as
table definition was not exported.
- Fix sub-partition unique and primary keys that lacks columns part of the
partition key. Thanks to downvoteit for the report.
- Path for function_per_file are mixed case enabled now. Thanks to Sergey
Petrov for the patch.
- Fix AUTOINCREMENT script to set last value to sequences for serial an
identity column for PG version < 12. Thanks to Jaouad Bouras for the report.
- Fix detection of ENUM data type for MySQL
- Fix issue when exporting table with a geometry column. The search for
the SDO_GTYPE need a FROM clause with a FQDN table when the connection
user is not the same as the table schema. Thanks to Argo64 for the report.
- Rewrite numeric operation with ADD_MONTH(), LAST_DAY() and TRUNC() to use
interval. Thanks to duursma for the report.
- Fix rewrite of CONNECT BY in cursors and just after a BEGIN. Thanks to
taptarap for the report.
- Add partition keys to unique index on partitioned table. Thanks to
downvoteit for the report.
- Fix case where global variable are tested against NULL. Thanks to duursma
for the report.
- Fix remove of %ROWTYPE in function argument and returned data type. Add
regression test. Thanks to Eric Delanoe for the report.
- Fix case clause in autoincrement parameters. Thanks to jbouras for the
report.
- Fix typo in ORACLE_FDW_TRANSFORM example.
- Fix progress bar output in quiet mode. Thanks to Sanyam Singhal for the
report.
- Fix error Can't locate object method "gzclose" via package "IO::File".
Thanks to Sanyam Singhal for the report.
- Fix cases where translation of function with out parameter was not done
correctly.
- Fix translation of function with out parameter that returns a value. For
example the following Oracle function:
CREATE FUNCTION foo(a int, OUT b int)
RETURN boolean IS
BEGIN
b := a;
RETURN true;
END;
is now translated by adding an extra out parameter for the return value:
CREATE OR REPLACE FUNCTION foo (a integer, OUT b integer,
OUT extra_param boolean)
RETURNS record AS $body$
BEGIN
b := a;
extra_param := true;
RETURN;
END;
$body$ LANGUAGE PLPGSQL STABLE;
Thanks to Akhil Reddy for the report.
- Fix undefined database connection handle. Thanks to Alexander for the report
- Fix case preservation for row count in Oracle side with TEST_COUNT action..
Thanks to Veka for the report.
- Only generate the Powershell script when we are running on a Windows
operating system.
- Fix #1400 and generate PowerShell script "export_schema.ps1". Thanks to
moh-hassan for the report.
- Fix rewriting assignment of a global variable using SELECT INTO. Thanks to
duursma for the report.
- Fix partition export for MySQL. Thanks to Sanyam Singhal for the report.
- Apply WHERE clause to FDW data export.
- Fix useless ST geometry parsing. Thanks to jieguolove for the report.
- Replace backslash with slash in BFILE filename when destination data type is
text or efile.
- Fix RAW(16)/RAW(32) data export when MOFDIFY_TYPE is used on the column.
Thanks to Sergey Evseev for the report.
- Fix ST_SRID() call. Thanks to jieguolove for the report.
- Skip table data export when the table has no column defined. This was
generating a fatal error.
- Fix untranslated function returned data type when there was a comment just
after. The comment is removed. Thanks to taptarap for the report.
- Fix other fetching all-column-all-table properties for every table.
Thanks to Sergey Petrov for the report.
- Fix fetching all-column-all-table properties for every table. Thanks to
Sergey Petrov for the report.
- Remove any comments between RETURN and returned type to not break parsing.
Thanks to taptarap for the report.
- Fix global variables in the DECLARE section are not replaced if used with
a package name. Thanks to taptarap for the report.
- Remove renaming of dist configuration file under Windows OS. Thanks to
Julien Monticolo and ohamed Hassan for the report.
- Fix remaining data export query failure. Thanks to Sung Woo Chang for the
report.
- Fix data export, query to retrieve data was broken since change for GTT.
- Set function as VOLATILE when there is CALL in the body.
- Add support to PG14 procedure out parameters. Thanks to Rui Pereira for the
feature request.
- Fix missing parenthesis in index column expression with input file.
- Fix missing END keyword after embedded CASE clause in a package function.
Thanks to taptarap for the report.
- Fix conversion of dbms_lob.substr() where second and third parameters
must be inverted. Thanks to taptarap for the report.
- Fix an other case of wronf NOT NULL detection from input file.
- Fix detection of NOT NULL constraint in input file.
- Do not quit on error "Undefined subroutine &Ora2Pg::ReadLine", just
continue to be able to leverage an Oracle Wallet (SEPS) when no Oracle
user and password are provided. If you want to use the interactive mode
to type the username and password at command line you must install the
Perl package Term::ReadKey before. Thanks to Simon Pane for the report.
- Fix partitioning by LIST, only the first value of a list was exported.
Thanks to Sergey Grinko for the report.
- Fix quoting of DEFAULT NULL. Thanks to Veka for the report.
- Fix unwanted multiple CALL keywords. Thanks to taptarap for the report.
- Add assessment counter for FND_* packages.
- Fix LONG RAW export as bytea in COPY mode. Thanks to Helena Adiduyulmus for
the report.
- Add new configuration option NO_EXCLUDED_TABLE.
By default Ora2Pg exclude from export some Oracle "garbage" tables that
should never be part of an export. This behavior generates a lot of
REGEXP_LIKE expressions which are slowing down the export when looking at
tables. To disable this behavior enable this directive, you will have to
exclude or clean up later by yourself the unwanted tables. The regexp used
to exclude tables are defined in the array @EXCLUDED_TABLES in lib/Ora2Pg.pm
This behavior is independent to the EXCLUDE configuration directive. Thanks
to Peter Humaj for the feature request.
- Replace all remaining CURSORNAME%NOTFOUND with NOT FOUND
- Change translation to SYSDATE from LOCALTIMESTAMP to statement_timestamp()
in non PL/SQL code.
- Prevent append of SECURITY DEFINER when a procedure execute transaction
control statements (ex: COMMIT). When defined with this clause an error
is thrown. Thanks to Suman Michael for the report.
2022 02 10 - v23.1
This release fix several issues reported since past four months and
adds some new major features and improvements.
* Add use of greatest/least functions from new version of Orafce when
required to return NULL on NULL input like Oracle.
* ALLOW and EXCLUDE configuration values can now be read from a file.
Use -a filename or -e filename to specify the list of tables that need
to be filtered. This is useful if you have a lot of table to filter.
* Add possibility to use of System Change Number (SCN) for data export or
data validation by providing a specific SCN. It can be set at command
line using the -S or --scn option. You can give a specific SCN or if you
want to use the current SCN at first connection time set the value to
'current'. To use this last case the connection user must have the role
"SELECT ANY DICTIONARY" or "SELECT_CATALOG_ROLE", the current SCN is
looked at the v$database view.
Example of use:
ora2pg -c ora2pg.conf -t COPY --scn 16605281
This adds the following clause to the query used to retrieve data for example:
AS OF SCN 16605281
You can also use th --scn option to use the Oracle flashback capability by
specifying a timestamp expression instead of a SCN. For example:
ora2pg -c ora2pg.conf -t COPY --scn "TO_TIMESTAMP('2021-12-01 00:00:00', 'YYYY-MM-DD HH:MI:SS')"
This will add the following clause to the query used to retrieve data:
AS OF TIMESTAMP TO_TIMESTAMP('2021-12-01 00:00:00', 'YYYY-MM-DD HH:MI:SS')
or for example to only retrieve yesterday's data:
ora2pg -c ora2pg.conf -t COPY --scn "SYSDATE - 1"
* Add json output format to migration assessment. Thanks to Ted Yu for the patch.
* Add new TO_CHAR_NOTIMEZONE configuration directive to remove any timezone
information into the format part of the TO_CHAR() function. Disabled by default.
Thanks to Eric Delanoe for the report.
Note that the new default setting breaks backward compatibility, old behavior
was to always remove the timezone part.
* Add new configuration directive FORCE_IDENTITY_BIGINT. Usually identity
column must be bigint to correspond to an auto increment sequence so
Ora2Pg always force it to be a bigint. If, for any reason you want
Ora2Pg to respect the DATA_TYPE you have set for identity column then
disable this directive.
* Add command line option --lo_import. By default Ora2Pg imports Oracle BLOB
as bytea, the destination column is created
using the bytea data type. If you want to use large object instead of bytea,
just add the --blob_to_lo option to the ora2pg command. It will create the
destination column as data type Oid and will save the BLOB as a large object
using the lo_from_bytea() function. The Oid returned by the call to
lo_from_bytea() is inserted in the destination column instead of a bytea.
Because of the use of the function this option can only be used with actions
SHOW_COLUMN, TABLE and INSERT. Action COPY is not allowed.
If you want to use COPY or have huge size BLOB ( > 1GB) than can not be
imported using lo_from_bytea() you can add option --lo_import to the
ora2pg command. This will allow to import data in two passes:
1) Export data using COPY or INSERT will set the Oid destination column
for BLOB to value 0 and save the BLOB value into a dedicated file. It
will also create a Shell script to import the BLOB files into the
database using psql command \lo_import and to update the table Oid
column to the returned large object Oid. The script is named
lo_import-TABLENAME.sh
2) Execute all scripts lo_import-TABLENAME.sh after setting the
environment variables PGDATABASE and optionally PGHOST, PGPORT, PGUSER,
etc. if they do not correspond to the default values for libpq.
You might also execute manually a VACUUM FULL on the table to remove
the bloat created by the table update.
Limitation: the table must have a primary key, it is used to set the
WHERE clause to update the Oid column after the large object import.
Importing BLOB using this second method (--lo_import) is very slow so it
should be reserved to rows where the BLOB > 1GB for all other rows use
the option --blob_to_lo. To filter the rows you can use the WHERE
configuration directive in ora2pg.conf.
* Add command line option --cdc_ready to use current SCN per table when
exporting data and register them into a file named TABLES_SCN.log This
can be used for Change Data Capture (CDC) tools.
* Allow to export only invalid objects when EXPORT_INVALID is set to 2
* Disable per partition data export when a WHERE clause is define on the
partitioned table or that a global WHERE clause is defined.
Backward compatibility:
Ora2Pg used to removr any timezone information from the TO_CHAR() format
function. To recover this behavior set TO_CHAR_NOTIMEZONE to 1 in ora2pg.conf
Complete list of changes:
- Replace PERFORM by CALL when the stored procedure is a procedure. Thanks
to Rui Pereira for the report.
- Fix open cursor translation when using is in the query but not as keyword.
Thanks to taptarap for the report.
- Fix replacement of global variables in DECLARE section. Thanks to taptarap
for the report.
- Fix missing suffix in function name with autonomous transaction when export
schema was enabled and fix revoke and owner to wrapper function. Thanks to
Sergey Grinko for the report.
- Fix export of type declaration in packages without body. Thanks to Sergey
Grinko.
- Fix column name duplicates when exporting data of partition. Thanks to
Sergey Grinko for the report.
- Fix BLOB export with INSERT mode, call decode() was missing.
- Fix applying of DEFAULT_PARALLELISM_DEGREE hint that was not working
anymore for a long time. Thanks to Marcel Pils for the patch.
- Update documentation about PARALLEL_TABLES and view export. Thanks to
xinferum for the report.
- Fix unwanted quote escaping in global variable constant. Thanks to
sergey grinko for the report.
- Fix export of global variable when there is function in the default value.
- Fix end of statements in last merged PR.
- Add json output format to migration assessment. Thanks to Ted Yu for
the patch.
- Fix parsing of package when a comment follow the AS keyword. Thanks to
Eric Delanoe for the report.
- Adapt MAXVALUE for identity columns if the datatype has been changed to
integer.
- Fix a regression on data validation introduced with commit to fix data
export of virtual column.
- Fix Can't locate object method is_pk via package Ora2Pg error
- Exclude unique keys using expression to validate data.
- Fix ORDER BY clause for data validation.
- Fix error on open pragma when encoding is not set.
- Fix a regression in data export of virtual column. Thanks to Code-UV and
IgorM12 for the report.
- Fix a second regression with empty column name in target list to
retrieve data.
- Fix PG version to enable virtual column.
- Fix binmode when it is set to raw or locale to not call encoding() in
open pragma.
- Fix regression in export view as table. Thanks to Sebastian Albert for
the report.
- Update Copyright year.
- Quote tables names when necessary during TEST action.
- Fix undefined call to auto_set_encoding().
- Add test count of column per table and add output of the PG table struct
modified to be used with MODIFY_STRUCT.
- Fix handling of PRESERVE_CASE with update au sequences values
- Fix handling of PRESERVE_CASE with TEST_DATA
- Fix unwanted replacement of sysdate operation to epoch. Thanks to taptarap
for the report.
- Remove extra END clause at end of package function when a space or a
comment was present. Thanks to taptarap for the report.
- Fix missing import of module Encode. Thanks to Menelaos Perdikeas for
the report.
- Fix case where data type defined in function was not exported when
EXPORT_SCHEMA was enabled. Thanks to Eric Bourlon for the report.
- Fix missing EXECUTE on OPEN CURSOR statements. Thanks to taptarap for
the report.
- Fix missing declaration of min() function in Oracle.pm. Thanks to
nicscanna for the report.
- Fix SYSDATE subtract of seconds instead of days
- Fix PERFORM replacement in CTE. Thanks to taptarap for the report.
- Fix wrong stored procedure code conversion when use types named with
"default" and broken decode to case translation. Thanks to taptarap
for the report.
- Add missing import of FTS indexes in script import_all.sh. Thanks to
vijaynsheth for the report.
- Fix another procedure parsing with return. Thanks to Eric Bourlon for
the report.
- Fix case where parenthesis are not added to index creation.
- Add creation of the uuid extension when it is used.
- Add HTML report of tables and columns with name > 63 characters.
- Add report of DBMS_ERROR and Quartz Scheduler tables found.
- Add mark (date?) on columns of DATE data type in Oracle to check if
it should be translated into date instead of default timestamp.
- SHOW_COLUMN: mark column data type with (numeric?) when it is a NUMBER
without precision.
- SHOW_TABLE+SHOW_COLUMN: Add mark of tables and columns name > 63
characters
- Fix translation of TYPE ... AS TABLE OF ...
- Fix parsing of function call in check constraints. Thanks to Menelaos
Perdikeas for the report.
- Fix missing data export file for partitioned tables when TRUNCATE_TABLE
was disabled. Thanks to Menelaos Perdikeas for the report.
- Fix named parameter inserted in procedure call with inout parameters.
Thanks to Rui Pereira for the report.
- Fix unwanted quoting of index columns clause when there is an operation.
Thanks to Menelaos Perdikeas for the report.
- Move comment in procedure parameters before the BEGIN. Thanks to Eric
Bourlon for the report.
- Fix parsing of FOR CUSOR followed by a parenthesis. Thanks to Eric Bourlon
for the report.
- Fix parsing of TYPE ... IS REF CUSOR declaration in procedures. Thanks to
Eric Bourlon for the report.
- Add replacement of SDO_CS.TRANSFORM into ST_Transform. Thanks to mukesh3388
for the report.
- Add missing table namer to index renaming.
- Create a function for index renaming for code reuse.
- Fix support translation of type VARRAY from store procedure. Thanks to
Eric Bourlon for the report.
- Fix conversion of SQL%ROWCOUNT when part of a string concatenation. Thanks
to boubou191911 for the report.
- Remove other non alphanumeric character from index name. Thanks to Menelaos
Perdikeas for the report.
- Fix date formatting when error is logged with INSERT failure. Thanks to
xinjirufen for the report.
- Remove possible comma from index renaming.
- Fix drop of indexes with renaming when there is a function call. Thanks to
Menelaos Perdikeas for the report.
- Fix empty geometry type since the move of ORA2PG_SDO_GTYPE into
lib/Ora2Pg/Oracle.pm
- Move most of the Oracle specific code to a dedicated Perl library
lib/Ora2Pg/Oracle.pm with the same functions as lib/Ora2Pg/MySQL.pm
This will help to maintain and extend Ora2Pg to other RDMS. There is
still Oracle database related specific code in the main library but
it will be also moved later. There should not be any regression or
usage change with this huge patch.
- Fix translation of type with not null clause. Thanks to Yasir1811 for
the report.
2021 11 15 - v23.0
This release fix several issues reported since past five months and
adds some new major features and improvements.
* Add new option --blob_to_lo that can be used to export BLOB as large
objects. It can only be used with action SHOW_COLUMN, TABLE and INSERT.
When used with TABLE action, the BLOB column will be translated into oid
PostgreSQL data type. When used with the INSERT export action BLOB data
will be store as large object in the pg_largeobjects table and the oid
referencing this large object will be stored in the main table instead
of a bytea.
It is not possible to use oid with COPY because this feature use function
lo_from_bytea() that stores the large object in the external table and
returns the oid.
This feature works with or without the use of oracle_fdw to import the
data and option -J can be used to improve the speed of the INSERT import
provide that there is a numeric unique key on the table.
Thanks to rodiq for the feature request.
* Add command line option -W | --where clause to set the WHERE clauses to
apply to the Oracle query to retrieve data. It can be used multiple time.
It will override the WHERE configuration directive if there is a global
WHERE clause or the same table WHERE clause definition. Otherwise the
clause will be appended.
* Add data validation feature consisting in comparing data retrieved from a
foreign table pointing to the source Oracle table and a local PostgreSQL
table resulting from the data export. By default Ora2Pg will extract 10000
rows from both side, you can change this value using DATA_VALIDATION_ROWS.
When it is set to zero all rows of the tables will be compared.
Data validation requires that the table has a primary key or unique index
and that the key columns is not a LOB.
Due to differences in sort behavior between Oracle and PostgreSQL, if the
collation of unique key columns in PostgreSQL is not 'C', the sort order of
is different compared to Oracle. In this case the data validation will fail.
Ora2Pg will stop comparing two tables after 10 errors, result is dumped to
an output file named data_validation.log.
* Add DATA_VALIDATION_ORDERING configuration directive enabled by default.
Order of rows between both sides are different once the data have been
modified. In this case data must be ordered using a primary key or a
unique index, that mean that a table without such object can not be
compared. If the validation is done just after data import in mode single
process and without any data modification the validation can be done on all
tables without any ordering.
* Add DATA_VALIDATION_ERROR to stop validating data from a table after a
certain amount of row mismatch. Default is to stop after 10 rows
validation errors.
* Allow multiprocess for TEST_DATA action to validate data import. Use -P
or PARALLEL_TABLES to set the number of parallel tables checked. Output
is now done to a file named data_validation.log saved in the current
directory.
* Add replacement of UTL_RAW.CAST_TO_RAW with encode().
* Add rewrite of XMLTYPE() with xmlparse(DOCUMENT convert_from(..., 'utf-8')).
* Add VARCHAR_TO_TEXT configuration directive. By default VARCHAR2 without
size constraint are tranlated into text PG data type. If you want to use
varchar instead, disable this directive.
* Add detection of XML function for migration assessment cost.
* Add DBMS_RANDOM to the list of Oraclism handled by Orafce.
* Add support to mysql_fdw foreign data wrapper to export data
PostgreSQL tables. Thanks to Yoni Sade for the feature request.
* Allow to transform all NUMBER(*,scale) to an other data type by a
redefinition like NUMBER(*\,2):decimal in the DATA_TYPE configuration
directive. Thanks to Florent Jardin for the patch.
* Add information on how to use SSL encrypted connection to documentation.
* Add TEST_COUNT action to just report the row count diff between Oracle and
Backward compatibility changes:
- Add FORCE_PLSQL_ENCODING configuration directive. By default Ora2Pg
encode all functions code to ut8, this sometime can generate double
encoding. To change this behavior, disable this configuration directive.
Thanks to rynerisraid and lee-jongbeom for the report.
- Change behavior regarding RAW columns. Now RAW(16) and RAW(32) columns or
RAW columns with "SYS_GUID()" as default value are now automatically
translated into uuid. Data will be automatically migrated as PostgreSQL
uuid data type provided by the "uuid-ossp" extension. To recover the old
behavior to export data as bytea whatever is the precision, the following
must be set with DATA_TYPE configuration: RAW(16):bytea,RAW(32):bytea
Complete list of changes:
- Fix USE_LOB_LOCATOR handling.
- Fix data validation using oracle_fdw where zero after decimal is not strip
unlike with PG.
- Apply MODIFY_STRUCT redefinition to test actions
- Fix PG filter when DATA_VALIDATION_ORDERING is disabled
- Apply RAW to uuid transformation for data validation
- Apply boolean transformation for data validation
- Do not export data for virtual column for PG >= 13.
- Fix wrong replacement function with name including a regexp_* function in
his name. Thanks to Rui Pereira for the report.
- Remove comments in the from clause before rewrite outer join (+), the entire
FROM clause will be rewritten and we don't know where to restore.
- Fix export of columns information for data verification.
- Fix TEST_VIEW for row count returned by views to exclude views created in
extensions.
- Fix comment on procedures
- Fix translation of MySQL type UNSIGNED
- Fix test count of indexes for MySQL database.
- Fix test MySQL sequence count.
- Do not display error messages when user and db is first checked in the
import_all.sh script
- Fix ordering of check constraints
- Fix mysql table scan when table name is using reserved word. Thanks to
Stanley Sung for the report.
- Fix double BOTH keyword in TRIM function. Thanks to Rui Pereira for the
report.
- Fix aliases placed in a wrong way. Thanks to Rui Pereira for the report.
- Fix parsing of procedure broken on keyword RETURN. Thanks to Pavithra
Jayasankar.
- Fix case where default partition is taken as a value. Thanks to Karsten
Lenz for the report.
- Fix conversion of NUMBER without precision in PL/SQL code to respect
settings PG_NUMERIC_TYPE, PG_INTEGER_TYPE and DEFAULT_NUMERIC. Fix
translation of INTEGER/BINARY_INTEGER that was wrongly exported as
numeric. Thanks to Philippe Beaudoin for the report.
- Documentation fix. Thanks to mperdikeas for the patch.
- Fix case where SQL%ROWCOUNT was not replaced by GET DIAGNOSTIC. Thanks to
Awdotia Romanowna for the report.
- Fix quote of unique constraints name. Thanks to Veka for the report.
- Fix looking at package function metadata when there is a huge amount of
package.
- Fix error when trying to remove temporary files.
- Fix wrong translation of a call to a procedure with PRAGMA AUTONOMOUS
TRANSACTION through dblink. Thanks to Rui Pereira for the report.
- Remove schema name in front of index name. Thanks to Menelaos Perdikeas
for the report.
- Fix virtual column generated from an other column of the table (supported
in PG 12). Thanks to Veka for the report.
- Fix case of columns names in boolean transformation when oracle_fdw is used
to export data. Thanks to veka for the report.
- Remove extra parenthesis with sub query and TABLE function. Thanks to Rui
Pereira for the report.
- Fix WHERE clause not removed in ROWNUM replacement. Thanks to Rui Pereira
for the report.
- Exclude extensions tables from table test count. Thanks to Yoni Sade for the
report.
- Fix pg_attribute column adsrc removed in PG 12. Thank to Thorsten Hochreuter
for the patch.
- Fix unwanted aliases after row_number() over(). Thanks to Rui Pereira for
the report.
- Fix several spelling issues. Thanks to Florian Eckert for the patch.
- Fix wrong condition to import constraints in import_all.sh. Thanks to
Thorsten Hochreuter for the report.
- Fix BITMAP_AS_GIN detection. Thanks to Nishanth Bejgam for the patch.
- Fix parsing of views from file and add PASSWORD, KEY and REF to the list
of reserved keywords.
- Fix replacement of CURSOR ... IS when there is comment after IS.
- Fix comment in auto generated file global_variables.conf
- Fix XML data export that was transformed by the call to function
extract(/).getClobVal(), it is now replaced by a direct call to
getClobVal().
- Improve COPY FREEZE data export when FILE_PER_TABLE is enabled, the
transactions are now managed per individual file and not following
the main file. Thanks to Yoni Sade for the report.
- Fix addition to UNLOGGED keyword on foreign table when exporting
data using oracle_fdw. Thanks to Veka for the report.
- Fix FK error when using TRUNCATE before data export with oracle_fdw.
- Fix export of user defined type. Actually type definitions are extracted
from ALL_SOURCE which contain the original CREATE TYPE and eventually
all the ALTER TYPE commands. Previously those type as considered as not
supported by Ora2Pg.
- replace date(n) by timestamp.
- Always remove the fqdn SYS schema before functions call.
- Add report of GTT in SHOW_TABLE action.
- Fix empty partition values for Oracle 9i.
- Add creation of schema in user defined type export when EXPORT_SCHEMA is
enabled to fix an error when the schema has not already been created.
- Fix some wordings and exclude from export user defined type starting
with SYS_PLSQL_ found in a 9i export. It looks that they are internal
to PL/SQL code.
- Exclude DBMS_SQL from the DBMS count in migration assessment when
USE_ORAFCE is enabled.
- Handle case where indexes name include the schema at create time
- Fix PL/SQL numeric datatype conversion
2021 07 02 - v22.1
This is a maintenance release to extend the feature of data export through the
oracle_fdw PostgreSQL extension to migration that use the public schema and
do not preserve case.
There is also some other fixes:
- Fix compile_schema() call that breaks valid function based indexes by
adding compile_all => FALSE to DBMS_UTILITY.compile_schema().
Thanks to Pawel Fengler for the patch.
- Force foreign table for data export as readonly to avoid accidental
write if import schema is not cleaned.
- Fix data export to file not possible since last changes for oracle_fdw
export. Thanks to Niels Jespersen for the report.
2021 06 26 - v22.0
This release fix several issues reported since past three months and
adds some new features and improvements. I must thanks MigOps Inc who
hire me to drive Oracle to PostgreSQL migrations and to develop Ora2Pg.
It's been a long time that I was looking for such a company and it is
an amazing gift for the 20 years of Ora2Pg. All improvements and new
new features developed during my work at MigOps will be available in
the public GitHub repository, here are the new ones.
- Add export of data using oracle_fdw when FDW_SERVER is set and export
type is COPY or INSERT. Multi-process using -P or -J is fully supported
but option -j is useless in this case. Boolean transformation of some
columns or data type is also supported. Actually, expect that it works
just like data migration without oracle_fdw. This can improve the data
migration speed from 30 to 40% especially for BLOB export.
- Improve export performances with huge number of objects by avoiding join
between Oracle catalog tables.
- Set a maximum of assessment score for tables, indexes, sequences,
partitions, global temporary table and synonym following the number of
objects.
- Add detection of XML functions to the assessment cost.
- Allow to change the assessment cost unit value in the export_all.sh script
when ora2pg is used with options --init_project and --cost_unit_value.
- Remove pragma restrict_references from P/PSQL code, it is useless.
- Add the oracle schema to search_path in SQL files generated and improve
the migration assessment when USE_ORAFCE is enabled.
- Apply ALLOW and EXCLUDED filtered stored procedures at package extraction
level. Previous this patch there was no way to not export some package
functions or to exclude them from assessment.
- Add new tests to check sequences last values and number of identity columns
in both side.
- Apply ALLOW/EXCLUDE without object to table object by default in TEST
action.
New configuration directives:
- Add ORACLE_FDW_TRANSFORM configuration directive to apply a transformation
to a column when exporting data. Value must be a semicolon separated list of
TABLE[COLUMN_NAME, <replace code in SELECT target list>]
For example to replace string 'Oracle' by 'PostgreSQL' in a varchar2 column
use the following.
ERROR_LOG[DBMS_TYPE:regexp_replace("DBMS_TYPE",'Oracle','PostgreSQL')]
Thanks to MigOps for the patch.
- Add DROP_IF_EXISTS configuration directive to add a statement
"DROP <OBJECT> IF EXISTS" before creating the object. Can be
useful in an iterative work. Default is disabled. Thanks to
dherzhau for the feature request.
Backward compatibility:
There is a backward compatibility issue with old configuration files
where FDW_SERVER is set by default. This directive was not used when
exporting data, this is not the case anymore as it instruct Ora2Pg to
use the given foreign server to use oracle_fdw to migrate the data.
Here is the full list of changes and acknowledgements:
- Fix replacement of TO_CLOB() function, now it is just removed and the
parenthesis are kept. Thanks to Rui Pereira for the report.
- Fix incorrect detection of cursor on dynamic query. Thanks to Rui
Pereira for the report.
- Fix quoting column names with spaces and dots. Dots are replaced by
underscore. Thanks to Veka for the report.
- Fix one case where DEFINED_PKEY with PRESERVE_CASE was not handled
correctly. Thanks to Veka for the report.
- Fix quoting of reserved keywords in CREATE INDEX columns names. Thanks
to Veka for the report.
- Fix column name starting with number not quoted in COMMENT. Thanks to
Veka for the report.
- Fix addition of PERFORM on call to stored procedures not prefixed by
the package name. Thanks to Rui Pereira fo the report.
- Fix search of ora2pg_conf.dist under Windows instead of ora2pg.conf.dist
when --init_project is used. Thanks to Julien Monticolo for the report.
- Fix translation from file of check constraint when created on same column,
only the last one was exported. Also shortened the prefix for constraint
naming, ora2pg_ckey becomes o2pc, ora2pg_ukey is now o2pu and ora2pg_fkey
is renamed into o2pf. Thanks to anvithaprabhu8 for the report.
- Replace wildcard precision * for numeric by 38.
- Fix incomplete listagg() conversion. Thanks to avandras for the report.
- Fix potential problem in last_day conversion when USE_ORAFCE is off and
a number is added or subtracted to the last day. Thanks to atlterry for
the report.
- Do not apply utf8 conversion of comments to input files.
- Fix termination of last writer process when parallel and quiet mode are
used together. Thanks to David Harper for the patch.
- Remove precision in number of digit in timestamp microseconds when
setting NLS_TIMESTAMP_FORMAT at session startup. Now use:
ALTER SESSION SET NLS_TIMESTAMP_FORMAT='YYYY-MM-DD HH24:MI:SS.FF'
instead of '.FF6'. This was possibly the cause of sporadic error:
ORA-03106: fatal two-task communication protocol error (DBD ERROR: OCIStmtFetch)
Default is precision 6 so it may not change anything. Thanks to
Abhijeet Bonde for the report.
- Fix call to deprecated MySQL spatial function when version is after 5.7.6.
Thanks to naveenjul29 for the report.
- Fix false positive detection for nested table with MySQL export.
- Fix sequence export read from file.
- Add PG_VERSION to the documentation. Thanks to xinjirufen for the report.
- Fix error report when preparing query to ALL_IND_COLUMNS IC. Thanks to
ganeshakorde for the report.
- ora2pg_scanner: Fix detection of service_name in DSN.
- Fix error raise_application_error() with named parameters.
- Fix replacement of out parameters in triggers.
- Change sequence export result storage to hash instead of array.
- Fix package export when there is a comment between PACKAGE BODY and the
name of the package.
- Fix Perl error "malformed utf-8 character in substitution" when there is
character not in utf8 in the comment or constants.
- Fix detection of MySQL FUNCTION vs PROCEDURE for version >= 5.5. Thanks
to naveenjul29 for the report.
- Exclude nested tables from the export as it is not supported and it
always generate an error. A warning is raised.
- Fix column case in check constraints when PRESERVE_CASE is enabled.
- Fix search_path in direct PG data export when PG_SCHEMA is set.
- Fix -Infinity insert for direct PG data export.
- Fix drop indexes when PRESERVE_CASE is enabled.
- Remove potential double affectation for function with out parameter.
- Create immutable to_char function when used in an index.
- Replace dmake by gmake on Windows installation instruction. Thanks to
Julien Monticolo for the report.
- Fix MySQL version conditions. Thanks to Christoph Berg for the report.
- Fix HASH partitioning for duplicate WITH clause.
- Fix tests when no schema are set to compare all objects in all schemes.
Thanks to gp4git and dlc75 for the report.
- Apply ALLOW/EXCLUDE without object to table object by default in TEST
action. Thanks to Yony Sade for the feature request.
- Add DROP_IF_EXISTS configuration directive. Thanks to dherzhau for the
feature request.
- Fix regression in removing %ROWTYPE from function parameters. Thanks to
Eric Delanoe for the report.
- Fix not adding default values to parameters when this is an OUT parameter.
Thanks to Eric Delanoe for the report.
- Fix ALL_DIRECTORIES call be using table name relative to USER_GRANTS.
Thanks to Yoni Sade for the report.
- Change all remaining call to static ALL_* tables to a call relative to
USER_GRANTS. Thanks to Yoni Sade for the report.
- Fix export or partitioned table with unsupported partitioning type
like PARTITION BY REFERENCE. The table is created without partition
and a warning it fired as well as the following message in the output
file as a comment: -- Unsupported partition type, please check
2021 04 01 - v21.1
This release fix several issues reported since past six months and
as usual adds some new features and improvements.
* Now that Orafce 3.15.0 has a definition for the REGEXP_* function,
makes the translation optional to USE_ORAFCE directive.
* Add set application name in connection to Oracle/MySql/PostgreSQL.
* Add translation of REGEXP_COUNT() and change assessment cost.
* Rewrite the way REGEXP_LIKE() is translated into regexp_match to
support modifiers. This rewrite also fix default behavior between
Oracle and PostgreSQL.
* Replace DBMS_LOB.GETLENGTH() by PostgreSQL octet_length() function.
* Add types correspondences for VARCHAR2 and NVARCHAR2 in DATA_TYPE
configuration directive.
* Add autodetection and support of geometry type, srid and dimension
for ArcGis geometries.
* Add conversion of default value in function parameters.
* Add -u | --unit option to ora2pg_scanner to be able to set the
migration cost unit value globally.
* Replace DBMS_LOB.SUBSTR() by SUBSTR()
* Remove TO_CLOB() it is useless, manual cast could be necessary.
* Replace IS JSON validation clause in CHECK constraints by
(CASE WHEN $1::json IS NULL THEN true ELSE true END)
When the code is invalid an error is fired.
* DISTINCT and UNIQUE are synonym on Oracle
Backward compatibility changes:
- Force a column to be bigint if this is an identity column. Thanks
to MigOps.com for the patch.
- Fix EMPTY_LOB_NULL, enable/disable was inverted, keeping default
to enabled. Take care that in old ora2pg.conf it is disabled so it
will break backward compatibility with old configuration.
- Replace NO_LOB_LOCATOR with USE_LOB_LOCATOR and NO_BLOB_EXPORT
with ENABLE_BLOB_EXPORT to avoid confusion with double negative
variable. Backward compatibility is preserved with a warning.
- SRID for SDO_GEOMETRY export is now taken from the value not forced
from the metadata table.
Here is the full list of changes and acknowledgements:
- Take Geometry SRID from the data and fallback to SRID defined in
metadata when not found. Thanks to Sebastian Albert for the report.
- Fix case where Ora2Pg temporary substitution of '' by placeholder
was not restored. Thanks to MigOps.com for the patch.
- Fix identity column export on unsupported Oracle 18c options.
Thanks to MigOps.com for the patch.
- Fix export of columns indexes created with single quote.
Thanks to MigOps.com for the patch.
- Fix replacement of keyword PROCEDURE by FUNCTION in constraints
constants definition. Thanks to marie-joechahine for the report.
- Replace IS JSON validation clause in CHECK constraints. Thanks to
marie-joechahine for the report and MigOps.com for the patch.
- Add support to ON OVERFLOW clause in LISTAGG replacement.
Thanks to MigOps.com for the patch.
- Fix incorrect handling of HAVING+GROUP BY rewriting.
Thanks to MigOps.com for the patch.
- Add replacement of TO_NCHAR by a cast to varchar. Thanks to
MigOps.com for the patch.
- Fix replacement of NOTFOUND when there is extra space or new line
in the WHEN EXIT clause. Thanks to MigOps.com for the patch.
- Fix a regression in NO_VIEW_ORDERING, it was not taken in account
anymore. Thanks to RonJojn2 for the report.
- Replace DATA_TYPE with DTD_IDENTIFIER in MySQL catalog queries for
version prior 5.5.0. Thanks to zejeanmi for the report.
- Fix import script to import sequences before tables. Thanks to
MigOps.com for the patch.
- Fix detail report of custom type in migration assessment. Thanks
to MigOps.com for the patch.
- Fix duplicate schema prefixed to SYNONYM. Thanks to dlc75 for the
reports.
- Replace NO_LOB_LOCATOR with USE_LOB_LOCATOR and NO_BLOB_EXPORT with
ENABLE_BLOB_EXPORT to avoid confusion with double negative variable.
Thanks to Rob Johnson for the report.
- Fix some missing replacements of NVL and rewrite !=-1 into != -1.
Thanks to MigOps.com for the patch.
- Fix ROWNUM followed by + or - operator and when no aliases are
provided. Thanks to MigOps.com for the patch.
- Add DBSFWUSER to the list of user/schema exclusion. Thanks to
MigOps.com for the patch.
- Fix regexp to not append subquery aliases on JOIN clause. Thanks
to Rui Pereira for the report.
- Handle PRESERVE_CASE and EXPORT_SCHEMA in sequence name. Thanks
to marie-joechahine for the report.
- Add CREATE SCHEMA statement to sequence export when EXPORT_SCHEMA
is enabled. Thanks to marie-joechahine for the report.
- Fix duplicate index name on subpartition. Thanks to Philippe
Beaudoin for the report.
- Exclude sequences used for IDENTITY column (ISEQ$$_). Thanks to
marie-joechahine for the report.
- Fix parsing from file of CREATE SEQUENCE. Thanks to Rui Pereira
for the report.
- In export_all.sh script use the database owner provided if it is a
superuser instead of postgres user. Thanks to jjune235 for the
feature request.
- Fix parsing of triggers when there is a CASE inside the code.
Thanks to Rui Pereira for the report.
- Add set application name in connection to Oracle/MySql/PostgreSQL.
Thanks to Yoni Sade for the patch.
- Fix double column alias when replacing ROWNUM. Thanks to Rui
Pereira for the report.
- Add translation of the REGEXP_COUNT function and change assessment
cost.
- Rewrite the way REGEXP_LIKE is translated into regexp_match to
support modifiers. This rewrite also fix default behavior between
Oracle and PostgreSQL. Thanks to otterrisk for the report.
- Add IS JSON to assessment. Thanks to marie-joe Chahine for the
report.
- Fix multi-columns RANGE partitioning. Thanks to Philippe Beaudoin
for the report.
- Improve reordering columns. Sort by fieldsize first, if same size
then it sorts by original position. Thanks to Sebastien Caunes for
the patch.
- Append partition's column to the primary key of the table as it
must be part of the PK on PostgreSQL. Thanks to xinjirufen for the
report.
- Fix partition export where PRESERVE_CASE was applied to Oracle
side. Thanks to schleb1309 for the report.
- Fix trigger export with column restriction. Thanks to Sebastien
Caunes for the report.
- Update installation information.
- Fix table reordering following data type. Thanks to Sebastien
Caunes for the patch.
- Fix incorrect variable name corresponding to DATA_EXPORT_ORDER
making this directive inefficient. Thanks to Ron Johnson for the
report.
- Fix translation of check constraint when read from file
- Fix EMPTY_LOB_NULL, enable/disable as inverted, keep default to
enabled. Take care that in old ora2pg.conf it is disabled so it
will break backward compatibility with old configuration.
- Fix false positive detection of input filename is the same as
output file.
- Rename variables SCHEMA_ONLY, DATA_ONLY and CONSTRAINTS_ONLY in
script import_all.sh to conform to their real use. Thanks to
Sebastien Caunes for the report.
- Fix comment detection breaking the package header parsing and
global variable detection.
- Fix ROWNUM detection for replacement by LIMIT
- Fix escaping of psql command in configuration file comment and
set default value for PG_VERSION to 12.
- Replace precision by exactness in documentation. Thanks to
Sebastien Caunes for the report.
- Prevent reducing DATA_LIMIT when NO_BLOB_EXPORT is enabled.
Thanks to Thomas Reiss for the report.
- Fix geometry type detection.
- Add autodetection of geometry type, srid and dimension for
ArcGis geometries. Thanks to changmao01 for the feature request.
- Fix call to ST_GeomFromText when no SRID is found.
- Fix case where OVERRIDE SYSTEM VALUE clause could be added if PG
version is < 10. Thanks to changmao01 for the report.
- Fix unwanted call to internal GEOM library for ArcGis geometries.
Thanks to changmao01 for the report.
- Exclude schema SDE (ArGis) from export. Thanks to changmao01 for
the report.
- prevent looking twice to same custom data type definition.
- Fix previous patch to catch SDO_GEOMETRY on lowercase regexp.
- Limit detection of geometry data type to SDO_GEOMETRY.
- Fix column name replacement in view definition. Thanks to Amit
Sanghvi for the report.
- Fix REPLACE_COLS parsing to allow space in column name. Thanks
to Amit Sanghvi for the report.
- Fix translation from file of triggers with WHEN clause. Thanks
to Rui Pereira for the report.
- Fix column name kept lowercase in the MOD() clause when -J is
used. Thanks to Code-UV for the report.
- Keep case of PG_SCHEMA definition when used in TEST action.
- Fix data export for columns with custom data type. Thanks to
Aymen Zaiter for the report.
- Fix missing bracket with || operator in CREATE INDEX. Thanks to
traxverlis for the report.
- Fix export of single row unique function base index. Example:
CREATE UNIQUE INDEX single_row_idx ON single_row ((1));
Thanks to unrandom123 for the report.
- Update documentation about schemas used in TEST action.
- Disable materialized view export with MySQL export it is not
supported. Thanks to naveenjul29 for the report.
- Fix table alias detection in Oracle (+) join rewrite.
- Fix an infinite loop in Oracle (+) join rewrite when there is no
table aliases and the table is prefixed by its schema. Thanks to
Olivier Picavet for the report.
- Fix MODIFY_STRUCT when column name need to be escaped. Thanks to
helmichamsi10 for the report.
- Fix empty PARTITION BY () clause. Thanks to Aymen Zaiter.
- Fix export of global variable from package description when there
is no package body. Thanks to naveenjul29 for the report.
- Add package description export when dumping package source,
previously only the package body was dump. This will allow to
check global variables export.
- Whilst working on the Reproducible Builds effort (https//reproducible-builds.org/)
it appears that ora2pg could not be built reproducibly. Thanks to
Chris Lamb for the patch.
- Fix case of NUMBER(*,10) declaration. Oracle has a precision of 1 to 38
for numeric. Even if PostgreSQL allow a precision of 1000 use 38 to
replace junk parameter. Thanks to xinjirufen for the report.
- Add conversion of default value in function parameters, like syssdate
rewriting for example. Thanks to unrandom123 for the report.
- Fix a regression in data encoding when exporting data introduced in
commit fa8e9de. Thanks to gp4git for the report.
- Add debug information about the environment variables used before
connecting to Oracle.
- Fix case of duplicate between unique index and unique constraint with
multiple columns. Thanks to gp4git.
2020 10 12 - v21.0
This release fix several issues reported since last release and adds
several new features and improvements.
* Add clause OVERRIDING SYSTEM VALUE to INSERT statements when the
table has an IDENTITY column.
* Considerably increase the speed to generate the report about the
migration assessment, especially for database with huge number of
objects.
* Reduce time passed in the progress bar. Following the number of
database objects we were spending too much time in refreshing the
progress bar.
* Add number of identity columns in migration assessment report.
* Make assessment details report initially hidden using HTML5 tags
<details><summary>
* Improve speed of BLOB/CLOB data export. Oracle recommends reading
from and writing to a LOB in batches using a multiple of the LOB
chunk size. This chunk size defaults to 8k (8192). Recent tests
show that the best performances can be reach with higher value
like 512K or 4Mb.
* Add progress bar when --oracle_speed is use in single process mode.
* Automatically activate USER_GRANTS when the connection user has no DBA
privilege. A warning is displayed.
* Complete port to Windows by using the Windows separator on stdout
redirection into a file at ora2pg command line call and improve
ora2pg_scanner port on Windows OS.
* Add rewrite of MySQL JOIN with WHERE clause instead of ON.
* Add MGDSYS (Oracle E-Business Suite) and APEX_040000 to the list
of schemas excluded from the export.
* Supply credentials interactively when a password is not defined in
the configuration file. Need the installation of a new Perl module
Term::ReadKey.
* Add supports oracle connections "as sysdba" with username "/" and
an empty password to connect to a local oracle instance.
* Add translation of PRIVATE TEMPORARY TABLE from Oracle 18c into
PostgreSQL basic temporary table, only the default behavior for
on commit change.
New command line options:
* Add new command line option to ora2pg_scanner: -b | --binpath DIR
to set the full path to directory where the ora2pg binary stays.
Might be useful only on Windows OS.
* Add -r | --relative command line option and PSQL_RELATIVE_PATH
configuration directive. By default Ora2Pg use \i psql command to
execute generated SQL files if you want to use a relative path
following the script execution file enabling this option will use
\ir. See psql help for more information.
New configuration directives:
* NO_VIEW_ORDERING:
By default Ora2Pg try to order views to avoid error at import time
with nested views. With a huge number of views this can take a very
long time, you can bypass this ordering by enabling this directive.
* NO_FUNCTION_METADATA
Force Ora2Pg to not look for function declaration. Note that this
will prevent Ora2Pg to rewrite function replacement call if needed.
Do not enable it unless looking forward at function breaks other
export.
* LOB_CHUNK_SIZE
See explanation in the new features and improvement list.
* ALTERNATIVE_QUOTING_REGEXP
To support the Alternative Quoting Mechanism ('Q' or 'q') for String
Literals set the regexp with the text capture to use to extract the
text part. For example with a variable declared as
c_sample VARCHAR2(100 CHAR) := q'{This doesn't work.}';
the regexp to use must be:
ALTERNATIVE_QUOTING_REGEXP q'{(.*)}'
ora2pg will use the $$ delimiter, with the example the result will
be:
c_sample varchar(100) := $$This doesn't work.$$;
The value of this configuration directive can be a list of regexp
separated by a semi colon. The capture part (between parenthesis) is
mandatory in each regexp if you want to restore the string constant.
Backward compatibility changes:
- Default for NO_LOB_LOCATOR is now 1 to benefit from the LOB_CHUNK_SIZE
performances gain.
- Enable schema compilation (COMPILE_SCHEMA set to 1) by default to
speed up DDL extraction.
- Change the behavior of Ora2Pg with the parameters that follows a
parameter with a default value. Ora2Pg used to change the order of the
parameter's function to put all parameters with a default value at end
of the list which need a function call rewrite. This have been abandoned
now any parameter without default value after a parameter with a default
value will be appended DEFAULT NULL.
Here is the full list of changes and acknowledgements:
- Fix unwanted references to PK/UK when DROP_INDEXES is enabled.
- Fix comparison between function name in TEST report.
- Fix duplicates on retrieving partitions information.
- Improve SHOW_TABLE report about partitioned tables information.
- Drop code about removing DEFAULT NULL in functions parameters. Thanks to
chaluvadi286 for the report.
- Fix two other case where materialized view can be listed in the table list.
- Fix case where materialized view can be listed in the table list. Thanks
to Thomas Reiss for the report.
- Fix %ROWTYPE removing to be restricted to REF CURSOR. Thanks to
jagmohankaintura-tl for the report.
- Fix PG functions count when comparing Oracle functions count in TEST action.
Remove useless -l option to import_all.sh auto generated script.
- Fix PRESERVE_CASE on schema name for functions extracted from a package.
- Fix search_path adding public default schema.
- Apply PRESERVE_CASE to partition by involved columns.
- Add IF EXIXTS to create schema to avoid error when import_all.sh is run
several time.
- Fix sort order of comment on columns for tables and views.
- Fix warning about data export from nonexistent table resulting of index
lookup on nested table.
- Fix infinite loop in global variables package extraction. Thanks to Thomas
Reiss for the report.
- Fix global variables and packages export when comments are present in the
package description.
- Add information about XML_PRETTY size limit to 4000
- Fix column name in indexes when PRESERVE_CASE is enabled. Thanks
to Julien traxverlis for the report.
- Fix Top 10 of largest tables sort order. Thanks to Tom Vanzieleghem
for the patch.
- Fix duplicates between indexes and constraints. Thanks to sdpdb and
Jon Betts for the report.
- Fix SYSDATE replacement and possible infinite loop in SYSDATE parsing.
Thanks to pbidault for the report.
- Fix export of Oracle TEXT indexes with USE_UNACCENT disabled. Thanks to
Eric Delanoe for the report.
- Add new configuration directive ALTERNATIVE_QUOTING_REGEXP to support
the Alternative Quoting Mechanism ('Q' or 'q') for String Literals.
Thanks to just-doit for the report.
- Fix OF clause missing in update triggers. Thanks to just-doit for
the report.
- Fix IS NULL translation in WHERE clause of UPDATE statement. Thanks
to Eric Delanoe for the report.
- Remove DDL export of LOG indexes on materialized views.
- Fix unexpected materialized view listed in table export. Thanks to
jagmohankaintura-tl for the report.
- Fix default values with single quote in create table DDL. Thanks to
justdoit for the report.
- Fix double quote in CREATE TRIGGER code and applying of preserve case
on column name.
- Supply credentials interactively when a password is not defined in
configuration file. Thanks to rpeiremans for the patch.
- Add supports oracle connections "as sysdba" with username "/" and
an empty password to connect to a local oracle instance. Thanks to
rpeiremans for the patch.
- Fix documentation about materialized view export.
- Fix export order of comments on columns.
- Fix export of views comments when no schema is used for export and
export schema is activated.
- Fix cast in replacement with TO_NUMBER and TO_CHAR in indexes. Thanks
to Kiran for the report.
- Add MGDSYS (Oracle E-Business Suite) to the list of schemas excluded
from the export. Thanks to naveenjul29 for the report.
- Add more information about PG_DSN use. Thanks to Pepan7 for the report.
- Update copyright year.
- Fix regression where "SET client_encoding TO ..." was missing data file
header. Thanks to Emmanuel Gaultier for the report.
- Fix EDITABLE vs EDITIONABLE parsing. Thanks to Naveen Kumar for the report.
- Fix typos in documentation. Thanks to swallow-life, ChrisYuan, Edward Betts,
Jack Caperon and cavpollo for the patches.
- Add OVERRIDING SYSTEM VALUE to INSERT statement when the table has an
IDENTITY column. Thanks to Robin Windey for the report
- Remove empty parenthesis of identity column options
- Limit sequence/identity column value to bigint max
- Add an example of DBD::Oracle DSN with 18c.
- Fix parsing of identity column from file. Thanks to deepakp555 for the
report.
- Fix quoting of identifier when PRESERVE_CASE is enable and no particular
schema is specified. Thanks to mkgrgis for the report.
- Move setting of search_path before truncate table. Thanks to Michael Vitale
for the report.
- Add explanation about TEST and SIZE migration assessment values.
- Mark XMLTYPE as having LOB locator.
- Fix XMLTYPE columns that are exported as lob locator. Thanks to Tamas for
the report.
- Fix a problem of data export throughput that was slowing down all along
the export when multiprocess for output was not used. Ora2Pg was forking
a process for each chunk of data (see DATA_LIMIT) which is useless when
write output is done on a single process (-j 1) and slow down the export.
Thanks to markhooper99 and Tamas for reporting, testing and finding the
source of the issue.
- Fix progress bar in multiprocess mode, update was not displayed at each
chunk of data processed.
- Add internal debug information for progress bar.
- Add debug information for SHOW_REPORT
- Fix a long pending issue with custom data type export. Thanks to
jhollandsworth for the patch.
- Fix LOB data export with value changed to NULL when the CLOB value was 0.
Thanks to jhollandsworth for the report.
- Fix escape format issue with COPY and bytea. Thanks to Christoph Noel and
dwbrock62 for the report.
- Add LD_LIBRARY_PATH and PATH prerequisite to run ora2pg.
- Fix use of the HIGH_VALUE column in partition listing with Oracle 9i. Thanks
to Francisco Puga for the report.
- Update the table row count logic to incorporate the PostgreSQL table FQN as
established through the set_pg_relation_name routine. Thanks to Jacob
Roberts for the patch.
- Add the PostgreSQL FQN when printing the results in the TEST function. Thanks
to Jacob Roberts for the patch.
- Do not look forward function with the SHOW_* action
- Fix BLOB export where \x was escaped. Thanks to Christophe Noel for the
report.
- Update Ora2Pg.pm to fix symbol in column name in create index statement.
Thanks to kpoluektov for the patch.
- Fix package function extraction when there is a start of comment (/*) in
a constant string. Thanks to Tiago Anastacio for the report.
- Fix type detection in package declaration. Thanks to Tiago Anastacio for
the report.
- Avoid displaying error ORA-22831 when exporting LOB. This error can
appears when LOB chunk size is different from default 8192. The error
has no incidence on the export so we can just ignore it. This patch
also use DBD::Oracle ora_lob_chunk_size() method to gather chunk the
chunk size of the LOB, fallback to 8192 if not available. Thanks to
joedbadmin for the report.
- Disable direct report of Oracle errors, all error should be handled at
Ora2Pg level.
- Fix MySQL data export with allow/exclude objects. Thanks to Manuel Pavy for
the report.
- Fix exclude/allow object feature in MySQL export that was not working since
release 19.0. Thanks to Manuel Pavy for the report.
- Add rewrite of MySQL JOIN with WHERE clause instead of ON. Thanks to Marc
Rechte for the report.
- Fix issue with custom type when multiprocess is used.
- Fix progress bar on final total estimated data in multiprocess mode.
- Fix ORACLE_HOME path in README.md. Thanks to Lubos Cisar for the patch.
- Fix missing replacement with PERFORM in CASE ... WHEN statements. Thanks to
Eric Delanoe for the report.
- Fix duplicate ora2pg command in iteration.
- Improve ora2pg_scanner port on Windows OS. Thanks to Marie Contencin for the
report.
- Add perl call to all ora2pg commands when the scanner is executed on
Windows system as the shebang is not recognized. Thanks to Marie Contencin
for the report.
- Fix several issue with compressed output. Thanks to Bach Nga for the report.
- Fix translation of CURSOR IS SELECT with a comment before the SELECT.
Thanks to Izaak van Niekerk for the report.
- Fix export of procedures as PostgreSQL procedures with version 11.
- Add APEX_040000 to the schemas exclusion list. Thanks to Don Seiler for the
report.
- Fix possible unquoted default values. Thanks to Marc Rechte for the report.
- Fix MySQL SET TRANSACTION clause when TRANSACTION is set to readonly or
readwrite this is not supported so fall back in READ COMMITTED isolation
level in this case. Thanks to Marc Rechte for the report.
- Fix export of functions, column DATA_TYPE does not exists in table
INFORMATION_SCHEMA.ROUTINES before MySQL 5.5.0. Replace it with column
DTD_IDENTIFIER for prior version. Thanks to Marc Rechte for the report.
- Fix double quote in CREATE TRIGGER code and applying of preserve case on
column name.
2019 01 18 - v20.0
This release fix several issues reported during the last three months
and adds several new features and improvement. The change of major
version is related to backward compatibility break with the removed of
most PG_SUPPORTS_* configuration directives and their replacement with
the new PG_VERSION directive.
New features and configuration directives in this release:
* Add PG_VERSION configuration directive to set the PostgreSQL major
version number of the target database. Ex: 9.6 or 10. Default is
current major version at time of a new release. This replace the
old PG_SUPPORTS_* configuration directives.
* Removed all PG_SUPPORTS_* configuration directives minus
PG_SUPPORTS_SUBSTR that is related to Redshift engine.
* Export of BFILE as bytea is now done through a PL/SQL function to
extract the content of a BFILE and generate a bytea data suitable
for insert or copy into PostgreSQL.
* Foreign keys that reference a partitioned table are no more
exported.
* Show table name on Oracle side during export using at connection
time: DBMS_APPLICATION_INFO.SET_ACTION(table_name);
* When the date format is ISO and the value is a constant the call
to to_date() is removed and only the constant is preserved. For
example: to_date(' 2013-04-01 00:00:00','SYYYY-MM-DD HH24:MI:SS')
is replaced by a simple call to: ' 2013-04-01 00:00:00'.
This rewrite is limited to PARTITION export type when directive
PG_SUPPORTS_PARTITION is enabled.
* Add DATA_EXPORT_ORDER configuration directive. By default data
export order will be done by sorting on table name. If you have
huge tables at end of alphabetic order and are using multiprocess,
it can be better to set the sort order on size so that multiple
small tables can be processed before the largest tables finish.
In this case set this directive to size. Possible values are name
and size. Note that export type SHOW_TABLE and SHOW_COLUMN will
use this sort order too, not only COPY or INSERT export type.
* Add NO_BLOB_EXPORT configuration directive. Exporting BLOB could
take time and you may want to export all data except the BLOB
columns. In this case enable this directive and the BLOB columns
will not be included into data export. The BLOB column must not
have a NOT NULL constraint. Thanks to Ilya Vladimirovich for the
* Add PREFIX_SUB_PARTITION to enable/disable sub-partitioning table
prefixing in case of the partition names are a part of the sub-
partition names.
* Add special replacement for case of epoch syntax in Oracle:
(sysdate - to_date('01-Jan-1970', 'dd-Mon-yyyy'))*24*60*60
is replaced by the PostgreSQL equivalent:
(extract(epoch from now()))
Here is the full list of changes and acknowledgements:
- Export indexes and constraints on partitioned table with pg >= 11.
- Fix incorrect replacement of NLS_SORT in indexes.
- Bring back DISABLE_UNLOGGED feature. Thanks to Jean-Christophe
Arnu for the patch
- Fix CREATE SCHEMA statement that was not written to dump file.
- Fix DBMS_APPLICATION_INFO.set_action() call, old Oracle version
do not support named parameters.
- Fix duplicate index name on partition. Thanks to buragaddapavan
for the report.
- Add support to new configuration directive PG_VERSION to control
the behavior of Ora2Pg following PostgreSQL version.
- Fix error in creation of default partition with PostgreSQL 10.
Thanks to buragaddapavan for the report.
- Fix missing export of single MAXVALUE partition, this will produce
the following range partition: ... FOR VALUES FROM (MINVALUE) TO
(MAXVALUE) Previous behavior was to not export partition as it is
better to not partition the table at all. However it is declared
in Oracle so it is better to export it to see what can be done.
Thanks to buragaddapavan for the report.
- Do not export foreign keys that reference a partitioned table.
Remove NOT VALID on foreign keys defined on a partitioned
table if present. Thanks to Denis Oleynikov for the report.
- Fix export of BFILE as bytea. Ora2Pg now use a PL/SQL function to
extract the content of a BFILE and generate a bytea data suitable
for insert or copy into PostgreSQL. Thanks to RickyTR for the
report.
- Add TIMEZONE_REGION and TIMEZONE_ABBR to migration assessment, no
direct equivalent in PostgreSQL. Remove NSLSORT not used in
migration assessment. Thanks to buragaddapavan for the report.
- Fix output of multiple export type specified in TYPE directive.
- Rewrite and renaming of _get_sql_data() function into
_get_sql_statements().
- Limit CURSOR weight in migration assessment to REF CURSOR only,
other case are all covered. REF CURSOR might need a review to see
if they need to be replaced with a SET OF RECORD.
- Fix replacement of EMPTY_CLOB() or EMPTY_BLOB() with empty string
when EMPTY_LOB_NULL is disabled and NULL when it is enabled.
- Prefix output file with the export type in multiple export type
mode, ex: sequence_output.sql or table_output.sql. Thanks to
buragaddapavan for the report.
- Fix export of data from an Oracle nested table. Thanks to rejo
oommen for the report.
- Removed cast to timestamp from partition range. Thanks to
buragaddapavan and rejo-oommen for the report.
- Fix partition default syntax. Thanks to rejo-oommen for the
report.
- Apply missing SYSUSERS schemas exclusion on columns and partition
listing. Thanks to rejo-oommen for the report.
- Add warning about parameter order change in output file.
- Show table name on Oracle side during export using at connection
time: DBMS_APPLICATION_INFO.SET_ACTION(table_name);
Thanks to Denis Oleynikov for the feature request.
- Report change in ORA_RESERVED_WORDS into documentation.
- Add references in the keyword list of ORA_RESERVED_WORDS.
- Fix the missing white space in some lines while creating
import_all.sh file. Thanks to Fabiano for the patch.
- Fix translation of infinity value for float. Thanks to Damien
Trecu for the report.
- Fix default value in timestamp column definition when a timezone
is given. Thanks to buragaddapavan for the report.
- Fix missing export of index and constraint in a partitioned
table when DISABLE_PARTITION is enabled. Thanks to Denis Oleynikov
for the report.
- Prevent PARTITION BY when DISABLE_PARTITION is enabled. Thanks to
Denis Oleynikov for the report.
- Add DATA_EXPORT_ORDER configuration directive. By default data
export order will be done by sorting on table name. If you have
huge tables at end of alphabetic order and are using multiprocess,
it can be better to set the sort order on size so that multiple
small tables can be processed before the largest tables finish.
In this case set this directive to size. Possible values are name
and size. Note that export type SHOW_TABLE and SHOW_COLUMN will
use this sort order too, not only COPY or INSERT export type.
Thanks to Guy Browne for the feature request.
- Fix remove leading ':' on Oracle variable taking care of regex
character class. Thanks to jselbach for the report.
- Add NO_BLOB_EXPORT configuration directive. Exporting BLOB could
take time and you may want to export all data except the BLOB
columns. In this case enable this directive and the BLOB columns
will not be included into data export. The BLOB column must not
have a NOT NULL constraint. Thanks to Ilya Vladimirovich for the
feature request.
- Fix incorrect rewrote of the first custom type in a row. Thanks
to Francesco Loreti for the patch.
- Remove double quote in type definition en set type name in lower
case when PRESERVE_CASE is disabled.
- Add PREFIX_SUB_PARTITION to enable/disable sub-partitioning table
prefixing in case of the partition names are a part of the sub-
partition names.
- Fix epoch replacement case in CREATE TABLE statements.
- Apply epoch replacement to default value in table declaration.
- Add special replacement for case of epoch syntax in Oracle:
(sysdate - to_date('01-Jan-1970', 'dd-Mon-yyyy'))*24*60*60
is replaced by the PostgreSQL equivalent:
(extract(epoch from now()))
Thanks to rejo-oommen for the feature request.
- A few typos in --help sections. Thanks to Christophe Courtois
for the report.
- Fix export of primary key on partition table. Thanks to chmanu
for the patch.
- Fix malformed user defined type export. Thanks to Francesco Loreti
for the report.
2018 09 27 - v19.1
This release fix several issues reported during the last month and
add support to PostgreSQL 11 HASH partitioning.
It also adds some new features and configuration directives:
* Add export of default partition and default sub partition.
* Add export of HASH partition type.
* Add support of stored procedure object.
* Add replacement of NLSORT in indexes or queries. For example:
CREATE INDEX test_idx ON emp
(NLSSORT(emp_name, 'NLS_SORT=GERMAN'));
is translated into
CREATE INDEX test_idx ON emp
((emp_name collate "german"));
The collation still need to be adapted, here probably "de_DE".
NLSSORT() in ORDER BY clause are also translated.
* Prevent duplicate index with primary key on partition to be
exported.
* PostgreSQL native partitioning does not allow direct import of
data into already attached partitions. We now force direct import
into main table but we keep Oracle export of data from individual
This release also adds two new command line options:
--oracle_speed: use to know at which speed Oracle is able to send
data. No data will be processed or written
--ora2pg_speed: use to know at which speed Ora2Pg is able to send
transformed data. Nothing will be written
Use it for debugging purpose. They are useful to see Oracle speed to
send data and at what speed Ora2Pg is processing the data without
reaching disk or direct import into PostgreSQL.
Two new configuration directive has been added:
* PG_SUPPORTS_PROCEDURE : PostgreSQL v11 adds support to stored
procedure objects. Disabled by default.
- PARALLEL_MIN_ROWS: set the minimum number of tuples in a table
before calling Oracle's parallel mode during data export.
Default to 100000 rows.
Note that PG_SUPPORTS_PARTITION and PG_SUPPORTS_IDENTITY are now
enabled by default to use PostgreSQL declarative partionning and
identity column instead of serial data type.
Here is the full list of changes and acknowledgements:
- Fix automatic quoting of table or partition name starting with
a number. Thanks to Barzaqh for the report.
- Add information about custom directory installation. Thanks to
joguess for the report.
- Update list of action in documentation.
- Fix export of spatial geometries. Thanks to burak yurdakul for
the report.
- Fix translation of default value in CREATE TABLE DDL when using
a function. Thanks to Denis Oleynikov for the report.
- Prevent moving index on partition during tablespace export.
Thanks to Maxim Zakharov for the report.
- Fix upper case of partition name in triggers.
- Enforce KEEP_PKEY_NAMES when USE_TABLESPACE is enabled. Thanks
to Maxim Zakharov for the patch.
- Fix parsing of Oracle user login in dblink input from a file.
- Fix multiple duplication of range clause in partition export.
- Add bench of total time and rows to migrate data from Oracle
in debug mode with speed average.
- Fix sub partition prefix name.
- Fix unset oracle username when exporting DBLINK from database.
Thanks to Denis Oleynikov for the report.
- Remove NO VALID to foreign keys on partitioned table. Thanks to
Denis Oleynikov for the report.
- Fix crash of Ora2Pg on regexp with dynamic pattern base on package
code. Thank to Alain Debie and MikeCaliffCBORD for the report.
- PostgreSQL native partitioning does not allow direct import of
data into already attached partitions. When PG_SUPPORTS_PARTITION
is enable we now force direct import into main single table but
we keep Oracle export of data from individual partition. Previous
behavior was to use main table from both side. Thanks to Denis
Oleynikov for the report.
- Add the PARALLEL_MIN_ROWS configuration directive to prevent
Oracle's parallel mode to be activated during data export if the
table have less than a certain amount of rows. Default is 100000
rows. This prevent unnecessary fork of Oracle process. Thanks to
Denis Oleynikov for the feature request.
- Fix composite partition MODULUS value. Thanks to Denis Oleynikov
for the report.
- Fix count of partitions that was not including subpartition count.
- Force PostgreSQL user in FDW user mapping to be PG_USER when it is
defined.
- Sometimes Oracle indexes can be defined as follow:
CREATE INDEX idx_err_status_id
ON err_status (status_id, 1);
which generate errors on PostgreSQL. Remove column names composed
of digit only from the translation. Thanks to Denis Oleynikov for
the report.
- Move Oracle indexes or PK defined on partitioned tables to each
partition as PostgreSQL do not support UNIQUE, PRIMARY KEY,
EXCLUDE, or FOREIGN KEY constraints on partitioned tables.
Definition are created in file PARTITION_INDEXES_output.sql
generated with the PARTITION export type. Thanks to Denis
Oleynikov for the feature request.
- Fix parallel data load from Oracle partitioned tables by using
a unique alias. Thanks to Denis Oleynikov for the report.
- Fix export of composite partitioned (range/hash) table when
PG_SUPPORTS_PARTITION is disabled. Thanks to Denis Oleynikov
for the report.
- Remove composite sub partition from the list of partition, this
return a wrong partition count.
- Fix MODULUS value in hash sub partitioning.
- Index and table partitions could be on separate tablespaces.
Thanks to Maxim Zakharov for the patch.
- Fix case where procedure object name is wrongly double quoted.
Thanks to danghb for the report.
- Fix parser to support comment between procedure|function name
and IS|AS keyword. Thanks to danghb for the report.
- Remove dependency to List::Util for the min() function.
2018 08 18 - v19.0
This major release fix several issues reported by users during last
year. It also adds several new features and configuration directives.
New features:
- Add export of Oracle HASH partitioning when PG_SUPPORTS_PARTITION
is enabled. This is a PostgreSQL 11 feature.
- Add SUBTYPE translation into DOMAIN with TYPE and PACKAGE export.
- Add automatic translation of
KEEP (DENSE_RANK FIRST|LAST ORDER BY ...) OVER (PARTITION BY ...)
into
FIRST|LAST_VALUE(...) OVER (PARTITION BY ... ORDER BY ...).
- Add PCTFREE to FILLFACTOR conversion when PCTFREE is upper than
the default value: 10.
- Replace DELETE clause not followed with FROM (optional in Oracle).
- Remove Oracle extra clauses in TRUNCATE command.
- Allow use of NUMBER(*) in DATA_TYPE directive to convert all
NUMBER(*) into the given type whatever is the length. Ex:
DATA_TYPE NUMBER(*):bigint.
- Add a PARALLEL hint to all Oracle queries used to migrate data.
- Add export of Identity Columns from Oracle Database 12c.
- Add translation of UROWID datatype and information in documentation
about why default corresponding type OID will fail at data import.
- Remove unwanted and unused keywords from CREATE TABLE statements:
PARALLEL and COMPRESS.
- Remove TEMPORARY in DROP statements.
- Improve speed of escape_copy() function used for data export.
- Add translation of Oracle functions NUMTOYMINTERVAL() and
NUMTODSINTERVAL().
- Add counting of jobs defined in Oracle scheduler in the migration
assessment feature.
- Add CSMIG in the list of Oracle default system schema
- Fully rewrite data export for table with nested user defined types
DBD::Oracle fetchall_arrayref() is not able to associate complex
custom types to the returned arrays, changed this call to use
fetchrow_array() also used to export BLOB.
- QUERY export will now output translated queries as well as
untranslated ones. This break backward compatibility, previously
only translated query was dumped.
- Auto detect UTF-8 input files to automatically use utf8 encoding.
- Support translation of MySQL global variables.
- Add translation of preprocessor in Oracle external table into
program in foreign table definition. Allow translation of external
table from file.
- Add translation to NVL2() Oracle function.
- Translate CONVERT() MySQL function.
- Translate some form of GROUP_CONCAT() that was not translated.
- Remove call to CHARSET in cast() function, replace it by COLLATE
every where else. This must cover most of the cases but some
specials use might not, so please reports any issue with this
behavior.
- Add -c | --config command line option to ora2pg_scanner to set
custom configuration file to be used instead of ora2pg default:
/etc/ora2pg/ora2pg.conf
- Improve CONNECT BY and OUTER JOIN translation.
- And lot of MySQL to PostgreSQL improvements.
Several new configuration directives have been added:
- Add DEFAULT_PARALLELISM_DEGREE to control PARALLEL hint use
when exporting data from Oracle. Default is disabled.
- Make documentation about KEEP_PKEY_NAMES more explicit about
kind of constraints affected by this directive.
- Add PG_SUPPORTS_IDENTITY configuration directive to enable
export of Oracle identity columns into PostgreSQL 10 feature.
If PG_SUPPORTS_IDENTITY is disabled and there is IDENTITY column
in the Oracle table, they are exported as serial or bigserial
columns. When it is enabled they are exported as IDENTITY columns
like:
CREATE TABLE identity_test_tab (
id bigint GENERATED ALWAYS AS IDENTITY,
description varchar(30)
) ;
If there is non default sequence option set in Oracle, they will
be appended after the IDENTITY keyword. Additionally in both cases
Ora2Pg will create a file AUTOINCREMENT_output.sql with a function
to update the associated sequences with the restart value set to
"SELECT max(colname)+1 FROM tablename". Of course this file must
be imported after data import otherwise sequence will be kept to
start value.
- Add DISABLE_UNLOGGED configuration directive. By default Ora2Pg
export Oracle tables with the NOLOGGING attribute into UNLOGGED
tables. You may want to fully disable this feature because you
will lost all data from unlogged table in case of PostgreSQL crash.
Set it to 1 to export all tables as normal table. When creating a
new migration project using --init_project, this directive is
activated by default. This is not the case in the default
configuration file for backward compatibility.
- Add FORCE_SECURITY_INVOKER configuration directive. Ora2Pg use
the function's security privileges set in Oracle and it is often
defined as SECURITY DEFINER. To override those security privileges
for all functions and use SECURITY DEFINER instead, enable this
directive.
- Add AUTONOMOUS_TRANSACTION in configuration to enable translation
of autonomous transactions into a wrapper function using dblink
or pg_background extension. If you don't want to use this feature
and just want to export the function as a normal one without the
pragma call, disable this directive.
- Add documentation about COMMENT_SAVEPOINT configuration directive.
- Major rewrite in PACKAGE parser to better support global variables
detection. Global variable that have no default values are now
always initialized to empty string in file global_variables.conf
so that we see that they exists. This might not change the global
behavior.
I especially want to thank Pavel Stehule and Eric Delanoe who spent
lot of time this year to help me to improve the PL/SQL to plpgsql
translation and also Krasiyan Andreev who help a lot to finalize
the MySQL to PostgreSQL migration features.
Here is a complete list of changes and acknowledgments:
- Fix translation of "varname cursor%ROWTYPE;". Thanks to Philippe
Beaudoin for the report.
- Fix return of autonomous transaction dblink call when function has
OUT parameter. Thanks to Pavel Stehule for the report.
- Add Oracle to PostgreSQL translation of windows functions
KEEP (DENSE_RANK FIRST|LAST ORDER BY ...) OVER (PARTITION BY ...)
Thanks to Swapnil bhoot929 for the feature request.
- Fix "ORA-03113: end-of-file on communication channel" that what
generated by a too long query send to Oracle. The size of queries
sent to Oracle to retrieve object information depend of the ALLOW
and EXCLUDE directives. If you have lot of objects to filter you
can experience this kind of non explicit error. Now Ora2pg use
bind parameter to pass the filters values to reduce the size of
the prepared query. Thanks to Stephane Tachoire for the report.
- Add SUBTYPE translation into DOMAIN with TYPE and PACKAGE export.
Thanks to Francesco Loreti for the feature request.
- Fix PLS_INTEGER replacement.
- Remove precision for RAW|BLOB as type modifier is not allowed for
type "bytea".
- Fix call of schema.pckg.function() in indexes with a replacement
with pckg.function(). Thanks to w0pr for the report.
- Fix translation of UPDATE trigger based on columns:
"BEFORE UPDATE OF col1,col2 ON table".
Thanks to Eric Delanoe for the report.
- Remove single / from input file that was causing a double END in
some case. Thanks to Philippe Beaudoin for the report.
- Limit translation of PCTFREE into FILLFACTOR when PCTFREE is upper
than the Oracle default value: 10. With PostgreSQL 100 (complete
packing) is the default.
- Add PCTFREE to FILLFACTOR conversion. Thanks to Maxim Zakharov
for the patch.
- Remove TRUNCATE extra clauses. Thanks to e7e6 for the patch.
- Fix type conversion when extra \n added after ;. Thanks to
Maxim Zakharov for the patch.
- Fix DELETE clause not followed with FROM (optional in Oracle).
Thanks to Philippe Beaudoin for the patch.
- Limit call to ALL_TAB_IDENTITY_COLS to version 12+. Thanks to
Andy Garfield for the report.
- Fix comment parsing. Thanks to Philippe Beaudoin for the report.
- Allow use of NUMBER(*) in DATA_TYPE directive to convert all
NUMBER(*) into the given type whatever is the length.
Thanks to lingeshpes for the feature request.
- Fix bug in function-based index export. Thanks to apol1234 for
the report.
- Add PARALLEL hint to all data export queries. Thanks to jacks33
for the report.
- Make documentation about KEEP_PKEY_NAMES more explicit about kind
of constraints affected by this directive.
- Fix export of identity columns by enclosing options between
parenthesis and replacing CACHE 0 by CACHE 1. Thanks to swmcguffin
devtech for the report.
- Add parsing of identity columns from file.
- Fix unwanted replacement of IF () in MySQL code. Thanks to
Krasiyan Andreev for the report.
- Fix to_char() translation, thanks to Eric Delanoe for the report.
- Fix untranslated PERFORM into exception. Thanks to Pavel Stehule
for the report.
- Add _get_entities() function to MySQL export. It returns nothing,
AUTO_INCREMENT column are translated with corresponding types,
smallserial/serial/bigserial.
- Fix look at encrypted column on Oracle prior to 10. Thanks to
Stephane Tachoires for the patch.
- Add export of Identity Columns from Oracle Database 12c. Thanks
to swmcguffin-devtech for the feature request.
- Prevent Ora2Pg to scan ALL_SCHEDULER_JOBS for version prior to 10
Thanks to Stephane Tachoires for the patch.
- Fix pull request #648 to log date only when debug is enabled and
use POSIX strftime instead of custom gettime function.
- Add system time to debug log info. Thanks to danghb for the patch.
- Fix parsing of trigger from file and exception.
- Fix very slow export of mysql tablespace when number of table is
large. Thanks to yafeishi for the report.
- Fix translation of CAST( AS unsigned). Thanks to Krasiyan Andreev.
- Fix MySQL character length to use character_maximum_length
instead of equal character_octet_length. Thanks to yafeishi for
the report.
- Fix custom replacement of MySQL data type. Thanks to Krasiyan
Andreev for the report.
- Fix replacement of call to open cursor with empty parenthesis.
Thanks to Philippe Beaudoin for the report.
- Fix MySQL data type conversion in function declaration. Thanks to
Krasiyan Andreev for the report.
- Fix error with -INFINITY as default value for date or timestamp
columns.
- Fix procedure call rewrite with unwanted comma on begin of
parameter list. Thanks to Pavel Stehule for the report.
- Fix handling of foreign keys when exporting data and DROP_FKEYS
is enabled and ALLOW/EXCLUDE directive is set. Now Ora2Pg will
first drop all foreign keys of a table in the export list and all
foreign keys of other tables pointing to the table. After data
import, it will recreate all of these foreign keys. Thanks to
Eric Delanoe for the report.
- Fix broken transformation of procedure call with default parameter
Thanks to Pavel Stehule for the report.
- Translate call to TIMESTAMP in partition range values into a cast.
Thanks to markiech for the report.
- Fix CONNECT BY translation when the query contain an UNION. Thanks
to mohammed-a-wadod for the report.
- Fix CONNECT BY with PRIOR on the right side of the predicat.
- Fix outer join translation when the (+) was in a function, ex:
WHERE UPPER(trim(VW.FRIDAY))= UPPER(trim(FRIDAY.NAME(+))).
- Order outer join pending tables in from clause.
- Order by object name comments and indexes export.
- Fix outer join translation when the table is not in the from
clause. Thanks to Cyrille Lintz for the report.
- Try to fix potential Oracle schema prefixing PostgreSQL schema
name in CREATE SCHEMA. Thanks to Cyrille Lintz for the report.
- Fix error in TRIM() translation. Thanks to Cyrille Lintz for the
report.
- Add translation of UROWID datatype and information in documentation
about why default corresponding type OID will fail at data import.
Thanks to Cyrille Lintz for the report.
- Fix bug in exporting boolean default values in column definition.
- Fix bug in column parsing in CREATE TABLE.
- Adapt default value for data type changed to boolean.
- Fix bad handling of -D (data_type) option.
- Change behavior in the attempt to set MySQL global variable type.
Now variable type will be timestamp if the variable name contains
datetime, time if the name contains only time and date for date.
Thanks to Krasiyan Andreev for the report.
- Fix function replacement in MySQL declare section. Thanks to
Krasiyan Andreev fr the report.
- Apply REPLACE_ZERO_DATE to default value in table declaration.
Thanks to Krasiyan Andreev for the report.
- Add support to embedded comment in table DDL.
- Fix replacement of data type for MySQL code. Thanks to Krasiyan
Andreev for the report.
- Fix MySQL type replacement in function. Thanks to Krasiyan Andreev
for the report.
- Improve speed of escape_copy() function used for data export.
Thanks to pgnickb for the profiling.
- Add translation of Oracle functions NUMTOYMINTERVAL() and
NUMTODSINTERVAL(). Thanks to Pavel Stehule for the report.
- Counting jobs defined in Oracle scheduler. Thanks to slfbovey
for the patch.
- Fix several issue in create table DDL parser:
- remove double quote of object name when a list of column is
entered
- split of table definition to extract column and constraint
parts is now more efficient
- remove dot in auto generated constraint name when a schema
is given in table name
- fix default values with space that was breaking the parser
- Remove use of bignum perl module that reports error on some
installation. Thanks to Cyrille Lintz for the report.
- Fix a typo preventing perldoc to complete. Thanks to slfbovey
for the patch.
- Fully rewrite data export for table with nested user defined types
DBD::Oracle fetchall_arrayref() is not able to associate complex
custom types to the returned arrays, changed this call to use
fetchrow_array() also used to export BLOB. Thanks to lupynos for
the report.
- Fix renaming of temporary files during partitions data export.
- Fix Oracle use of empty string as default value for integers.
Oracle allow such declaration: SOP NUMBER(5) DEFAULT '' which
PostgreSQL does not support. Ora2Pg now detect this syntax and
replace empty string with NULL. Thanks to ricdba for the report.
- Add detection of Oracle version before setting datetime format,
needed for Oracle 8i compatibility.
- Export of tables from Oracle database are now ordered by name by
default. Thanks to Markus Roth for the report.
- Fix an other case of missing translation of UNSIGNED into bigint.
Thanks to Krasiyan Andreev for the report.
- Force replacement of double quote into single quote for MySQL view
and function code.
- Fix case when SET @varname := ... is used multiple time in the
same function. Thanks to Krasiyan Andreev for the report.
- Fix case where SET @varname := ... was not translated. Thanks to
Krasiyan Andreev for the report.
- Adjust the regex pattern of last patch.
- Fix unwanted newline after hint replacement that could break
comments. Thanks to Pavel Stehule for the report.
- Fix if() replacement in query. Thanks to Krasiyan Andreev for the
report.
- Remove extra parenthesis in some form of JOIN. Thanks to Krasiyan
Andreev for the report.
- Fix untranslated call to UNSIGNED, now translated as bigint.
- Thanks to Krasiyan Andreev for the report.
- Fix translation of double(p,s) into decimal(p,s).
- Remove use of SET when an assignment is done through a SELECT
statement. Thanks to Krasiyan Andreev for the report.
- Fix non-quoted reserved keywords in INSERT / COPY statements when
exporting data. Thanks to Pavel Stehule for the report.
- Fix partition data export to file, temporary files for partition
output was not renamed at export end then data was not loaded.
- Fix double operator := during function with out param rewrite.
- Fix commit f1166e5 to apply changes when FILE_PER_FUNCTION is
disable or when an input file is given.
- Fix translation of LOCATE(). Thanks to Krasiyan Andreev for the
report.
- Fix case where MySQL GROUP_CONCAT() function was not translated.
Thanks to Krasiyan Andreev for the report.
- Fix :new and :old translation in triggers.
- Fully rewrite function call qualification process, the second pass
now is only use to requalify call to pkg.fct into pkg_ftc when
PACKAGE_AS_SCHEMA is disable. The replacement of all function
calls using double quote when a non supported character is used or
when PRESERVE_CASE is enabled has been completely removed as this
takes too much time to process for just very few case. So by
default now Ora2Pg will not go through the second pass. This can
change in the future especially if this is more performant to
process PERFORM replacement. Thanks a lot to Eric Delanoe for his
help on this part.
- Exclude function and procedure not from package to be used in
requalify call. Thanks to Eric Delanoe for the report.
- Fix function name qualification in multiprocess mode.
- Fix unqualified function call due to unclose file handle.
- Prevent try to requalify function call if the function is
not found in the file content.
- Remove ALGORITHM=.*, DEFINER=.* and SQL SECURITY DEFINER from
MySQL DDL code.
- An other missing change to previous commit on qualifying function
call.
- Limit function requalification to export type: VIEW, TRIGGER,
QUERY, FUNCTION, PROCEDURE and PACKAGE.
- Auto detect UTF-8 input files to automatically use utf8 encoding.
- Remove all SHOW ERRORS and other call to SHOW in Oracle package
source as they was badly interpreted as global variable.
- Fix MySQL CREATE TABLE ... SELECT statement.
- Fix pending translation issue on some DATE_FORMAT() case.
Thanks to Krasiyan Andreev for the report.
- Fix translation of IN (..) in MySQL view. Thanks to Krasiyan
Andreev for the report.
- Fix MySQL date format with digit.
- Fix DATE_FORMAT, WHILE and IFNULL translation issues.
- Fix not translated MySQL IF() function.
- Fix other MySQL translation issues for @variable. Thanks to
Krasiyan Andreev for the report.
- Fix issue in MySQL IF translation with IN clause. Thanks to
Krasiyan Andreev for the report.
- Clarify comment about XML_PRETTY directive. Thanks to TWAC
for the report.
- Fix remaining MySQL translation issues for @variable reported
in issue #590.
- Fix no translated := in SET statement.
- Fix output order of translated function.
- Fix non printable character or special characters that make
file encoding to ISO-8859 instead of utf8. Thanks to twac for
the report.
- Prevent MySQL global variable to be declared twice. Thanks to
Krasiyan Andreev for the report.
- Support translation of MySQL global variables. Session variable
@@varname are translated to PostgreSQL GUC variable and global
variable @varname are translated to local variable defined in a
DECLARE section. Ora2Pg tries to gather the data type by using
integer by default, varchar if there is a constant string ('...')
in the value and a timestamp if the variable name have the keyword
date or time inside. Thanks to Krasiyan Andreev for the feature
request.
- Fix DATE_ADD() translation.
- Add translation of preprocessor in Oracle external table into
program in foreign table definition. Thanks to Thomas Reiss for
the report. Allow translation of external table from file.
- Fix case where IF EXISTS might not be append when it is not
supported by PG.
- Translate CONVERT() MySQL function. Thanks to Krasiyan Andreev
for the report.
- Translate some form of GROUP_CONCAT() that was not translated.
Thanks to Krasiyan Andreev for the report.
- Apply same principe with COMMIT in MySQL function code than in
Oracle code. It is kept untouched to be able to detect a possible
change of code logic. It can be automatically commented if
COMMENT_COMMIT_ROLLBACK is enabled. Also I have kept the START
TRANSACTION call but it is automatically commented.
- Add mysql_enable_utf8 => 1 to MySQL connection to avoid issues
with encoding. Thanks to Krasiyan Andreev for the report.
- Prevent removing of comment on MySQL function and add a "COMMENT
ON FUNCTION" statement at end of the function declaration. Thanks
to Krasiyan Andreev for the report.
- Fix translation of types in MySQL function parameter. Thanks to
Krasiyan Andreev for the report.
- Remove START TRANSACTION from MySQL function code. Thanks to
Krasiyan Andreev for the report.
- Fix previous patch, we do not need to look forward for function
or procedure definition in VIEW export and there is no package
with MySQL. Thanks to Krasiyan Andreev for the report.
- Fix call to useless function for MySQL function.
- Add rewrite of MySQL function call in function or procedure code
translation and some other translation related to MySQL code.
- Fix ora2pg_scanner when exporting schema with $ in its name.
Thanks to Aurelien Robin for the report.
- Disable number of microsecond digit for Oracle version 9. Thanks
to Aurelien Robin for the report.
- Do not look at encrypted column for DB version < 10. Thanks to
Aurelien Robin for the report.
- Fix MySQL call to charset in cast function. MySQL charset "utf8"
is also set to COLLATE "C.UTF-8". Thanks to Krasiyan Andreev for
the report.
- Fix two bug in CONNECT BY and OUTER JOIN translation.
- Forgot to handle exception to standard call to IF in MySQL IF()
translation. Thanks to Krasiyan Andreev for the report.
- Forgot to apply previous changes to procedure.
- Fix IF() MySQL replacement when nested and when containing an
IN (...) clause. Thanks to Krasiyan Andreev for the report.
- Fix double BEGIN on MySQL function export. Thanks to Krasiyan
Andreev for the report.
- Fix enum check constraint name when PRESERVE_CASE is enabled.
- Fix case where object with LINESTRING and CIRCULARSTRING was
exported as MULTILINESTRING instead of MULTICURVE.
- Fix export of MULTICURVE with COMPOUNDCURVE. Thanks to Petr Silhak
for the report.
- Fix several issue in MySQL table DDL export. Thanks to Krasiyan
Andreev for the report.
- Fix MySQL auto_increment data type translation and columns export
order.
- Fix translation of MySQL function CURRENT_TIMESTAMP(). Thanks to
Krasiyan Andreev for the report.
- Fix export of MySQL alter sequence name when exporting auto
increment column. Thanks to Krasiyan Andreev for the report.
- Replace IF() call with CASE ... END in VIEW and QUERY export for
MySQL. Thanks to Krasiyan Andreev for the feature request.
- Replace backquote with double quote on mysql statements when read
from file.
- Fix bug in REGEXP_SUBSTR replacement.
- Prevent replacement with same function name from an other package.
Thanks to Eric Delanoe for the report.
- Apply same STRICT rule for SELECT INTO to EXECUTE INTO. Thanks to
Pavel Stehule for the report.
- Fix extra parenthesis removing when a OR clause is present. Thanks
to Pavel Stehule for the report.
- Keep autonomous pragma commented when conversion is deactivated
to be able to identify functions using this pragma.
- Fix bug in replacement of package function in string constant.
- Fix malformed replacement of array element calls. Thanks to Eric
Delanoe for the report.
- Fix unwanted replacement of TO_NUMBER function. Thanks to Torquem
for the report.
- Add an example of DSN for MySQL in ORACLE_DSN documentation.
Thanks to François Honore for the report.
- Fix typo in default dblink connection string. Thanks to Pavel
Stehule for the report.
- Add information about Oracle Instant Client installation. Thanks
to Jan Birk for the report.
- Replace Oracle array syntax arr(i).x into arr[i].x into PL/SQL
code. Thanks to Eric Delanoe for the report.
- Use a more generic connection string for DBLINK. It will use
unix socket by default to connect and the password must be set
in .pgpass. This will result in the following connection string:
format('port=%s dbname=%s user=%', current_setting('port'),
current_database(), current_user)
If you want to redefine this connection string use DBLINK_CONN
configuration directive. Thanks to Pavel Stehule for the feature
request.
- Fix missing RETURN NEW in some trigger translation. Thanks to
Pavel Stehule for the report.
- Fix a missing but non mandatory semi-comma.
- Keep PKs/unique constraints which are deferrable in Oracle also
deferrable in PostgreSQL. Thank to Sverre Boschman for the patch.
- Fix parsing and translation of CONNECT BY. Thanks to bhoot929
for the report.
- Fix FDW export when exporting all schema. Thanks to Laurenz Albe
for the report.
- Add a note about multiple value in export type that can not
include COPY or INSERT together with others export type.
- Fix duplicate condition. Thanks to Eric Delanoe for the report.
- Fix unwanted translation into PERFORM after INTERSECT.
- Comment savepoint in code. Thanks to Pavel Stehule for the patch.
- Fix "ROLLBACK TO" that was not commented. Thanks to Pavel Stehule
for the report.
- Fix restore of constant string when additional string constant
regex are defined in configuration file.
- Fix translation of nextval with sequence name prefixed with their
schema.
- Cast call to TO_DATE(LOCALTIMESTAMP,...) translated into
TO_DATE(LOCALTIMESTAMP::text,...). Thanks to Keshav kumbham
for the report.
- Remove double quote added automatically by Oracle on view
definition when PRESERVE_CASE is not enable. Thanks to JeeIPI for
the report.
- Fix translation of FROM_TZ with a call to function as first
parameter. Thanks to TrungPhan for the report.
- Fix package export when FILE_PER_FUNCTION is set. Thanks to
Julien Rouhaud for the report.
- Add translation of REGEXP_SUBSTR() with the following rules:
Translation of REGEX_SUBSTR( string, pattern, [pos], [nth])
converted into
SELECT array_to_string(a, '')
FROM regexp_matches(substr(string, pos), pattern, 'g')
AS foo(a)
LIMIT 1 OFFSET (nth - 1);
Optional fifth parameter of match_parameter is appended to 'g'
when present. Thanks to bhoot929 for the feature request.
- Add count of REGEX_SUBSTR to migration assessment cost.
- Add translation support of FROM_TZ() Oracle function. Thanks
to trPhan for the feature request.
- Forces ora2pg to output a message when a custom exception code
has less than 5 digit.
- Fix errcode when Oracle custom exception number have less than
five digit. Thanks to Pavel Stehule for the report.
- Fix case where custom errcode are not converted. Thanks to Pavel
Stehule for the report.
- Fix print of single semicolon with empty line in index export.
- Fix problem with TO_TIMESTAMP_TZ conversion. Thanks to Keshav-
kumbham for the report.
- Fix unwanted double quote in index column with DESC sorting.
Thanks to JeeIPI for the report.
- Fix non detection case of tables in from clause for outer join
translation. Thanks to Keshav for the report.
- Fix unwanted replacement of = NULL into IS NULL in update
statement. Thanks to Pavel Stehule for the report.
- Force schema name used in TEST action to lowercase. Thanks to
venkatabn for the report.
- Fix export of spatial geometries with CURVEPOLYGON + COMPOUNDCURVE
Thanks to kabog for the report.
2017 09 01 - v18.2
This release fix several issues reported during the last six months.
It also adds several new features and configuration directives:
- Add translation of SUBSTRB into substr.
- Allow use of array in MODIFY_TYPE to export Oracle user defined
type that are just array of some data type. For example:
CREATE OR REPLACE TYPE mem_type IS VARRAY(10) of VARCHAR2(15);
can be directly translated into text[] or varchar[]. In this case
use the directive as follow: MODIFY_TYPE CLUB:MEMBERS:text[]
Ora2Pg will take care to transform all data of this column into
the correct format. Only arrays of characters and numerics types
are supported.
- Add translation of Oracle function LISTAGG() into string_agg().
- Add TEST_VIEW action to perform a simple count of rows returned by
views on both database.
- Translate SQL%ROWCOUNT into GET DIAGNOSTICS rowcount = ROW_COUNT
and add translation of SQL%FOUND.
- Add translation of column in trigger event test with IS DISTINCT,
for example: IF updating('ID') THEN ... will be translated into:
IF TG_OP = 'UPDATE' AND NEW.'ID' IS DISTINCT FROM OLD.'ID' then...
- Replace UTL_MATH.EDIT_DISTANCE function by fuzzymatch levenshtein.
- Allow use of MODIFY_STRUCT with TABLE export. Table creation DDL
will respect the new list of columns and all indexes or foreign
key pointing to or from a column removed will not be exported.
- Add export of partition and subpartition using PostgreSQL native
partitioning.
- Auto detect encrypted columns and report them into the assessment.
SHOW_COLUMN will also mark columns as encrypted.
- Add information to global temporary tables in migration assessment.
- Add experimental DATADIFF functionality.
- Allow use of multiprocess with -j option or JOBS to FUNCTION and
PROCEDURE export. Useful if you have thousands of these objects.
- Force RAW(N) type with default value set to sys_guid() as UUID
on PostgreSQL.
- Replace function with out parameter using select into. For example
a call to: get_item_attr( attr_name, p_value );
where p_value is an INOUT parameter, will be rewritten as
p_value := get_item_attr( attr_name, p_value );
If there is multiple OUT parameters, Ora2Pg will use syntax:
SELECT get_item_attr( attr_name, p_value )
INTO (attr_name, p_value);
- Add translation of CONNECT BY using PostgreSQL CTE equivalent.
This translation also include a replacement of LEVEL and
SYS_CONNECT_BY_PATH native Oracle features. On complex queries
there could still be manual editing but all the main work is done.
- Add support to user defined exception, errcode affected to each
custom exception start from 50001.
- Translate call to to_char() with a single parameter into a cast
to varchar. Can be disabled using USE_ORAFCE directive.
- Improve ora2pg_scanner to automatically generates migration
assessment reports for all schema on an Oracle instance. Before
the schema name to audit was mandatory, now, when the schema
is not set Ora2Pg will scan all schema. The connexion user need
to have DBA privilege. Ora2Pg will also add the hostname and SID
as prefix in the filename of the report. This last change forbids
ora2pg_scanner to overwrite a report if the same schema name is
found in several databases.
Several new configuration directives have been added:
- Add USE_ORAFCE configuration directive that can be enabled if you
want to use functions defined in the Orafce library and prevent
Ora2Pg to translate call to these functions. The Orafce library
can be found here: https://github.com/orafce/orafce
By default Ora2pg rewrite add_month(), add_year(), date_trunc()
and to_char() functions, but you may prefer to use the Orafce
functions that do not need any code transformation. Directive
DATE_FUNCTION_REWRITE has been removed as it was also used to
disable replacement of add_month(), add_year() and date_trunc()
when Orafce is used, useless now.
- Add FILE_PER_FKEYS configuration directive to allow foreign key
declaration to be saved in a separate file during schema export.
By default foreign keys are exported into the main output file or
in the CONSTRAINT_output.sql file. If enabled foreign keys will be
exported into a file named FKEYS_output.sql
- Add new COMMENT_COMMIT_ROLLBACK configuration directive. Call to
COMMIT/ROLLBACK in PL/SQL code are kept untouched by Ora2Pg to
force the user to review the logic of the function. Once it is
fixed in Oracle source code or you want to comment this calls
enable the directive.
- Add CREATE_OR_REPLACE configuration directive. By default Ora2Pg
use CREATE OR REPLACE in function DDL, if you need not to override
existing functions disable this configuration directive, DDL will
not include OR REPLACE.
- Add FUNCTION_CHECK configuration directive. Disable this directive
if you want to disable check_function_bodies.
SET check_function_bodies = false;
It disables validation of the function body string during CREATE
FUNCTION. Default is to use de postgresql.conf setting that enable
it by default.
- Add PG_SUPPORTS_PARTITION directive, disabled by default.
PostgreSQL version prior to 10.0 do not have native partitioning.
Enable this directive if you want to use PostgreSQL declarative
partitioning instead of the old style check constraint and trigger.
- Add PG_SUPPORTS_SUBSTR configuration directive to replace substr()
call with substring() on old PostgreSQL versions or some fork
like Redshift.
- Add PG_INITIAL_COMMAND to send some statements at session startup.
This directive is the equivalent used for Oracle connection,
ORA_INITIAL_COMMAND. Both can now be used multiple time now.
- Add DBLINK_CONN configuration directive. By default if you have
an autonomous transaction translated using dblink extension the
connection is defined using the values set with PG_DSN, PG_USER
and PG_PWD. If you want to fully override the connection string
use this directive to set the connection in the autonomous
transaction wrapper function. For example:
DBLINK_CONN port=5432 dbname=pgdb host=localhost user=pguser password=pgpass
- Add STRING_CONSTANT_REGEXP configuration directive. Ora2Pg replace
all string constant during the pl/sql to plpgsql translation,
string constant are all text include between single quote. If you
have some string placeholder used in dynamic call to queries you
can set a list of regexp to be temporary replaced to not break the
parser. For example:
STRING_CONSTANT_REGEXP <cfqueryparam value=".*">
The list of regexp must use the semi colon as separator.
- Add FUNCTION_STABLE configuration directive. By default Oracle
functions are marked as STABLE as they can not modify data unless
when used in PL/SQL with variable assignment or as conditional
expression. You can force Ora2Pg to create these function as
VOLATILE by disabling this configuration directive.
- Add new TO_NUMBER_CONVERSION configuration directive to control
TO_NUMBER translation behavior. By default Oracle call to function
TO_NUMBER will be translated as a cast into numeric. For example,
TO_NUMBER('10.1234') is converted into PostgreSQL call:
to_number('10.1234')::numeric.
If you want you can cast the call to integer or bigint by changing
the value of the configuration directive. If you need better
control of the format, just set it as value, for example:
TO_NUMBER_CONVERSION 99999999999999999999D9999999999
will convert the code above as:
TO_NUMBER('10.1234', '99999999999999999999D9999999999')
Any value of the directive that it is not numeric, integer or
bigint will be taken as a mask format. If set to none, then no
conversion will be done.
- Add LOOK_FORWARD_FUNCTION configuration directive which takes a
list of schema to get functions/procedures meta information that
are used in the current schema export. When replacing call to
function with OUT or INOUT parameters, if a function is declared
in an other package then the function call rewriting can not be
done because Ora2Pg only knows about functions declared in the
current schema. By setting a comma separated list of schema as
value of the directive, Ora2Pg will look forward in these packages
for all functions, procedures and packages declaration before
proceeding to current schema export.
- Add PG_SUPPORTS_NAMED_OPERATOR to control the replacement of the
PL/SQL operator used in named parameter => with the PostgreSQL
proprietary operator := Disable this directive if you are using
PG < 9.5
- Add a warning when Ora2Pg reorder the parameters of a function
following the PostgreSQL rule that all input parameters following
a parameter with a default value must have default values as well.
In this case, Ora2Pg extracts all parameters with default values
and put them at end of the parameter list. This is to warn you
that a manual rewrite is required on calls to this function.
New command line options have been added:
- Add -N | --pg_schema command line option to be able to override
the PG_SCHEMA configuration directive. When this option is set
at command line, EXPORT_SCHEMA is automatically activated.
- Add --no_header option with equivalent NO_HEADER configuration
directive to output the Ora2Pg header but just the translated
code.
There is also some behavior changes from previous release:
- Remove SysTimestamp() from the list of not translated function,
it is replaced with CURRENT_TIMESTAMP for a long time now.
- Change migration assessment cost to 84 units (1 day) for type
TABLE, INDEX and SYNONYM and to 168 units (2 days) for TABLE
PARTITION and GLOBAL TEMPORARY TABLE, this is more realistic.
- Set minimum assessment unit to 1 when an object exists.
Improve PL/SQL code translation speed.
- Change behavior of COMPILE_SCHEMA directive used to force Oracle
to compile schema before exporting code. When this directive is
enabled and SCHEMA is set to a specific schema name, only invalid
objects in this schema will be recompiled. When SCHEMA is not set
then all schema will be recompiled. To force recompile invalid
object in a specific schema, set COMPILE_SCHEMA to the schema name
you want to recompile. This will ask to Oracle to validate the
PL/SQL that could have been invalidate after a export/import for
example. The 'VALID' or 'INVALID' status applies to functions,
procedures, packages and user defined types.
- Default transaction isolation level is now set to READ COMMITTED
for all action excluding data export.
- Oracle doesn't allow the use of lookahead expression but you may
want to exclude some objects that match the ALLOW regexp you have
defined. For example if you want to export all table starting
with E but not those starting with EXP it is not possible to do
that in a single expression.
Now you can start a regular expression with the ! character to
exclude all objects matching the regexp given just after. Our
previous example can be written as follow: ALLOW E.* !EXP.*
it will be translated into
REGEXP_LIKE(..., '^E.*$') AND NOT REGEXP_LIKE(..., '^EXP.*$')
in the object search expression.
- Fix quoting of PG_SCHEMA with multiple schema in search path. The
definition of the search path now follow the following behavior:
* when PG_SCHEMA is define, always set search_path to its value.
* when EXPORT_SCHEMA is enabled and SCHEMA is set, the search_path
is set the name of the schema.
- Remove forcing of export_schema when pg_schema is set at command
line. This could change the behavior of some previous use of these
variables and the resulting value of the search_path but it seems
much better understandable.
- Rewrite translation of raise_application_error to use RAISE
EXCEPTION with a message and the SQLSTATE code. Oracle user
defined code -20000 to -20999 are translated to PostgreSQL
user define code from 45000 to 45999. Call to
raise_application_error(mySQLCODE, myErrmsg);
will be translated into
RAISE EXCEPTION '%', myErrmsg USING ERRCODE = mySQLCODE;
- Remove migration assessment cost for TG_OP and NOT_FOUND they
might be fully covered now.
Here is the complete list of changes:
- Fix bad inversion of HAVING/GROUP BY clauses. Thanks to bhoot929
for the report.
- Fix case of non translation of type in CAST() function. Thanks to
Keshavkumbham for the report.
- Fix spatial data export when using direct import into PostgreSQL
and WKT or INTERNAL format. This can still be improved. Thanks to
Valeria El-Samra for the report.
- Improve translation of trunc() into date_trunc. Thanks to bhoot929
for the report.
- Translate to_char() without format into a simple cast to varchar.
Thanks to bhoot929 for the report.
- Fix line comment which does not disable multi-line comment.
Thanks to Pavel Stehule for the report.
- Fix overridden of output file global_variables.conf with
multiple packages. Thanks to Oliver Del Rosario for the report.
- Fix export of data stored in a nested user defined type. Thanks
to lupynos for the report.
- Fix data export from Oracle user defined types, where output of
ROW(...) does not distinguish numeric from string or other types
that need to be formatted. Thanks to Petr Silhak for the report.
- Fix broken replacement of package procedure name. Thanks to Pavel
Stehule for the report.
- Add FLOWS_010600 to the objects exclusion listr.
- Improve view/trigger migration assessment accuracy.
- Fix OUTER JOIN (+) translation, all join filters with constant
was written into the WHERE clause by default. Write them into the
JOIN clause.
- Fix weight of the number of triggers and views in the report with
a limit of 2 man-days, of course SQL difficulties are still add
after this limit.
- Fix alias added to condition which is not a sub query. Thanks to
nitinmverma for the report.
- Fix wrong translation of OUTER JOIN with subquery in FROM clause.
Thanks to nitinmverma for the report.
- Fix typo preventing exclusion of system SYNONYM.
- Fix an other case of bad translation of END fct_name. Thanks to
nitinmverma for the report.
- Fix unwanted translation of REGEXP_SUBSTR() in REGEXP_SUBSTRING().
Thanks to nitinmverma for the report.
- Fix broken translation of decode(). Thanks to nitinmverma for the
report.
- Fix error "Malformed UTF-8 character (unexpected continuation byte
0xbf, with no preceding start byte) in pattern match" by including
an arbitrary non-byte character into the pattern. Thanks to Bob
Sislow for the report.
- Fix missing translation of trunc() with date_trunc(). Thanks to
nitinmverma for the report.
- Add migration assessment weight to concatenation.
- Fix space between operator, ex: a < = 15 must be translated as
a <= 15. Thanks to nitinmverma for the report.
- Handle translation of named parameters in function calls. Thanks
to Julien Rouhaud for the patch.
- Fix missing renaming of _initial_command() method.
- Fix right outer join translation by converting them to left outer
join first. Thanks to Julien Rouhaud for the hint.
- Fix TEST on default value count and functions belonging to others
than public schema, especially functions of packages.
- Fix default number of man-days in migration assessment. Thanks to
Nate Fitzgerald for the report.
- Add host information into the filename of the report to prevent
some additional case of report overriding. Thanks to Aurelien
Robin for the report.
- Prevent ora2pg script to complain if no ora2pg.conf file is found
when a DSN is passed at command line and that user connection is
set in the environment variables.
- Do not declare a function stable if there is update/insert/delete
statement inside.
- Improve ora2pg_scanner to generate report of each instance schema
when the schema to audit is not set. Thanks to Thomas Reiss for
the patch.
- Fix parser failure with quote in comments. Thanks to Eric Delanoe
for the report.
- Fix case where NVL is not replaced by COALESCE.
- Add parenthesis to simple package.function call without parameter.
- Fix replacement of INSTR() with optional parameters. Thanks to
Pavel Stehule for the report.
- Translate SQL%ROWCOUNT to GET DIAGNOSTICS rowcount = ROW_COUNT.
Thanks to Pavel Stehule for the report.
- Add translation of SQL%FOUND. Thanks to Pavel Stehule.
- Remove qualifier in create type, "CREATE TYPE fusion.mytable AS
(fusion.mytable fusion.finalrecord[]);" becomes "CREATE TYPE
fusion.mytable AS (mytable fusion.finalrecord[]);". Thanks to
Julien Rouhaud for the report.
- Fix extra comma in FROM clause of triggers in outer join
translation. Thanks to Pavel Stehule fora the report.
- Use record for out parameters replacement only where there is more
than one out parameter. Thanks to Pavel Stehule for the patch.
- Add date type to the inout type array. Thanks to Pavel Stehule for
the report.
- Remove possible last spaces in inout type detection.
- Fix REGEXP_LIKE translation.
- Fix count of default values during test action.
- Fix removing of double quote over column name in view declaration.
- Do not set default value if it is NULL, this is already the case.
- Fix data export that was truncated to the last DATA_LIMIT lines.
Thanks to Michael Vitale for the report.
- Fix an other bug in rewriting call to function with OUT parameter.
Thanks to Pavel Stehule for the report.
- Fix autodetection of composite out parameters.
- Merge typo on PLPGSQL
- Fix typo to PLPGSQL keyword. Thanks to Vinayak Pokale.
- Fix regexp failure in is_reserved_words() method. Thanks to
Patrick Hajek for the patch.
- Use only one declaration of ora2pg_r RECORD, it is reusable.
- Fix transformation of procedure CALL with OUT parameter that can't
works when procedure/function has minimally one OUT parameter is
of composite type. Thanks to Pavel Stehule for the patch.
- Second attempt to fix outer join translation in triggers. Thanks
to Pavel Stehule for precious help.
- Fix RAISE NOTICE replacement with double % placeholder. Thanks to
Pavel Stehule for the report.
- Fix call to function replacement for function registered with
double quote. Thanks to Pavel Stehule for the report.
- Change assessment score of TG_OP.
- Fix replacement of outer join in triggers by adding pseudo tables
NEW and OLD to the list of tables. Thanks to Pavel Stehule for the
report.
- Handle custom exception in declare section of triggers.
- Fix FUNCTION_CHECK option, it will be set in all file header.
Thanks to Pavel Stehule for the report.
- Replace call to ALL_TABLES to USER_TABLES when USER_GRANTS is
enabled. Thanks to Bob Sislow.
- Removed PRAGMA EXCEPTION_INIT() from declare section. Thanks to
Pavel Stehule for the report.
- Fix constant string that was breaking the parser. Thanks to Pavel
Stehule for the report.
- Fix missing space between EXCEPTION and WHEN. Thanks to Pavel
Stehule for the report.
- Fix broken function header resulting in missing space before OUT
keyword. Thanks to Pave Stehule for the report.
- Fix invalid RAISE command. Thanks to Pavel Stehule for the report.
- Add port information to ORACLE_DSN in documentation and config
file. Thanks to Bob Sislow for the report.
- Fix broken declaration in triggers related to FOR cycle control
variable when there is no declare section. Thanks to Pavel
Stehule for the report.
- Fix broken declaration in triggers related to FOR cycle control
variable. Thanks to Pavel Stehule for the report.
- Fix unterminated C style comment in trigger. Thanks to Pavel
Stehule for the report.
- Fix bug in package+function precedence replacement. Thanks to
Eric Delanoe for the report.
- Fix unwanted and broken export of tables created with CREATE TABLE
tablename OF objType. Thanks to threenotrump for the report.
- Add explanation on using REPLACE_AS_BOOLEAN when REPLACE_TABLES or
REPLACE_COLS is also used on the same object. Thanks to Brendan
Le Ny for the report.
- Fix unwanted data export of materialized view. Thanks to Michael
Vitale for the report. Fix ORA-00942 with table that are not yet
physically created and has no data.
- Fix calling functions with same name declared in several packages.
The one declared in current package now takes precedence. Thanks
to Eric Delanoe for the report.
- Change zero-length lob/long to undef workaround for a bug in
DBD::Oracle with the ora_piece_lob option (used when no_lob_locator
is enabled) where null values fetch as empty string for certain
types. Thanks to Alice Maz for the patch.
- Fix createPoint() spatial method issue. Thanks to jwl920919.
- Fix comment on REPLACE_COLS directive.
- Fix an other issue in transformation of TYPE x IS REF CURSOR.
Thanks to Pavel Stehule for the report.
- Fix an other case of broken declaration in triggers related to FOR
cycle control variables. Thanks to Pavel Stehule for the report.
- Fix broken declaration in triggers related to FOR cycle control
variables with empty DECLARE section.
- Fix other case of replacement by EXIT WHEN NOT FOUND.
- Fix output of global_variables.conf file when OUTPUT_DIR is not
set. Fix non replacement of global variables.
- Remove some AUTHID garbage in procedure declaration generated by
a migration.
- Fix trigger name quoting with non supported character. Thanks to
Michael Vitale for the report.
- Fix use of nextval() in default value.
- Fix alias append in from clause of the extract() function. Thanks
to Narayanamoorthys for the report.
- Disable direct export to partition for PostgreSQL 10 if directive
PG_SUPPORTS_PARTITION is enabled, import must be done in the main
table.
- Do not export partitions of a materialized view. Thanks to Michael
Vitale for the report.
- Fix wrong replacement of keyword after END. Thanks to Pavel
Stehule for the report.
- Remove Oracle hints from queries, they are not supported and can
break comments. Thanks to Pavel Stehule for the report.
- Fix unstable transformation of TYPE x IS REF CURSOR. Thanks to
Pavel Stehule for the report.
- Fix data export failure when no table match the ALLOW/EXCLUDE
filter. Thanks to threenotrump for the report.
- Add missing search_path to FKEY dedicated file. Thanks to Michael
Vitale for the report.
- Apply default oject name exclusion to synonym.
- Skip some PL/SQL translation in migration assessment mode.
- Change default type for virtual column whit round() function.
Thanks to Julien Rouhaud for the report.
- Fix bug with EXIT WHEN command. Thanks to Pavel Stehule.
- Fix an other wrong replacement of DECODE in UPDATE statement.
Thanks to Pavel Stehule for the report.
- Fix wrong replacement of DECODE. Thanks to Pavel Stehule.
- Fix unwanted replacement of INTO STRICT when this is an INSERT
statement. Thanks to Pavel Stehule for the report.
- Fix potential regexp error with special character in outer join
filters. Thanks to Adrian Boangiu for the report.
- Fix parsing of PK from file. Thanks to Julien Rouhaud.
- Fix parsing of FK from file. Thanks to Julien Rouhaud.
- Fix count of unique and primary key in TEST export. Thanks to
Liem for the report.
- Fix reserved keyword rewrite using double quote when directive
USE_RESERVED_WORDS is enabled. Thanks to Michael Vitale.
- Remove EVALUATION CONTEXT object type from migration assessment.
- Add QUEST_SL_% pattern to the list of table that must be excluded
from export.
- Fix data export when BFILE are translated as text. Thanks to
Michael Vitale for the report.
- Fix export of package when a comment is placed just before the
AS/IS keyword. Thanks to Michael Vitale for the report.
- Fix other cases of function call replacement when they are
declared in different packages and one with OUT parameter and
the other one with only IN parameters. Thanks to Eric Delanoe.
- Fix inconsistent behavior of import_all script with -h and -d
Under Linux: When -h not specified, script defaults to unix domain
sockets for psql and localhost for perl (which may error depending
on pg_hba.conf). Now defaults to more performing sockets. -d
wasn't passing DB name to some psql calls where it's necessary.
Thanks to BracketDevs for the patch.
- Fix file handle close when compression is enabled. Thanks to
Sebastian Albert for the report.
- Add information about ora2pg behavior during data export to files
when files already exists. Thanks to Michael Vitale.
- Update readme to provide tar command for bzip2 file. Thanks to
Tom Pollard for the patch
- Fix unwanted FTS_INDEXES empty file and append unaccent extension
creation if not exists. Thanks to Michael Vitale for the report.
- Fix missing explicitly declared variable for cycle in trigger.
Thanks to Pavel Stehule for the report.
- Fix global type/cursor declaration doubled in package export.
- Always translate Oracle SELECT ... INTO to SELECT ... INTO STRICT
in plpgsql as Oracle seems to always throw an exception. Thanks
to Pavel Stehule for the report.
- Fix too much semicolons on end of function. Thanks to Pavel
Stehule for the report.
- Fix ALTER TABLE to set the owner when table is a foreign table.
Thanks to Narayanamoorthys for the report.
- Fix case of untranslated procedure call when there was parenthesis
in the parameters clause. Thanks to Pavel Stehule for the report.
- Fix broken variable declaration with name containing DEFAULT.
Thanks to Pavel Stehule for the report.
- Change query ORDER BY clause to view export query.
- Fix missing replacement of quote_reserved_words function by new
quote_object_name. Thanks to Sebastian Albert for the report.
- Fix REPLACE_COLS replacement of column name in UNIQUE constraint
definition. Thanks to Bernard Bielecki for the report.
- Fix export of Oracle unlogged table that was exported as normal
tables.
- Fix regression in package function calls rewrite leading to append
unwanted comma when replacing out parameters. Thanks to Pavel
Stehule and Eric Delanoe for the report.
- Fix removing of function name after END keyword. Thanks to Pavel
Stehule for the report.
- Fix bug in package function extraction.
- Improve VIEW export by only looking for package function name and
fix a bug that was including unwanted "system" package definition.
Also fix a potential bad rewriting of function call. Thanks to
Eric Delanoe for the report.
- Fix an other case of missing PERFORM replacement. Thanks to Pavel
Stehule for the report.
- Fix remplacement of "EXIT WHEN cursor%NOTFOUND". Thanks to Pavel
Stehule for the report.
- Fix missing conversion of type in cast function. Thanks to Michael
Vitale for the report.
- Fix TO_NUMBER that is now translated as a cast to numeric to
correspond to the default behavior in Oracle. Thanks to Pavel
Stehule for the report.
- Fix replacement of function call from different schema, especially
in overloaded cases.
- Remove OUT parameter from the argument list of function call.
Thanks to Pavel Stehule for the report.
- Fix wrong replacement in FOR ... IN loop inserting EXCLUDE in the
statement. Thanks to Pavel Stehule for the report.
- Translate Oracle proprietary VALUES without parenthesis with the
proprietary syntax of POstgreSQL. Thanks to Pavel Stehule for the
report.
- Fix function header translation when a comment is present between
closing parenthesis and the IS keyword. Thanks to Pavel Stehule
for the report.
- Fix RETURNS in autonomous transaction call when there is OUT
parameters. Thanks to Pavel Stehule for the report.
- Fix call to BMS_UTILITY.compile_schema() when COMPILE_SCHEMA is
enable. Thanks to PAvel Stehule for the report.
- Fix export of function and procedure with same name in different
schema. Thanks to Pavel Stehule for the report.
- Fix detection and replacement of global variable in package that
was producing invalid code export. Fix progress bar on procedure
export.
- Fix regression in global variables default value export.
- Rewrite multiprocess for procedure and function export to solve
some performances issues.
- Do not waste time trying to replace function call when it is not
found in the current code.
- Fix default value for FILE_PER_FUNCTION when parallel mode is
enabled.
- Output a fatal error with export type TABLE and multiple schema set
to PG_SCHEMA when EXPORT_SCHEMA is enabled.
- Fix replacement of function name with package prefix.
- Fix documentation of PG_SCHEMA directive, especially on the use of
a schema list. Thanks to Michael Vitale for the report.
- Fix translation of INSTR() function.
- Improve speed in function translation by not calling twice
Ora2Pg::PLSQL::convert_plsql_code() on declare and code section.
Thanks to Pavel Stehule for the profiling.
- Fix unwanted replacement of SYSDATE, SYSTIMESTAMP and some other
when they are part of variable or object name. Add rewrite of
REF CURSOR during type translation.
- Require support of LATERAL keyword for DATADIFF (Pg >= 9.3).
Patch from Sebastian Albert.
- Do not call replace_sdo_function(), replace_sdo_operator() and
replace_sys_context() if the string SDO_ or SYSCONTEXT is not
found. This might save some performances.
- Remove the RETURNS clause when there is an OUT parameter
PostgreSQL choose correct type by self. Thanks to Pavel Stehule
for the report.
- Add a note about performance improvement by updating stats on
Oracle. Thanks to Michael Vitale for the report.
- Remove newline characters in REVOKE statement when embedded in
a comment. Thanks to Pavel Stehule for the report.
- Fix replacement with PERFORM into package extracted from an
Oracle database. Thanks to Eric Delanoe for the report.
- Fix translation of call to function with out parameters.
Thanks to Pavel Stehule for the report.
- Fix case where call to procedure without parameter was not
prefixed by PERFORM or when called in a exception statement.
Thanks to Eric Delanoe for the report.
- Add function quote_object_name to handle all cases where object
name need to be double quoted (PRESERVE_CASE to 1, PostgreSQL
keyword, digit in front or digit only and non supported character.
Thanks to liemdt1811 for the report.
- Add a note about RAW(n) column with "SYS_GUID()" as default value
that is automatically translated to type of the column 'uuid'
by Ora2Pg.
- Remove old column count check to use char_length. Thanks to
Alice Maz for the patch.
- Fix some raise_application_error that was not replaced with a
global rewrite of remove comments and text constants to solve
some other issues like rewriting of package function call in
dynamic queries. Thanks to PAvel Stehule for the report.
- Fix cycle variable not generated for LOOP IN SELECT in trigger.
Thanks to Pavel Stehule for the report.
- Fix procedures with OUT parameters not processed in triggers.
Thanks to Pavel Stehule for the report.
- Remove other case where PERFORM must be or must not be inserted.
- Remove case where PERFORM can be inserted. Thanks to Pavel
Stehule and Eric Delanoe for the report.
- Fix missing ; in some raise_application_error translation. Thanks
to Pavel Stehule for the report.
- Fix missing PERFORM in front of direct call to function and the
rewrite of direct call to function with out parameters. Thanks
to Eric Delanoe for the report.
- Fix translation of rownum when the value is not a number. Thanks
to Pavel Stehule for the report.
- Fix missing space between cast and AS keyword. Thanks to Pavel
Stehule for the report.
- Fix translation of views and add support to comment inside views.
Thanks to Pavel Stehule for the report.
- Fix removing of AS after END keyword. Thanks to Pavel Stehule for
the report.
- Fix type in CAST clause not translated to PostgreSQL type. Thanks
to Pavel Stehule for the report.
- Treat citext type as text. Thanks to Tomasz Wrobel for the patch.
- Fix packages migration assessment that was broken with parser
rewriting on package extraction.
- Rewrite parsing of PL/SQL packages to better handle package
specification and especially types and global variables from this
section.
- Fix raise_application_error translation by removing extra boolean
parameter.
- Improve comments processing.
- Fix package function name replacement adding a dot before package
name. Thanks to Eric Delanoe for the report.
- Add collect of functions/procedures metadata when reading DDL
from file.
- Fix replacement of function prefixed with their schema name.
Thanks to Eric Delanoe for the report.
- Try to minimized comment placeholders by aggregating multiline
comments.
- Remove new line character from _get_version() output.
- Fix ENABLE_MICROSECOND test condition on NLS_TIMESTAMP_TZ_FORMAT
setting. Thanks to Didier Sterbecq for the report.
- Fix another issue with Oracle 8i and table size extraction.
- Fix query to show column information on Oracle 8i
- Fix query to look for virtual column on Oracle 8i
- Fix query to list all table by size on Oracle 8i
- Prevent ora2pg to look for external table definition in Oracle 8i.
- Fix a regression on timestamp format setting for Oracle 8i.
- Fix some regression on queries with Oracle 8i. Thanks to Didier
Sterbecq for the report.
- Add a function to collect metadata of all functions.
- Don't create empty partition index file when there's no partition.
- Fix wrong translation in OPEN ... FOR statement. Thanks to Eric
Delanoe for the report.
- Fix call of method close() on an undefined value. Thanks to Eric
Delanoe for the report.
- Fix partition data export issues introduced with previous patches.
- Fix unterminated IF / ELSIF block in subpartition export.
- Fix subpartition export. Thanks to Maurizio De Giorgi for the
report.
- Force DATA_LIMIT default value to 2000 on Windows OS instead of
10000 to try to prevent constant OOM error. Thanks to Eric Delanoe
for the report.
- Fix default partition table that was not used PREFIX_PARTITION.
Thanks to ssayyadi for the report.
- Limit datetime microsecond format to micro second (.FF6) as the
format can be FF[0..9] and PostgreSQL just have FF[0..6]
- Add to_timestamp_tz Oracle function translation. Thanks to Eric
Delanoe for the feature request.
- Fix custom data type replacement in function code. Thanks to Pavel
Stehule for the report.
- Fix non working INPUT_FILE configuration directive when action is
QUERY. Thanks to Eric Delanoe for the report.
- Fix unwanted global variable implicit declaration to handle
autonomous transaction parameters. Thanks to Eric Delanoe for the
report.
- Fix call to dblink in function with PRAGMA AUTONOMOUS_TRANSACTION
and no arguments. Thanks to Eric Delanoe for the report.
- Fix package constant translation. Thanks to Eric Delanoe.
- Fix unwanted alias on join syntax. Thanks to Eric Delanoe
for the report.
- Fix regression on dbms_output.put* translation. Thanks to Eric
Delanoe for the report.
- Fix handling of comments in statements to try to preserve them at
maximum in the outer join rewriting.
- Do not declare variable when it is an implicit range cursor, it
do not need to be declared.
- Export implicit variable in FOR ... IN ... LOOP as an integer if
it don't use a select statement and export it as a RECORD when a
statement is found. Thanks to Eric Delanoe and Pavel Stehule for
the report.
- Reduce migration assessment weight for CONNECT BY.
- Fix derived table pasted two times in from clause. Thanks to Pavel
Stehule for the report.
- Fix some other unexpected ";" in function code. Thanks to Pavel
Stehule for the report.
- Remove %ROWTYPE in return type of function. Thanks to Pavel
Stehule for the report.
- Fix doubled AND in expression when a parenthesis is in front after
rewriting. Thanks to Eric Delanoe for the report.
- Fix unexpected ";" in function after post-body END when a comment
is present. Thanks to Eric Delanoe for the report.
- Fix unexpected ";" in some function variable declaration when a
comment is at end of the declare section. Thanks to Eric Delanoe
for the report.
- Remove %ROWTYPE in function that have not been replaced with RECORD
for cursor declaration. Thanks to Eric Delanoe for the report.
- Fix removing of WHEN keyword after END. Thanks to Pavel Stehule for
the report.
- Fix missing table name with alias in from clause due to comments in
the clause. I have also merge right and left outer join translation
function into a single one, most of the code was the same.
- Fix output order of outer join. Thanks to Pavel Stehule for the
report.
- Fix untranslated outer join in nested sub query. Thanks to Pavel
Stehule for the report.
- Rewrite again the decode() translation as a single function call
for all replacement before any other translation.
- Append table filter to check constraints extraction. Thanks to
danghb for the report.
- Fix issue with parenthesis around outer join clause. Thanks to
Pavel Stehule for the report.
- Move remove_text_constant_part() and restore_text_constant_part()
function into the main module.
- Include decode() replacement in recursive function call. Thanks
to Pavel Stehule for the report.
- Prevent removing of parenthesis on a sub select. Thanks to Pavel
Stehule for the report.
- Fix missing table exclusion/inclusion in column constraint export.
Thanks to danghb for the report.
- Fix an alias issue in view parsed from file.
- Fix parsing of view from file when no semi comma is found.
- Remove FROM clause without alias from migration assessment.
- Fix order of outer join during translation. Thanks to Pavel
Stehule for the report.
- Fix case of missing alias on subquery in FROM clause. Thanks to
Pavel Stehule for the report.
- Fix missing alias replacement in nested subqueries. Thanks to
Pavel Stehule for the report.
- Fix wrong addition of aliases to using() in join clause
- Fix nested decode replacement producing invalid CASE expression.
Thanks to Pavel Stehule for the report.
- Append aliases to subqueries in the from clause that do not have
one. Thanks to Pavel Stehule for the report.
2017 02 17 - v18.1
This release fix several issues reported on outer join translation
thanks to the help of Pavel Stehule and reapply the commit on virtual
column export that was accidentally removed from v18.0. It also adds
several new features:
- Remove CHECK constraints for columns converted into boolean using
REPLACE_AS_BOOLEAN column.
- Oracle function are now marked as stable by default as they can
not modify data.
Two new configuration directives have been added:
- DATE_FUNCTION_REWRITE: by default Ora2pg rewrite add_month(),
add_year() and date_trunc() functions set it to 0 to force Ora2Pg
to not translate those functions if translated code is broken.
- GRANT_OBJECT: when exporting GRANT you can now specify a comma
separated list of objects which privileges must be exported.
Default is to export privileges for all objects. For example
set it to TABLE if you just want to export privilege on tables.
and a new command line option:
- Add -g | --grant_object command line option to ora2pg to be able
to extract privilege from the given object type. See possible values
with GRANT_OBJECT configuration directive.
Here is the complete list of changes:
- Remove empty output.sql file in current directory with direct data
import. Thanks to kuzmaka for the report.
- Fix shell replacement of $$ in function definition in Makefile.PL
embedded configuration file. Thanks to kuzmaka for the report.
- Fix shell replacement of backslash in Makefile.PL embedded
configuration file. Thanks to kuzmaka for the report.
- Add warning level to virtual column notice.
- Fix comment in where clause breaking the outer join association.
Thanks to Pavel Stehule for the report.
- Add parsing and support of virtual column from DDL file.
- Reapply commit on virtual column export that was accidentally
removed in commit d5866c9. Thanks to Alexey for the report.
- Fix mix of inner join and outer join not translated correctly.
Thanks to Pavel Stehule for the help to solve this issue.
- Fix additional comma in column DEFAULT value from DDL input file.
Thanks to Cynthia Shang for the report.
- Fix comments inside FROM clause breaking translation to ANSI outer
joins. Thanks to Pavel Stehule for the report.
- Fix replacement of sdo_geometry type into function. Thanks to
Saber Chaabane for the report.
- Fix subquery in outer join clause. Thanks to Saber Chaabane for
the report.
- Fix duplicated subqueries placeholder in the from clause.
Thanks to Saber Chaabane for the report.
- Fix replacement of subquery place older during outer join rewrite.
Thanks to Saber Chaabane for the report.
- Add DATE_FUNCTION_REWRITE configuration directive. By default
Ora2pg rewrite add_month(), add_year() and date_trunc() functions
set it to 0 to force Ora2Pg to not translate those functions if
translated code is broken. Thanks to Pavel Stehule for the feature
request.
- Do not report error when -g is used but action is not GRANT.
Thanks to Shane Jimmerson for the report.
- Oracle function can not modify data, only procedure can do that,
so mark them as stable. Thanks to Pavel Stehule for the report.
- Missed some obvious combination like upper/lower case or no space
after AND/OR on outer join parsing and some other issues.
- Add missing call to extract_subqueries() recursively. Thanks to
Pavel Stehule for the report.
- Add full support of outer join translation in sub queries.
- Add translation of mixed inner join and Oracle outer join. Thanks
to Pavel Stehule for the report.
- Fix missing space between keyword AS and END from the decode()
transformation. Thanks to Pavel Stehule for the report.
- Fix parsing of outer join with UNION and translation to left join.
Thanks to Pavel Stehule for the report.
- Remove CHECK constraints for columns converted into boolean using
REPLACE_AS_BOOLEAN column. Thanks to Shane Jimmerson for the
feature request.
- Fix regression on SQL and PLSQL rewrite when a text constant
contained a semi-comma.
- Add the GRANT_OBJECT configuration directive. When exporting GRANT
you can specify a comma separated list of objects for which the
privileges will be exported. Default is export for all objects.
Here are the possibles values TABLE, VIEW, MATERIALIZED VIEW,
SEQUENCE, PROCEDURE, FUNCTION, PACKAGE BODY, TYPE, SYNONYM and
DIRECTORY. Only one object type is allowed at a time. For example
set it to TABLE if you just want to export privilege on tables.
You can use the -g option to overwrite it.
When used this directive prevent the export of users unless it is
set to USER. In this case only users definitions are exported.
- Add the -g | --grant_object command line option to ora2pg to be able
to extract privilege from the given object type. See possible values
with GRANT_OBJECT configuration directive.
- Improve replacement of ROWNUM by LIMIT+OFFSET clause.
- Fix extra semi-colon at end of statement.
- Override ora2pg.spec with Devrim's one but with String::Random
removing as it is no more used.
2017 01 29 - v18.0
This new major release adds several new useful features and lot of
improvements.
* Automatic rewrite of simple form of (+) outer join Oracle's
syntax. This major feature makes Ora2Pg become the first free
tool that is able to rewrite automatically (+) outer join in
command line mode. This works with simple form of outer join
but this is a beginning.
* Add export of Oracle's virtual column using a real column and
a trigger.
* Allow conversion of RAW/CHAR/VARCHAR2 type with precision in
DATA_TYPE directive. Useful for example to transform all RAW(32)
or VARCHAR2(32) columns into PostgreSQL special type uuid.
* Add export NOT VALIDATED state from Oracle foreign keys and check
constraints into NOT VALID constraints in PostgreSQL.
* Replace call to SYS_GUID() with uuid_generate_v4() by default.
* Add "CREATE EXTENSION IF NOT EXISTS dblink;" before an autonomous
transaction or "CREATE EXTENSION IF NOT EXISTS pg_background;".
* Major rewrite of the way Ora2Pg parse PL/SQL to rewrite function
calls and other PL/SQL to plpgsql replacement. There should not
be any limitation in rewriting when a function contains a sub
query or an other function call inside his parameters.
* Refactoring of ora2pg to not requires any dependency other than
the Perl DBI module by default. All DBD drivers are now optionals
and ora2pg will expect an Oracle DDL file as input by default.
* Add export of Oracle's global variables defined in package. They
are exported as user defined custom variables and available in
a session. If the variable is a constant or have a default value
assigned at declaration, ora2pg will create a new file with the
declaration (global_variables.conf) to be included in the main
configuration file postgresql.conf file.
* Create full text search configuration when USE_UNACCENT directive
is enabled using the auto detected stemmer or the one defined in
FTS_CONFIG. For example:
CREATE TEXT SEARCH CONFIGURATION fr (COPY = french);
ALTER TEXT SEARCH CONFIGURATION fr ALTER MAPPING FOR
hword, hword_part, word WITH unaccent, french_stem;
CREATE INDEX place_notes_cidx ON places
USING gin(to_tsvector('fr', place_notes));
Changes and incompatibilities from previous release:
* FTS_INDEX_ONLY is now enabled by default because the addition of
a column is not always possible and not always necessary where a
simple function-based index is enough.
* Remove use to setweigth() on single column FTS based indexes.
* Change default behaviour of Ora2Pg in Full Text Search index
export.
A new command line option and some configuration directive have
been added:
* Option -D | --data_type to allow custom data type replacement
at command line like in configuration file with DATA_TYPE.
* UUID_FUNCTION to be able to redefined the function called to
replace SYS_GUID() Oracle function. Default to uuid_generate_v4.
* REWRITE_OUTER_JOIN to be able to disable the rewriting of Oracle
native syntax (+) into OUTER JOIN if rewritten code is broken.
* USE_UNACCENT and USE_LOWER_UNACCENT configuration directives to
use the unaccent extension with pg_trgm with the FTS indexes.
* FTS_INDEX_ONLY, by default Ora2Pg creates an extra tsvector column
with a dedicated triggers for FTS indexes. Enable this directive
if you just want a function-based index like:
CREATE INDEX ON t_document USING
gin(to_tsvector('pg_catalog.english', title));
* FTS_CONFIG, use this directive to force the text search stemmer
used with the to_tsvector() function. Default is to auto detect
the Oracle FTS stemmer. For example, setting FTS_CONFIG to
pg_catalog.english or pg_catalog.french will override the auto
detected stemmer.
There's also lot fixes of issues reported by users from the past two
months, here is the complete list of changes:
- Fix return type in function with a single inout parameter and a
returned type.
- Prevent wrong rewrite of empty as null when a function is used.
Thanks to Pavel Stehule for the report.
- Add the UUID_FUNCTION configuration directive. By default Ora2Pg
will convert call to SYS_GUID() Oracle function with a call to
uuid_generate_v4 from uuid-ossp extension. You can redefined it
to use the gen_random_uuid function from pgcrypto extension by
changing the function name. Default to uuid_generate_v4. Thanks
to sjimmerson for the feature request.
- Add rewrite of queries with simple form of left outer join syntax
(+) into the ansi form.
- Add new command line option -D | --data_type to allow custom data
type replacement at command line like in configuration file with
DATA_TYPE.
- Fix type in ROWNUM replacement expression. Thanks to Pavel Stehule
for the report.
- Add replacement of SYS_GUID by uuid_generate_v4 and allow custom
rewriting of RAW type. Thanks to Nicolas Martin for the report.
- Fix missing WHERE clause in ROWNUM replacement with previous patch
thanks to Pavel Stehule for the report.
- Fix ROWNUM replacement when e sub select is used. Thanks to Pavel
Stehule for the report.
- Fix wrong syntax in index creation with DROP_INDEXES enabled.
Thanks to Pave Stehule for the report.
- Remove replacement of substr() by substring() as PostgreSQL have
the substr() function too. Thanks to Pavel Stehule for the report.
- Move LIMIT replacement for ROWNUM to the end of the query. Thanks
to Pavel Stehule for the report.
- Fix text default value between parenthesis in table declaration.
Thanks to Pavel Stehule for the report.
- Fix return type when a function have IN/OUT parameter. Thanks to
Pavel Stehule for the report.
- Mark uuid type to be exported as text. Thanks to sjimmerson for
the report.
- Add EXECUTE to open cursor with like "OPEN var1 FOR var2;". Thanks
to Pavel Stehule for the report.
- Fix replacement of local type ref cursor. Thanks to Pavel Stehule
for the report.
- Add EXECUTE keyword to OPEN CURSOR ... FOR with dynamic query.
Thanks to Pavel Stehule for the report.
- Fix case sensitivity issue in FOR .. IN variable declaration
replacement. Thanks to Pavel Stehule for the report.
- Fix wrong replacement of cast syntax ::. Thanks to Pavel Stehule
for the report.
- Reactivate numeric cast in call to round(value,n).
- Close main output data file at end of export.
- Add virtual column state in column info report, first stage to
export those columns as columns with associated trigger.
- Fix unwanted replacement of REGEXP_INSTR. Thanks to Bernard
Bielecki for the report.
- Allow rewrite of NUMBER(*, 0) into bigint or other type instead
numeric(38), just set DATA_TYPE to NUMBER(*\,0):bigint. Thanks to
kuzmaka for the feature request.
- Export partitions indexes into PARTITION_INDEXES_....sql separate
file named. Thanks to Nicolas Martin for the feature request.
- Fix fatal error when schema CTXSYS does not exists. Thanks to
Bernard Bielecki for the report.
- Fix missing text value replacement. Thanks to Bernard Bielecki
for the report.
- Fix type replacement in declare section when the keyword END was
present into a variable name.
- Export NOT VALIDATED Oracle foreign key and check constraint as
NOT VALID in PostgreSQL. Thanks to Alexey for the feature request.
- Add object matching of regex 'SYS_.*\$' to the default exclusion
list.
- Fix UTF8 output to file as the open pragma "use open ':utf8';"
doesn't works in a global context. binmode(':encoding(...)') is
used on each file descriptor for data output.
- Improve parsing of tables/indexes/constraints/tablespaces DDL from
file.
- Improve parsing of sequences DDL from file.
- Improve parsing of user defined types DDL from file.
- Export Oracle's TYPE REF CURSOR with a warning as not supported.
- Replace call to plsql_to_plpgsql() in Ora2Pg.pm by a call to new
function convert_plsql_code().
- Move export of constraints after indexes to be able to use USING
index in constraint creation without error complaining that index
does not exists.
- Add "CREATE EXTENSION IF NOT EXISTS dblink;" before an autonomous
transaction or "CREATE EXTENSION IF NOT EXISTS pg_background;".
- Improve parsing of packages DDL from file.
- When a variable in "FOR varname IN" statement is not found in the
DECLARE bloc, Ora2Pg will automatically add the variable to this
bloc declared as a RECORD. Thanks to Pavel Stehule for the report.
- Major rewrite of the way Ora2Pg parse PL/SQL to rewrite function
calls and other PL/SQL to plpgsql replacement. There should not
be limitation in rewriting when a function contains a sub query
or an other function call inside his parameters.
- Fix unwanted SELECT to PERFORM transformation inside literal
strings. Thanks to Pavel Stehule for the report.
- Fix bug in DEFAULT value rewriting. Thanks to Pavel Stehule for
the report.
- Fix replacement of DBMS_OUTPUT.put_line with RAISE NOTICE.
- Reset global variable storage for each package.
- Improve comment parsing in packages and prevent possible infinite
loop in global variable replacement.
- Add the REWRITE_OUTER_JOIN configuration directive to be able to
disable the rewriting of Oracle native syntax (+) into OUTER JOIN
if it is broken. Default is to try to rewrite simple form of
right outer join for the moment.
- Export types and cursors declared as global objects in package
spec header into the main output file for package export. Types
and cursors declared into the package body are exported into the
output file of the first function declared in this package.
- Globals variables declared into the package spec header are now
identified and replaced into the package code with the call to
user defined custom variable. It works just like globals variables
declared into the package body.
- Add auto detection of Oracle FTS stemmer and disable FTS_CONFIG
configuration directive per default. When FTS_CONFIG is set its
value will overwrite the auto detected value.
- Create full text search configuration when USE_UNACCENT directive
is enabled using the auto detected stemmer or the one defined in
FTS_CONFIG. For example:
CREATE TEXT SEARCH CONFIGURATION fr (COPY = french);
ALTER TEXT SEARCH CONFIGURATION fr ALTER MAPPING FOR
hword, hword_part, word WITH unaccent, french_stem;
CREATE INDEX place_notes_cidx ON places
USING gin(to_tsvector('fr', place_notes));
- Remove CONTAINS(ABOUT()) from the migration assessment, there no
additional difficulty to CONTAINS rewrite.
- Add ANYDATA to the migration assessment keyword to detect.
- Allow conversion of CHAR/VARCHAR2 type with precision in DATA_TYPE
directive. For example it's possible to transform all VARCHAR2(32)
columns only into PostgreSQL special type uuid by setting:
DATA_TYPE VARCHAR2(32):uuid
Thanks to sjimmerson for the feature request.
- Update year in copyrights
- Fix creation of schema when CREATE_SCHEMA+PG_SCHEMA are defined.
- Fix renaming of temporary file when exporting partitions.
- Move MODIFY_TYPE to the type section
- Update documentation about globals variables.
- Add export of Oracle's global variables defined in package. They
are exported as user defined custom variables and available in
a session. Oracle variables assignment are exported as call to:
PERFORM set_config('pkgname.varname', value, false);
Use of these variable in the code is replaced by:
current_setting('pkgname.varname')::global_variables_type;
the variable type is extracted from the pacjkage definition. If
the variable is a constant or have a default value assigned at
declaration, ora2pg will create file global_variables.conf with
the definition to include in postgresql.conf file so that their
values will already be set at database connection. Note that the
value can always modified by the user so you can not have exactly
a constant.
- Fix migration assessment of view.
- Remove call to FROM SYS.DUAL, only FROM DUAL was replaced.
- Replace call to trim into btrim.
- Improve rewrite of DECODE when there is function call inside.
- Add function replace_right_outer_join() to rewrite Oracle (+)
right outer join.
- Improve view migration assessment.
- Create a FTS section in the configuration file dedicated to FTS
control.
- Add USE_UNACCENT and USE_LOWER_UNACCENT configuration directives
to use the unaccent extension with pg_trgm.
- Do not create FTS_INDEXES_* file when there is no Oracle Text
indexes.
- Update query test score when CONTAINS, SCORE, FUZZY, ABOUT, NEAR
keyword are found.
- Remove use to setweigth() on single column FTS based indexes.
Thanks to Adrien Nayrat for the report.
- Update documentation on FTS_INDEX_ONLY with full explanation on
the Ora2Pg transformation.
- Refactoring ora2pg to not requires any dependency other than the
Perl DBI module by default. All DBD drivers are now optionals and
ora2pg will expect to received an Oracle DDL file as input by
default. This makes easiest packaging or for any distribution that
can not build a package because of the DBD::Oracle requirement.
DBD::Oracle, DBD::MySQL and DBD::Pg are still required if you want
Ora2Pg to migrate your database "on-line" but they are optional
because Ora2Pg can also convert input DDL file, this is the
default now. Thanks to Gustavo Panizzo for the feature request and
the work on Debian packaging.
- Remove String::Random dependency in rpm spec file, it is no used
even if it was mentioned into a comment.
- Exclude internal Oracle Streams AQ JMS types from the export.
Thanks to Joanna Xu for the report.
- Fix some other spelling issues. Thanks to Gustavo Panizzo for the
patch.
- Fix some spelling errors. Thanks to Gustavo Panizzo for the patch.
- Revert patch 697f09d that was breaking encoding with input file
(-i). Thanks to Gary Evans for the report.
- Add two new configuration directive to control FTS settings,
FTS_INDEX_ONLY and FTS_CONFIG.
2016 11 17 - v17.6
This release adds several new features:
* Adds export of Oracle Text Indexes into FTS or pg_trgm
based indexes,
* Add export of indexes defined on materialized views
* Allow export of materialized views as foreign tables
when export type is FDW.
* Add replacement of trim() by btrim().
Two new configuration directives have been added:
* USE_INDEX_OPCLASS: when value is set to 1, this will force
Ora2Pg to export all indexes defined on varchar2() and char()
columns using *_pattern_ops operators. If you set it to a value
greater than 1 it will only change indexes on columns where the
character limit is greater or equal than this value.
* CONTEXT_AS_TRGM: when enabled it forces Ora2Pg to translate
Oracle Text indexes into PostgreSQL indexes using pg_trgm
extension. Default is to translate CONTEXT indexes into FTS
indexes and CTXCAT indexes using pg_trgm. Some time using
pg_trgm based indexes is enough.
There's also some fixes of issues reported by users, here is the
complete list of changes:
- Fixed non-use of custom temp_dir (-T). Thanks to Sebastian
Albert for the patch.
- Make export of FTS indexes from materialized view work as
for tables.
- Fix drop of indexes during export of data when DROP_INDEXES
is enabled.
- Remove double quote in function and procedure name from an input
file to avoid creating a file with double quote in its name.
- Fix export of unique index associated to a primary key.
- Move OPTION (key "yes") of FDW table before NOT NUL constraint
and default clause.
- Fix some encoding issue during data export into file.
- Rename FTS indexes prefix output file into FTS_INDEXES and
export CTXCAT Oracle indexes as GIN pg_trgm indexes instead of
FTS indexes.
- Add export of indexes of type CTXCAT as FTS indexes.
- Export triggers and update order for FTS indexes to separate file
prefixed with FTS_INDEXES.
- Exclude from export synonyms starting with a slash that correspond
to incomplete deleted synonyms. Thanks to Nouredine Mallem for the
report.
- Add export of indexes defined on materialized views. Thanks to
Nouredine Mallem for the report.
- Fix export of foreign key and FTS indexes when looking at dba_*
tables and multiple different schemas have the same fk or context
indexes definition. Thanks to Nouredine Mallemfor the patch.
- Fix export of CONTEXT or FULLTEXT Oracle index into PostgreSQL
FTS with trigger and initial update statement.
- Add configuration directive USE_INDEX_OPCLASS to force Ora2Pg to
export all indexes defined on varchar2() and char() columns using
those operators. A value greater than 1 will only change indexes
on columns where the character limit is greater or equal than
this value.
- Fix FDW export of mysql tables. Thanks to yafeishi for the report.
- Fix decode() rewrite. Thanks to Jean-Yves Julliot for the report.
- Fix regression introduced into the export of NUMBER to integer
like PG types.
- Show partition name in progress bar instead of main table name.
2016 10 20 - v17.5
This is a maintenance release to fix several issues reported by users.
There is also some major improvement and new feature.
There is a new configuration directive or change default behavior:
* Fix export of CLOBs and NCLOB that was truncated to 64 Kb.
* PG_BACKGROUND : when enabled autonomous transactions will be
built using Robert Haas extension pg_background instead of dblink.
Default is to still used dblink as pg_background is available
only for PostgreSQL >= 9.5.
* All Perl I/O now use the open pragma instead of calling method
binmode(). This will force input and output to utf8 using the
Perl pragma:
use open ':encoding(utf8)';
when configuration directive BINMODE is not set or NLS_LANG is
set to UTF8.
* Ora2Pg will now export empty lob as empty string instead of NULL
when the source column has NOT NULL constraint and that directive
EMPTY_LOB_NULL is not activated.
* Improve and fix progress bar especially when using JOBS/-J option.
* Allow LOAD action to apply all settings defined in the input file
on each opened session, this allow to use LOAD with export schema
enabled. If settings are not set in the input file encoding and
search_path is set from the ora2pg configuration settings.
* NUMBER(*,0) is now exported as numeric(38) as well as a NUMBER
with DATA_SCALE set to 0, no DATA_PRECISION and a DATA_LENGTH
of 22. The last correspond to Oracle type INTEGER or INT.
* Allow conversion of type with precision in DATA_TYPE directive.
For example it is possible to transform all NUMBER(12,2) only
into numeric(12,2) by escaping the comma. Example:
DATA_TYPE NUMBER(12\,2):numeric(12\,2);...
* Write data exported into temporary files (prefixed by tmp_) and
renamed them at end of the export to be able to detect incomplete
export and override it at next export.
* Add export of type created in package declaration.
* Export foreign key when the referenced table is not in the
same schema.
* Enabled by default PG_SUPPORTS_CHECKOPTION assuming that your Pg
destination database is at least a 9.4 version.
* Add 12 units to migration assessment report per table/column
conflicting with a reserved word in PostgreSQL to reflect the
need of code rewriting.
* Output a warning when a column has the same name than a system
column (xmin,xmax,ctid,etc.)
* Replace SYSDATE by a call to clock_timestamp() instead of a call
to LOCALTIMESTAMP in plpgsql code.
* Add missing documentation about DISABLE_PARTITION directive used
to not reproduce partitioning into PostgreSQL and only export
partitioned data into the main table.
* Show partition name in progress bar instead of main table name.
Here is the complete list of other changes:
- Fix broken parallel table export (-P option).
- Fix export of CLOBs and NCLOB that was truncated to 64Kb. Thanks
to Paul Mzko for the patch.
- Fix database handle in error report.
- Fix use of wrong database handle to set search_path. Thanks to
Paul Mzko for the report.
- Ora2pg doesn't export schema ForeignKey constraint when connected
as different DBA user. Thanks to Paul Mzko for the patch.
- Fix Perl I/O encoding using open pragma instead of calling method
binmode(). Thanks to Gary Evans for the report.
- Force input to utf8 using Perl pragma: use open ':encoding(utf8)';
when BINMODE is not set or NLS_LANG is UTF8.
- Force ora2pg to export empty lob as empty string instead of NULL
when the source column has a NOT NULL constraint and directive
EMPTY_LOB_NULL is not activated. Thanks to Valeriy for the report.
- Fix missing CASCADE attribute on fkey creation during data export
when DROP_FKEY was enabled. Thanks to ilya makarov for the report.
- Fix issue on converting NUMBER(*,0) to numeric, should be ported
to numeric(38). Thanks to ilya makarov for the report.
- Correct query for ForeignKey export from oracle. Thanks to ilya
makarov for the patch.
- Fix schema change in direct import of data to PostgreSQL.
- Change query for foreign key extraction to keep the column order.
Thanks to ilya makarov for the report.
- Write data exported into temporary files (prefixed by tmp_) and
renamed them at end of the export to be able to detect incomplete
export and override it at next export. Thanks to Paul Mkzo for
the feature request.
- Fix infinite loop in blob extraction when error ORA-25408 occurs
during ora_lob_read() call. Thanks to Paul Mzko for the report.
- Fix order of columns in foreign keys definition. Thanks to ilya
makarov for the report.
- Fix export of partition by range on multicolumn. Thanks to Rupesh
Admane for the report.
- Update reserved keywords list. Thanks to Nicolas Gollet for the
report.
- Add ON DELETE NO ACTION on foreign key creation (DROP_FKEY) to
obtain the same output than during constraints export.
- Fix export of foreign key that was duplicating the columns in both
part, local and foreign. Thanks to ilya makarov for the report.
- Remove call to to_char(datecol, format) when exporting date and
timestamp. This formating is no more needed as we are now forcing
NLS_DATE_FORMAT and NLS_TIMESTAMP_FORMAT when opening a connection
to Oracle using:
ALTER SESSION SET NLS_DATE_FORMAT='YYYY-MM-DD HH24:MI:SS
and
ALTER SESSION SET NLS_TIMESTAMP_FORMAT='YYYY-MM-DD HH24:MI:SS
This may result on some speed improvment during data export.
- Fix parsing of packages from input file.
- Add export of type created in package declaration. Thanks to
dezdechado for the report.
- Fix converting of procedures with out arguments. Thanks to
dezdechado for the report.
- Update documentation about project management.
- Fix replacement of = NULL by IS NULL in update statement.
Thanks to dezdechado for the report.
- Fix parsing of trigger from file that was broken and new line
removed. Thanks to dezdechado for the report.
- Fix erasing of quotes from text in triggers. Thanks to dezdechado
for the report.
- Fix "return new" on trigger function when there is exception.
Thanks to dezdechado for the report and solution.
- Fix conversion of INTEGER and INT into numeric(38) instead of
numeric without precision. Thanks to dezdechado for the report.
- Fix export of foreign key when the referenced table is not in the
same schema. Thanks to Juju for the report.
- Fix ddl create schema when EXPORT_SCHEMA and CREATE_SCHEMA are
enabled but no schema is specified.
- Fix export of NCHAR that was converted as char but was loosing its
length definition. Thanks to lgerlandsen for the report.
- Fix parsing of views using WITH statements. Thank to dezdechado
for the report.
- Fix case that breaks views/triggers definition when a semicolon
is encountered in a string value when reading definition from
file. Thanks to dezdechado for the report.
- Fix included/excluded of sequences when using ALLOW/EXCLUDE
directives. Thanks to Roman Sindelar for the report.
- prepare options modified with some escaping improvements. Thanks
to ioxgrey for the patch.
- It seems that for a NUMBER with a DATA_SCALE set to 0, no
DATA_PRECISION and a DATA_LENGTH of 22 in ALL_TAB_COLUMNS, Oracle
use a NUMBER(38) instead. This correspond to Oracle type INTEGER
or INT. I don't really understand the reason of this behavior,
why not just using a data length of 38? ALL_TAB_COLUMNS and Ora2Pg
reports a data length of 22, now Ora2Pg will report NUMBER(38) like
the resulting type of the DESC command.
The following Oracle table:
CREATE TABLE TEST_TABLE ( FIELD_1 INTEGER, FIELD_2 NUMBER );
will be exported as follow by Ora2Pg:
[1] TABLE TEST_TABLE (owner: HR, 0 rows)
FIELD_1 : NUMBER(38) => numeric
FIELD_2 : NUMBER(22) => bigint
Oracle data type INTEGER and INT will be exported as numeric by
default instead of an integer.
- Fix parsing of function/procedure from file with comments after
BEGIN statement.
- Remove DEFERRABLE INITIALLY DEFERRED from CHECK constraints when
parsed from file. Thanks to Felipe Lavoura for the report.
- Fix double parenthesis in index definition when parsing index
creation from file. Thanks to Felipe Lavoura for the report.
- Fix parsing of COMMENT from file.
- Fix undetected native Oracle type bug. Thanks to kvnema for the
report.
- Fix unwanted text formatting with bind value in INSERT action
with direct import to PostgreSQL. Thanks to Oleg for the report.
- Fix inversion of UPDATE_RULE and DELETE_RULE in foreign key
creation for MySQL export. Thanks to Sebastian Albert for the
report.
- Update documentation about DEFER_FKEY and DROP_FKEY to report
the new behavior.
- Remove call to SET CONSTRAINTS ALL DEFERRED with direct import.
- Fix use of NULL value in bind parameter that should be undefined
(INSERT export mode only). Thanks to Oleg barabaka for the report.
- Remove replacement of direct call to functions with PERFORM, there
is too much false positive. Thanks to dezdechado for the reports.
- Fix a typo in SYSDATE replacement. Thank to dezdechado for report.
- Remove rewrite of concatenation in RAISE EXCEPTION. Thanks to
dezdechado for the report.
- Fix replacement of raise_application_error() when first argument
is a variable. Thanks to dezdechado for the report.
- Fix wrong insertion of PERFORM before some function calls. Thanks
to dezdechado for the report.
- Replace SYSDATE by a call to clock_timestamp() instead of call to
LOCALTIMESTAMP in plpgsql code. Thanks to aleksaan for the report.
- Allow use of comma for object name list separator instead of space
as workaround on Window OS.
- Fix documentation about MODIFY_TYPE. Thanks to Nicolas Gollet for
the report.
- Add missing documentation about DISABLE_PARTITION directive used
to not reproduce partitioning into PostgreSQL and only export
partitioned data into the main table. Thanks to Nicolas Gollet
for the report.
- Add information about how to export LONG RAW data type. They need
to be exported as BLOB before into Oracle to be exported as BYTEA.
- Fix case where select was wrongly replaced by perform in INSERT
INTO with SELECT statement. Thanks to dezdechado for the report.
- Fix links to ora2pg presentation. Thanks to Daniel Lenski for the
patch.
- Fix input parameters after one with a default value must also have
defaults. Thanks to v.agapov fot the patch.
- Fix debug mode that was interromping the last running table dump.
Thanks to calbiston for the report.
2016 04 21 - v17.4
Errata in first release attempt.
- Fix previous patch that does not handle blob case but just clob
- Forgot to change back the query when EMPTY_LOB_NULL is not activated.
- Put parenthesis around AT TIME ZONE expression
This is a maintenance release to fix several issues reported by users.
There is also some major data export speed improvement thanks to the
work of PostgreSQL Pro and a new RPM spec file provided by Devrim
Gunduz to be able to build RPM package for Ora2Pg.
There is a new configuration directive:
- EMPTY_LOB_NULL: when enabled force empty_clob() and empty_blob()
to be exported as NULL instead as empty string.
Here is the complete list of other changes:
- Add EMPTY_LOB_NULL directive to force empty_clob() and empty_blob()
to be exported as NULL instead as empty string. This might improve
data export speed if you have lot of empty lob. Thanks to Alex
Ignatov for the report.
- Fix import_all.sh script to import grant and tablespace separately
as postgres user and just after indexes and constraints creation.
- Add parsing of tablespace from "alter table ... add constraint"
with DDL input file. Thanks to Felipe Lavoura.
- Remove --single-transaction in import_all.sh script with TABLESPACE
import. Thanks to Guillaume Lelarge for the report.
- Fix Makefile.PL to used with latest spec file from Devrim Gunduz
and following the advice of calbiston.
- Update spec file to v17.6 and latest change to Makefile.PL
- Replace ora2pg.spec by postgressql.org spec file by Devrim Gunduz.
- Generate man page to avoids rpmbuild error.
- Fix Windows install. Thanks to Lorena Figueredo for the report.
- Remove "deferrability" call for mysql foreign keys. Thanks to
Jean-Eric Cuendet for the report.
- Fix issue in restoring foreign key for mysql data export. Thanks
to Jean-Eric Cuendet for the report.
- Remove connection test to PostgreSQL instance as postgres or any
superuser in import_all.sh
- Fix creation of configuration directory.
- Fix Makefile to dissociate CONFDIR and DOCDIR from PREFIX or
DESTDIR. Thanks to Stephane Schildknecht for the report.
- Fix date_trunc+add_month replacement issue. Thanks to Lorena
Figueredo for the report.
- Do not replace configuration directory in scripts/ora2pg if this
is a RPM build. Thanks to calbiston for the report.
- Return empty bytea when a LOB is empty and not NULL.
- Regular expressions and conditions checks improvement in method
format_data_type() to make it a bit faster on huge tables. Thanks
to Svetlana Shorina for the patch.
- Fix INSERT using on the fly data import with boolean values.
Thanks to jecuendet for the report.
- Allow MySQL data type to be converted into boolean. Thanks to
jecuendet for the report.
- Fix export of BIT mysql data type into bit bit varying. Thanks
to jecuendet for the report.
- Fix call to escape_copy/escape_insert function call.
2016 03 26 - v17.3
This release fix two regressions introduced in latest release.
* Fix major bug in data export. Thanks to Frederic Guiet for the report.
* Fix another regression with character data that was not escaped. Thanks
to Frederic Guiet for the report.
2016 03 24 - v17.2
This is a maintenance release to fix several issues reported in new
LOB extraction method. There is also some feature improvement:
* Allow NUMBER with precision to be replaced as boolean.
* Allow full relocation of Ora2Pg installation using for
example: perl Makefile.PL DESTDIR=/opt/ora2pg
Here is the complete list of other changes:
- Allow NUMBER with precision to be replaced as boolean. Thanks
to Silvan Auer for the report.
- Force empty LOB to be exported as NULL when NO_LOB_LOCATOR is
activated to have the same behavior.
- Fix case where a LOB is NULL and ora2pg reports error :
DBD::Oracle::db::ora_lob_read: locator is not of type OCILobLocatorPtr
LOB initialised with EMPTY_CLOB() are also exported as NULL
instead of \\x
- Fix replacement with PERFORM after MINUS. Thanks to Stephane
Tachoires for the report.
- Comment DBMS_OUTPUT.ENABLE calls. Thanks to Stephane Tachoire for
the report.
- Fix wrong replacement of SELECT by PERFORM after EXCEPT. Thanks
to Stephane Tachoire for the report.
- Apply ORACLE_COPIES automatic predicate on custom queries set with
REPLACE_QUERY if possible. Thanks to pawelbs for the report.
- Fix install of ora2pg.conf file in /etc/ instead of /etc/ora2pg/.
Thanks to pawelbs for the report.
- Add debug information before searching for custom type.
- Attempt to fix error "ORA-01002: fetch out of sequence" when exporting
data from a table with user defined types and ORACLE_COPIES. Thanks to
pawelbs and Alex Ignatov fir the report.
- Fix replacement of path to configuration file in scripts/ora2pg
- Remove report sample from documentation about migration assessment
report and replace it with a href link. Fix comment about export of
domain index.
- Always prefix table name with schema in Oracle row count, to prevent
failure when the schema is not the connexion default.
- Add pattern TOAD_PLAN_.* to the internal table exclusion list.
- Fix modification of database owner search_path in import_all.sh auto
generated script. Thanks to Stephane Tachoire for the report.
2016 02 29 - v17.1
This is a maintenance release to fix several issues reported in new
TEST action. There is also some feature improvement:
* Add OPTIONS (key 'true') on table FDW export when a column is
detected as a primary key.
* Add DELETE configuration directive that allow a similar feature
than the WHERE clause to replace TRUNCATE call by a "DELETE FROM
table WHERE condition". This feature can be useful with regular
"updates". Thanks to Sebastien Albert for the feature request.
Here is the complete list of other changes:
- Fix the counter of user defined types and sequences in TEST action
- Fix COPY import of data from column with user defined type with
NULL value.
- Fix DBD::Pg segmentation fault with direct INSERT import from
column with user defined type.
- Fix TEST action with multiple PG_SCHEMA export. Thanks to Michael
Vitale for the report.
- Fix documentation about PG_SCHEMA
2016 02 22 - v17.0
This new major release adds a new action type TEST to obtain a count
of all objects at both sides, Oracle and PostgreSQL, to perform a
diff between the two database and verify that everything have been
well imported. It also fixes several issues reported by users.
A new ora2pg command line option have been added to ora2pg script:
* Add --count_rows command line option to perform a real row count
on both side, Oracle and PostgreSQL, in TEST report.
Here is the complete list of changes and bugfixes:
- Prefix direct call to function with a call to PERFORM. Thanks to
Michael Vitale for the feature request.
- Fix revoke call on function with multiline parameters declaration.
- Fix auto setting of internal schema variable with mysql.
- Define ORACLE_HOME with the corresponding environment variable in
generic configuration when available and --init_project is used.
Thanks to Stephane Tachoires for the report.
- Fix documentation about exporting view as table.
- Remove some obsolete code and display information when a view is
exported as table.
- Fix empty LOB data export with Oracle Lob locator (NO_LOB_LOCATOR
set to 0).
- Fix data export of partitions with single process mode and when
FILE_PER_TABLE is enabled.
- Fix export of RAW data type.
- Fix missing $ to call to self variable. Thanks to NTLIS and Sirko
for the report.
- Force FKey to be initially immediate when deferred is not set.
Thanks to Stephane Tachoire for the report.
- Fix count of check constraint when a schema is forced.
- Allow TEST action on mysql database too with some improvements
and bug fix on the feature.
- Fix index column renaming in mysql export.
- Fix dblink extraction query when an exclusion is set.
- Fix sequence name auto generation for mysql serial number.
- Add --count_rows command line option to make optional the real
row count in TEST report. This is useful when you have lot of
data and do not want to loose time in call to count(*).
- Update documentation about the TEST action and usage, see
chapter "Test the migration".
- Apply schema context on PostgreSQL side with TEST action.
- Add TEST action type to ask Ora2Pg to count rows and all objects
at both sides, Oracle and PostgreSQL, to verify that everything
have been well imported.
- Fix missing export of foreign keys on multiple columns, ex:
ALTER TABLE products ADD CONSTRAINT fk_supplier_comp
FOREIGN KEY (supplier_id,supplier_name)
REFERENCES supplier(supplier_id,supplier_name)...
- Fix import of BLOB data using INSERT statements into the bytea.
Thanks to rballer for the patch.
- Fix missing export of FK when no schema is provided.
2016 01 13 - v16.2
This release fixes several issues, is more accurates on migration
assessment report and adds some new ora2pg command line options:
* Add --pg_dsn, --pg_user and --pg_pwd to be able to set the
connection to import directly into PostgreSQL at command line.
* Add -f option to script import_all.sh to force to not check
user/database existing and skip their creation.
Potential backward compatibility issues:
* PG_SUPPORTS_CHECKOPTION is now enabled by default, you may want
to migrate to PostgreSQL 9.4 or above.
* Remove modification of CLIENT_ENCODING in generic configuration
file with --init_project, use the default instead.
* Remove modification of directive NLS_LANG to AMERICAN_AMERICA.UTF8
in generic configuration file with --init_project, use the default
instead.
Here is the complete list of other changes:
- Adjust DBMS_OUTPUT calls to the migration assessment count.
- Fix migration assessment count of call to cursor%ISOPEN and
cursor%ROWCOUNT.
- Replace zero date also with prepared statement with online
PostgreSQL import and INSERT action. Thanks to Sebastian Albert
for the report.
- Remove REFERENCING clause in conditional triggers. Thanks to Raqua
for the report.
- Fix position of TG_OP condition when an exception is defined.
Thanks to Raqua for the report.
- Fix wrong replacement of SELECT with PERFORM when a comment was
found between an open parenthesis and the select statement.
Thanks to Raqua for the report.
- Fix procedure return type with OUT and INOUT parameter. Thanks to
Raqua for the report.
- Fix rewrite of triggers with referencing clause. Thanks to Raqua
for the report.
- Fix default number of --human_days_limit in usage.
- Fix replacement of placeholder %TEXTVALUE-d% to hide text string
in query during function call rewrite. Thanks to Lorena Figueredo
for the report.
- Fix progress bar when a WHERE clause is used to limit the number
of row to export.
- Fix error "DBD::Pg::db do failed: SSL error: decryption failed or
bad record mac" with pararellel table export (-P) and direct
import to PostgreSQL via a ssl connection. Thanks to pbe-axelor
for the report.
- Fix missing index name in indexes creation. Thanks to Raqua for
the report.
- Fix pg DSN in import_all.sh autogenerated script.
- Fix extraction of trigger. When the name of a column or something
contained INSERTING, DELETING or UPDATING was converted to TG_OP
= 'INSERT' or corresponding event. Thanks to Stanislaw Jankowski
for the patch.
- Fix multiple use of same column in check constraint and indexes
of partitions when there was several schema with the same objects.
- Fix default value for HUMAN_DAY_LIMIT to 5 when it is not defined
in ora2pg.conf.
- Fix double quote on column name in COPY export of partition tables
Thanks to Chris Brick for the report.
- Prevent case with several time same column in multicolumns unique
constraints. Fix typo in previous patch.
- Fix double quoted name with auto incremented sequence exported as
serial.
- Fix syntax error with MySQL data export with a WHERE clause using
LIMIT.
2015 11 30 - v16.1
This release fixes several issues and adds some very useful features:
* Generate automatically a new import_all.sh shell script when using option
--init_project to help automate all import into PostgreSQL.
See sh import_all.sh -? for more information.
* Export Oracle bitmap index as PostgreSQL btree_gin index. This require the
btree_gin extension and PostgreSQL >= 9.4. This is the default.
* Auto set DEFINED_PK to the first column of a table that have a unique key
defined that is a NUMBER. This allow data of any table with a numeric
unique key to be extracted using multiple connexions to Oracle using -J
option. Tables with no numeric unique key will be exported with a single
process.
* Improve BLOB export speed by using hex encoding instead of escape. This
might speed up be BLOB export by 10.
* Allow use of LOB locator to retrieve BLOB and CLOB data to prevent having
to set LONGREADLEN. Now LONGREADLEN is set to 8KB. Old behavior using
LONGREADLEN can still be enabled by setting NO_LOB_LOCATOR to 0, given
for backward compatibility. Default is to use LOB locator.
* Ora2Pg will also auto detect table with BLOB and automatically decrease
DATA_LIMIT to a value lower or equal to 1000. This is to prevent OOM.
* Improving indexes and constraints creation speed by using the LOAD action
and a file containing SQL orders to perform. It is possible to dispatch
those orders over multiple PostgreSQL connections. To be able to use this
feature, PG_DSN, PG_USER and PG_PWD must be set. Then:
ora2pg -t LOAD -c config/ora2pg.conf -i schema/tables/INDEXES_table.sql -j 4
will dispatch indexes creation over 4 simultaneous PostgreSQL connections.
This will considerably accelerate this part of the migration process with
huge data size.
* Domain indexes are now exported as b-tree but commented to let you know
where possible FTS are required.
* Add number of refresh ON COMMIT materialized view in detailed report.
* Allow redefinition of numeric type, ex: NUMBER(3)::bigint to fix wrong
original definition in Oracle.
* Allow export of all schemas from an Oracle Instance when SCHEMA directive
is empty and EXPORT_SCHEMA is enabled. All exported objects will be
prefixed with the name of their original Oracle schema or search_path will
be set to that schema name. Thanks to Magnus Hagander for the feature
request.
* Allow use of COPY FREEZE to export data when COPY_FREEZE is enabled. This
will only works with export to file and when -J or ORACLE_COPIES is not
set or default to 1. It can be used with direct import into PostgreSQL
under the same condition but -j or JOBS must be unset or default to 1.
Thanks to Magnus Hagander for the feature request.
Some new configuration directives:
* BITMAP_AS_GIN: enable it to use btree_gin extension to create bitmap
like index with pg >= 9.4. You will need to create the extension by
yourself: "create extension btree_gin;". Default is to create GIN index,
when disabled, a btree index will be created.
* NO_LOB_LOCATOR: to disable use of LOB locator and extract BLOB "inline"
using a less or more high value in LONGREADLEN.
* BLOB_LIMIT: to force the value of DATA_LIMIT for tables with BLOB. Default
is to automatically set this limit using the following code:
BLOB_LIMIT=DATA_LIMIT; while (BLOB_LIMIT > 1000) BLOB_LIMIT /= 10
* COPY_FREEZE: use it to use COPY FREEZE instead of simple COPY to speedup
import into PostgreSQL.
Here is the complete list of other changes:
- Limite package function name rewrite to call with parenthesis after the
function name to avoid rewriting names of other objects.
- Fix extra replacement of function name with package prefix. On some
condition it was done multiple time.
- Set REPLACE_ZERO_DATE to -INFINITY in generic configuration when --mysql
is enabled.
- Fix extraction of partition with MySQL that was not limited to a single
database.
- Do some replacement on ORACLE_DNS and SCHEMA into generic configuration
when --mysql is used for better understanding.
- Add call to round() on -J parallelization when the auto detected column
is a numeric with scale.
- Add COMMIT to the difficulties migration assessment keywords as it need
context analyzing.
- Add call to cursor's %ROWCOUNT, %ISOPEN and %NOTFOUND to difficulties
migration assessment keywords.
- Replace call to CURSOR%ROWTYPE by RECORD. Thanks to Marc Cousin for the
report.
- Fix ALTER FUNCTION ... OWNER TO ... and REVOKE statement where functions
parameters were missing.
- Add Get_Env to the Oracle functions list for migration assessment.
- Disable variable NO_LOB_LOCATOR and set LONGREADLEN to 8192 bytes to use
LOB locators to extract BLOB in generic configuration file.
- Fix call method "disconnect" on unblessed reference at line 9998. Thanks
to Stephane Tachoires for the report.
- Exclude from export objects name matching /.*\$JAVA\$.*/ and /^PROF\$.*/.
- Fix migration assessment report when created during the package export.
- Force writing Oracle package body in separate files when FILE_PER_FUNCTION
is enabled and PLSQL_PGSQL disable to obtain package source code.
- Fix case where sequence max value is lower than start value, in this case,
set max value = start value.
- Fix missing newline after each package file to import in global package.sql
file when FILE_PER_FUNCTION is enabled.
- Remove export of user PUBLIC in GRANT export.
- Set DISABLE_TRIGGERS to 1 in generic configuration file auto generated when
ora2pg option --init_project is used.
- Remove call to quote_reserved_words() with index column when it we detect
a function based index, too much false positive are rewritten with SQL code
like CASE...WHEN.
- Update export_schema.sh to remove .sql files when there is not such object
leaving export directory empty.
- Prevent creating TBSP_INDEXES_tablespace.sql when no tablespaces are found
- Update documentation on WHERE clause on how to limit the number of tuples
exported for Oracle and MySQL to test data import.
- Fix unlisted spatial indexes in assessment report.
- Fix double quote on index name with index renaming and reserved keyword.
- Do not try to export tablespaces, privileges and audited queries as non DBA
user when USER_GRANT is enabled.
- Remove carriage return from list file.
- Force SCHEMA to database name with MySQL migration.
- Fix missing declaration of _extract_sequence_info(). Thank to Yannick DEVOS
for the report.
- Add documentation about COPY_FREEZE directive and add a note about export
of all schema.
- Remove systematic schema name appended to table name on KETTLE export, this
must only be true when EXPORT_SCHEMA is enabled.
- Fix TO_NUMBER() wrong replacement when a function is called as a parameter.
- Fix non converted DECODE() when they was called in an XMLELEMENT function.
- Suppress MDSYS.SDO_* from MDSYS call in migration assessment cost.
- Remove use of DBMS_STANDARD called with raise_application_error function
- Fix STRING type replacement
- Recreate README as a text file, not a man page.
- Reformat changelog to 80 characters.
- Add -t | --test command line option to ora2pg_scanner to be able to test
all connections defined into the CVS list file.
2015 10 15 - v16.0
This major release improve PL/SQL code replacement, fixes several bugs and
adds some major new features:
* Full migration of MySQL database, it just work like with Oracle database.
* Full migration assessment report for MySQL database.
* New script, ora2pg_scanner, to perform a migration assessment of all
Oracle and MySQL instances on a network.
* Add technical difficulty level in migration assessment.
* Allow migration assessment on client queries extracted from AUDIT_TRAIL
(oracle) or general_log table (mysql).
* Ora2Pg has a "made in one night" brand new Web site (still need some work).
See http://ora2pg.darold.net/
Example of technical difficulty level assessment output for the sakila database
with some more difficulties:
Total 83.90 cost migration units means approximatively 1 man-day(s).
Migration level: B-5
Here are the explanation of the migration level code:
Migration levels:
A - Migration that might be run automatically
B - Migration with code rewrite and a human-days cost up to 10 days
C - Migration with code rewrite and a human-days cost above 10 days
Technical levels:
1 = trivial: no stored functions and no triggers
2 = easy: no stored functions but with triggers, no manual rewriting
3 = simple: stored functions and/or triggers, no manual rewriting
4 = manual: no stored functions but with triggers or views with code
rewriting
5 = difficult: stored functions and/or triggers with code rewriting
This is to help you to find the database that can be migrated first with small
efforts (A and B) and those who need to conduct a full migration project (C).
This release has also some new useful features:
* Export type SHOW_TABLE now shows additional information about table type
(FOREIGN, EXTERNAL or PARTITIONED with the number of partition).
* Connection's user and password can be passed through environment variables
ORA2PG_USER and ORA2PG_PASSWD to avoid setting them at ora2pg command line.
* Improve PL/SQL replacement on ADD_MONTH(), ADD_YEAR(), TRUNC(), INSTR() and
remove the replacement limitation on DECODE().
* Add detection of migration difficulties in views, was previously reserved
to functions, procedures, packages and triggers.
* Replace values in auto generated configuration file from command line
options -s, -n, -u and -p when --init_project is used.
* Adjust lot of scores following new functionalities in Ora2Pg, ex: dblink or
synomyms are now easy to migrate.
There is some new command line options to ora2pg script:
* -m | --mysql : to be used with --init_project and -i option to inform
ora2pg that we work with a MySQL format
* -T | --temp_dir : option to be able to set a distinct temporary directory
to run ora2pg in parallel.
* --audit_user : option to set the user used in audit filter and enable
migration assessment report on queries from AUDIT_TRAIL (oracle) or
general_log table (mysql).
* --dump_as_sheet and --print_header options to be able to compute a CSV
file with all migration assessment from a list of oracle database.
* --dump_as_csv option to report assessments into a csv file. It will not
include comments or details, just objects names, numbers and cost.
Backward compatibility:
- Change NULL_EQUAL_EMPTY to be disabled by default to force change in the
application instead of transforming the PL/SQL.
This release adds some new configuration directives:
* MYSQL_PIPES_AS_CONCAT: Enable it if double pipe and double ampersand
(|| and &&) should not be taken as equivalent to OR and AND.
* MYSQL_INTERNAL_EXTRACT_FORMAT: Enable it if you want EXTRACT() replacement
to use the internal format returned as an integer.
* AUDIT_USER: Set the comma separated list of user name that must be used
to filter from the DBA_AUDIT_TRAIL or general_log tables.
* REPLACE_ZERO_DATE: "zero" date: 0000-00-00 00:00:00 it is replaced by a
NULL by default, use it to use the date of your choice.
* INDEXES_RENAMING: force renaming of all indexes using tablename_columnsname
Very useful for database that have multiple time the same index name or
that use the same name than a table.
* HUMAN_DAYS_LIMIT: default to 5 days, used to set the number of human-days
limit for migration of type C.
Here is the full list of other changes:
- Remove list of acknowledgment that was not maintained anymore and some
person may feel injured. Acknowledgment for patches or bug reports are
always written to changelog, so this part reports to it now.
- Fix bad trigger export when objects was enclosed in double quote and fix
an additional bug in WHEN clause export. Thanks to Cyrille for the report.
- Update documentation.
- Update Makefile.PL with new script to install and new configuration
directives in auto generated configuration file.
- Update with new and missing files.
- Add a Perl Module dedicated to MySQL database object discovery and export,
lib/Ora2Pg/MySQL.pm.
- Fix function based index type replacement in previous commit.
- Do not report indexes with just DESC as function based index like Oracle
report it. Thanks to Marc Cousin for the report.
- Some excluded table was missing in the previous patch.
- Remove use of DBI InactiveDestroy call when a fork is done and replace it
to a single use AutoInactiveDestroy at connection. This require DBI>=1.614.
- Add SDO_* tables and OLS_DIR_BUSINESSES in table exclusion list to fix issue
#124 when no schema is provided. Thanks to Kenny Joseph for the report.
- Fix partition prefix.
- Remove UNIQUE keyword from spatial index.
- Fix alter triggers function with missing parenthesis. Thanks to Spike Hodge
MWEB for the report.
- Fix export of foreign keys when they was defined in lowercase. Thanks to
Spike for the report.
- Fix wrong offset when rewriting ROWNUM with LIMIT+OFFSET. Thanks to Marc
Cousin for the report.
- Allow -INFINITY to be used to replace zero date.
- Migration assessment in hour-day are now set to 1 man-day, we do not need
such a precision and it is easier to process csv report. Thanks to Stephane
Tachoire for the report.
- Fix some issue with FDW and WKT spatial export. Add migration assessment
of queries from the AUDIT_TRAIL table.
- Adjust assessment units of some objects and add QUERY migration weight.
- Rewrite information about migration levels.
- Fix speedometer in progress bar, it will now shows the current speed in
tuples/sec and the speed and time related to a table when export ended for
the object. Thanks to Alex Ignatov for the report.
- Fix break line when export data using INSERT mode. Thanks to Vu Bui for
the report.
- Do not display line about non existent objects in migration assessment
reports.
- Fix date default value for date when value is 0000-00-00 00:00:00
- Suppress display of title for function and trigger details when there is
no details.
- Remove INSTR() from the list of Oracle function that are not supported.
It is now replaced by position().
- Fix condition to call _get_largest_tables().
- Fix some minor issues in OUT/INOUT type returned by a function.
- Fix default value that may appears unquoted.
- Fix several issues on partition export: column with function, index on
default partition table and plsql to plpgsql translation in check condition.
- Fix some minor issues.
- Replace values from command line options -s, -n, -u and -p in --init_project
auto generated configuration file. Thanks to Stephane Tachoire for the
feature request.
- Fix wrong object count in SHOW_REPORT. Thanks to Stephane Tachoire for
the report.
- Use DBA_SEGMENTS to find database size when USER_GRANT is disable, aka user
is a DBA
- Remove report of Migration Level when --estimate_cost is not enabled.
- Add missing BINARY_INTEGER for type replacement.
- Always exclude function squirrel_get_error_offset() that is created by the
Squirrel Oracle plug-in.
- Adjust assessment scores following new functionalities in Ora2Pg, ex:
autonomous transaction, dblink or synomyms are now easy to migrate.
- Remove man page from source, it is auto generated by Makefile.PL and make.
- Fix unterminated DECODE replacement when there was more than 5 parameters
to DECODE() and remove the limitation to 10 parameters. There is no more
limit in the number of decode parameters. Thanks to Mael Rimbault for the
report.
- Remove inclusion of unwanted object when exporting a limited list of view
with ALLOW.
- Disable unsupported recursive query used to reorder views when Oracle
version is 11gR1. Thanks to Mael Rimbault for the patch.
- Add PLPGSQL replacement of INSTR() by POSITION(). Thanks to Mael Rimbault
for the report.
- Add difficulty level information in migration assessment, this include a
new configuration directive HUMAN_DAYS_LIMIT (default to 5 days) to set
the number of human-days limit for migration of type C.
- Add MERGE with a migration cost of 3, still need work be replaced by
ON CONFLICT.
- Remove some redundant regular expressions.
- Fix escaped commas not working properly in MODIFY_TYPE. A MODIFY_TYPE
value like `TABLE1:COL4:decimal(9\,6)` was leading to a column like
`col4 decimal(8#nosep#3)` in the SQL dump file that was generated. This
fixes the output to be `col4 decimal(8,3)`. Thanks to Nick Muerdter for
the patch.
- Strip default "empty_clob()" values from table dumps. This function does
not exist in Postgres and is not necessary. Thanks to Nick Muerdter for
the patch.
- Fix undesired double quoting of column name in function based indexes.
- Fix issue with Perl < 5.8 "Modification of a read-only value attempted"
- Fix retrieving of table size on Oracle 8i.
- Add auto double quoting of object name with unauthorized characters.
Thanks to Magnus Hagander for the feature request.
- Automatically double quote object name beginning with a number
- Fix missing DESC part in descending indexes. Thanks to Magnus Hagander
for the report.
- Fix case where a column name in oracle is just a number (e.g. the column
is called "7"), it will be created in postgres without quoted identifier,
which fails. Thanks to Magnus Hagander for the report.
- Fix "reqs/sec" display in debug mode. Thanks to Laurent Martelli for
the patch
- Fix export if Oracle procedure is created without a parameter. Thanks to
dirkgently007 for the report.
- Fix CSV report output.
- Fix triggers from file parser.
- Add a test on triggers return to handle case where it is triggered on
DELETE + other(s) event(s). In this case a test is done on the TG_OP to
return OLD if event is DELETE or NEW in other case. Thanks to Dominique
Legendre for the suggestion.
- Change NULL_EQUAL_EMPTY to be disabled by default to force change of the
application instead of transforming the PL/SQL.
- Change score of SYNONYM and DBLINK in the migration assessment.
- Add conversion of Oracle type STRING into varchar(n) or text.
- Add information about libaio1 requirement for instant client
- Remove extra space when calling ora2pg_get_efile() used to export BFILE
into EFILE. Thanks to Dominique Legendre for the export.
2015 06 01 - v15.3
This is a maintenance release only that fixes several minor bugs and typos.
The configuration file have been entirely rewritten to classify configuration
directives in section for better understanding.
Here is the full list of changes:
- Ora2Pg will use DEFAULT_SRID when call to sdo_cs.map_oracle_srid_to_epsg()
returns an ORA-01741 error. Mostly because there's no SRID defined for that
column in ALL_SDO_GEOM_METADATA. The error message will still be displayed
but a warning will explain the reason and ora2pg will continue with default
value. Thanks to kazam for the report.
- Add current setting for NLS_TIMESTAMP_FORMAT and NLS_DATE_FORMAT to the
SHOW_ENCODING report.
- Change default value for GEOMETRY_EXTRACT_TYPE to INTERNAL instead of WKT.
- Change generic configuration file behavior with BINMODE parameter commented
if it was previously uncommented. This will force to use the default value.
- Fix potential issue with max open file limit with unclosed temporary file.
Thanks to Marc Clement for the report.
- Fix use of SECURITY DEFINER in SYNONYM export.
- Fix parsing of editable function/procedure/package from input DML file.
- Fix case where variable $2 and $3 was null after a too early call of a new
substitution in read_view_from_file(). Thanks to Alex Ignatov for the patch.
- Add support to "create or replace editionable|noneditionable" from input DML
files. Thanks to Alex Ignatov for the report.
- Fix unknown column HIGH_VALUE from *_TAB_PARTITIONS in Oracle 8i. Thanks to
Sebastian Fischer for the patch.
- Fix call to ALL_MVIEW_LOGS object which not exists with Oracle 8i. Thanks to
Sebastian Fischer for the report.
- Fix Error ORA-22905 while Get the dimension of the geometry by looking at
number of element in the SDO_DIM_ARRAY. Thanks to jkutterer for the patch.
- Remove reordering export of view for Oracle database before 11g. Thanks to
kyiannis for the report.
- Fix several some typos and a bunch of misspelled. Thanks to Euler Taveira
for all the patches.
- Fix missing Oracle database version before looking at function security
definer. Thanks to kyiannis for the report.
2015 04 13 - v15.2
This new minor release fixes some issues and adds two new configuration
directives:
* ORA_INITIAL_COMMAND to be able to execute a custom command just after
the connection to Oracle, for example to unlock a security policy.
* INTERNAL_DATE_MAX to change the behavior of Ora2Pg with internal date
found in user defined types.
This version will also automatically re-order exported views taking into
account interdependencies.
Here is the full list of changes:
- Add INTERNAL_DATE_MAX configuration directive with default to 49 to be
used when reformatting internal date returned with a user defined type
and a timestamp column. DBD::Oracle only return the internal date format
01-JAN-77 12.00.00.000000 AM so it is difficult to know if the year value
must be added to 2000 or 1900. We takes the default behavior where date
are between 1950 and 2049.
- Remove extra CHAR and BYTE information from column type. Thanks to Magnus
Hagander for the report.
- Re-order views taking into account interdependencies. Thanks to Kuppusamy
Ravindran and Ulrike for the suggestion and the Oracle query.
- Fix case sensitivity in function based indexes. Thanks to Kuppusamy
Ravindran for the report.
- Fix PERFORM wrong replacement and infinite loop processing DECODE in some
condition. Thanks to Didier Brugat for the report.
- Fix replacement of boolean value in DEFAULT value at table creation.
Thanks to baul87 for the report.
- Add ORA_INITIAL_COMMAND configuration directive to be able to execute a
custom command just after the connection to Oracle, to unlock a policy for
example. Thanks to Didier BRUGAT for the feature request.
- Fix alias in from clause when an XML type is found. Thanks to Lance Jacob
for the record.
- Invert condition on excluding temporary file with Windows OS. Thanks to
kazam for the report.
- Remove start time and global number of rows from _dump_table() parameters
they are not used anymore.
- Remove use of temporary file on Windows operating system.
- Disable parallel table export when operating system is Windows.
- Fix export of objects with case sensitivity using ALLOW or EXCLUDE
directives. Thanks to Alexey Ignatov for the report.
- Fix export of triggers from recycle bin.
- Fix count of synonym in assessment report.
- Add list of tables created by OEM to the exclusion list.
- Fix look at default configuration file and set mode of export_schema.sh
to executable by default. Thanks to Kuppusamy Ravindran for the report.
- Add AUTHORIZATION to the list of PostgreSQL reserved word. Thanks to
Kuppusamy Ravindran for the report.
- Display a warning when an index has the same name as the table itself so
that you can renamed it before export. Thanks to Kuppusamy Ravindran for
the feature request.
- Fix export of function based indexes with multiple column. Thanks to
Kuppusamy Ravindran for the report.
- Modify ora2pg script to return 0 on success, 1 on any fatal error and 2
when a child process die is detected.
- Change the way the generic configuration file is handle during project
initialization. You can use -c option to copy your own into the project
directory. If the file has the .dist extension, ora2pg will apply the
generic configuration on it. Thanks to Kuppusamy Ravindran for the report
and features request.
- Add debug information when cloning the Oracle connection.
- Force return of OLD when the trigger is on DELETE event
2015 02 06 - v15.1
New minor release just to fix two annoying bugs in previous release.
- Fix replacement of function name which include SELECT in their name by
PERFORM. Thanks to Frederic Bamiere for the report.
- Fix creation of sources subdirectories when initializing a new migration project.
2015 02 04 - v15.0
This major release improve PL/SQL code replacement, fixes several bugs and
adds some new useful features:
- Add support to the PostgreSQL external_file extension to mimic BFILE
type from Oracle. See https://github.com/darold/external_file for
more information.
- Allow export of Oracle's DIRECTORY as external_file extension objects
This will also try to export read/write privilege on those directories.
- Allow export of Oracle's DATABASE LINK as Oracle foreign data wrapper
server using oracle_fdw.
- Allow function with PRAGMA AUTONOMOUS_TRANSACTION to be exported through
a dblink wrapper to achieve the autonomous transaction.
- Allow export of Oracle's SYNONYMS as views. Views can use foreign table
to create "synonym" on object of a remote database.
- Add trimming of data when DATA_TYPE is used to convert CHAR(n) Oracle
column into varchar(n) or text. Default is to trim both side any space
character. This behavior can be controlled using two new configuration
directives TRIM_TYPE and TRIM_CHAR.
- Add auto detection of geometry constraint type and dimensions through
spatial index parameters. This avoid the overhead of sequential scan
of the geometric column.
- Add support to export Oracle sub partition and create sub partition
for PostgreSQL with the corresponding trigger.
- ALLOW and EXCLUDE directives are now able to apply filter on the object
type. Backward compatibility can not be fully preserved, older definition
will apply to current export type only, this could change your export in
some conditions. See documentation update for more explanation.
- Add PACKAGE_AS_SCHEMA directive to change default behavior that use a
schema to emulate Oracle package function call. When disable, all calls
to package_name.function_name() will be turn into package_name_function_name()
just like a function call in current schema.
- Add FKEY_OPTIONS to force foreign keys options. List of supported options
are: ON DELETE|UPDATE CASCADE|RESTRICT|NO ACTION.
- Add rewriting of internal functions in package body, those functions will
be prefixed by the package name. Thanks to Dominique Legendre for the
feature request.
Some change can break backward compatibility and make configuration directives
obsolete:
- The ALLOW_PARTITION configuration directive has been removed. With new
extended filters in ALLOW/EXCLUDE directive, this one is obsolete.
Backward compatibility is preserved but may be removed in the future.
- ALLOW and EXCLUDE directives do not works as previously. Backward
compatibility may be preserved with some export type but may be broken
in most of them. See documentation.
- It is recommended now to leave the NLS_LANG and CLIENT_ENCODING commented
to let Ora2Pg handle automatically the encoding. Those directives may be
removed in the future.
Here is the full changelog of the release:
- Declares SYNONYM views as SECURITY DEFINER to be able to grant access to
objects in other schema.
- Fix wrong replacement of data type in function body. Thanks to Dominique
Legendre for the report.
- Fix missing column name replacement on trigger export when REPLACE_COLS
is defined. Thanks to Dominique Legendre for the report.
- Fix missing table replacement on trigger export when REPLACE_TABLES is
defined. Thanks to Dominique Legendre for the report.
- Fix case where IS NULL substitution was not working. Thanks to Dominique
Legendre for the report.
- Remove double exclusion clause when multiple export type is used with same
column name and no values defined.
- Allow parsing of DATABASE LINK and SYNONYM from a DDL file.
- Add DIRECTORY export type to export all Oracle directories as entries for
the external_file extension. This will also export read/write privilege
on those directories. Thanks to Dominique Legendre for the feature request.
- Review documentation about NULL_EQUAL_EMPTY.
- Fix missing code to replace IS NULL as coalesce(...). Thanks to Dominique
Legendre for the report.
- Add external_file schema to search_path when BFILE is set to EFILE in
directive DATA_TYPE. Thanks to Dominique Legendre for the request.
- Remove IF EXIST clause to oracle function created by Ora2Pg for BFILE
export. Thanks to Dominique Legendre for the report.
- Add support to the PostgreSQL external_file extension to mimic BFILE type
from Oracle. See https://github.com/darold/external_file for more information.
- Add auto detection of geometry constraint type and dimensions through the
spatial index parameters first. This avoid the overhead of sequential scan
of the geometric column.
- Remove lookup at package function when not required.
- Fix issue with database < 10g that do not have the DROPPED column into the
ALL_TABLES view. Thanks to Lance Jacob for the report.
- Add trimming of data when DATA_TYPE is used to convert CHAR(n) Oracle
column into varchar(n) or text column into PostgreSQL. Default is to
trim both side any whitespace character. This behavior can be controlled
using the new configuration directives TRIM_TYPE and TRIM_CHAR.
- Update copyright year.
- Add assessment cost for object TABLE SUBPARTITION and review cost for
object DATABASE LINK.
- Update documentation about SYNONYM export.
- Allow export of SYNONYMS as views with a new export type: SYNONYM.
- Fix object exclusion function with Oracle 8i and 9i. Thanks to Lance Jacob
for the report.
- Fix INTERVAL YEAR TO MONTH and DAY TO SECOND with precision.
- Remove unused pragma from the cost assessment.
- Suppress PRAGMA RESTRICT_REFERENCES, PRAGMA SERIALLY_REUSABLE and INLINE
from the PLSQL code. There is no equivalent and no use in plpgsql.
- Fix several issues in function/procedure/package extraction from file
input and some other related bug.
- Remove single slash and \\r from function code.
- Remove schema from package name with input file to avoid creating
function with SCHEMA.PKGNAME.FCTNAME
- Fix ALLOW/EXCLUDE ignored with type COPY or INSERT. Thanks to thleblond
for the patch.
- Fix setting of NLS_NUMERIC_CHARACTERS and NLS_TIMESTAMP_FORMAT with
multiprocess, the session parameters was lost with the cloning of the
database handle. Thanks to thleblond for the patch.
- Fix issue that could produce errors "invalid byte sequence" when dumping
data to pg database by forcing the client_encoding when PG_DSN is set.
Thanks to thleblond for the patch.
- Fix issue to add parenthesis with function with no parameters and wrong
use of PERFORM in cursor declaration. Thanks to hdeadman for the report.
- Fix broken export of function or procedure without parameter in package
body. Thanks to hdeadman for the report.
- Fix ERROR: "stack depth limit exceeded" generated by an infinite loop in
partition trigger when there is no default table when value is out of range.
- Add support to Oracle sub partition export.
- Fix issue with procedure in package without parameters.
- Enable DISABLE_SEQUENCE in generic configuration file.
- Fix unwanted alter sequence in data export when there is table allowed
or excluded.
- Fix initial default values of command line parameter that prevent value
in configuration file to be taken.
- Fix non working global definition of table in ALLOW and EXCLUDE directive
with COPY and INSERT export.
- Update ora2pg.spec, thanks to bbuechler for the patch.
- Close temporary files before deleting them, on Windows if they are not
explicitly closed there are not deleted. Thanks to spritchard for the
patch.
- Force schema name to be uppercase when PRESERVE_CASE is disable (default).
Thanks to Jim Longwill for the report.
- Add rewriting of internal functions in package body, those functions will
be prefixed by the package name. Thanks to Dominique Legendre for the
feature request.
- Fix type replacement in user defined type. Thanks to Dominique Legendre
for the report.
- Add filter with INSTEAD OF triggers on views to TRIGGER export type. Thanks
to Dominique Legendre for the feature request.
- Fix replacement of function name when PACKAGE_AS_SCHEMA is disabled.
- Fix PLSQL_PGSQL that was always set to 0 when -p was not used even if
configuration directive PLSQL_PGSQL was activated. Thanks to Dominique
Legendre for the report.
- Remove ALTER SCHEMA ... OWNER TO ... when CREATE_SCHEMA is not enable.
Thanks to Dominique Legendre for the report.
- Add DBLINK export to be created as foreign data wrapper server. Thanks to
the BRGM for the feature request.
- Remove ALLOW_PARTITION configuration directive, with extended filter in
ALLOW/EXCLUDE directive, this one is obsolete. Backward compatibility is
preserved.
- Add documentation about extended filters in ALLOW and EXCLUDE directive.
- Update documentation about VIEW_AS_TABLE and remove statement change with
export TYPE is VIEW.
- Add filter to grant export on functions, sequences, views, etc.
- Fix GRANT in ALLOW or EXCLUDE filters.
- Add commented order: "REVOKE ALL ON FUNCTION ... FROM PUBLIC;" when the
function is declared as SECURITY DEFINER.
- Prevent collecting column information with SHOW_TABLE export type.
- Fix default value SYSTIMESTAMP to CURRENT_TIMESTAMP, and remove DEFAULT
empty_blob(). Thanks to hdeadman for the report.
- ALLOW and EXCLUDE directives are now able to apply filter on the object
type. Backward compatibility can not be fully preserved, older definition
will apply to current export type only, this could change your export in
some conditions. See documentation update for more explanation. Thanks to
the BRGM for the feature request.
- Force function to be created with SECURITY DEFINER when AUTHID in table
ALL_PROCEDURES is set to DEFINER in Oracle. This only works with Oracle
>= 10g. Thanks to Dominique Legendre for the feature request.
- Add PACKAGE_AS_SCHEMA configuration directive to change default behavior
to use a schema to emulate Oracle package function call. When disable all
call to package_name.function_name() will be turn into package_name_function_name()
just like a function call in current schema. Thanks to the BRGM for the
feature request.
- Add a note to documentation about the way to convert srid into Oracle
database instead of in Ora2Pg. Thanks to Dominique Legendre for the hint.
- Fix documentation about SHOW_ENCODING export type.
- Remove use of REGEX_LIKE with Oracle version 9. Thanks to Lance Jacob for
the report.
- Replace new FKEY_OPTIONS by FKEY_ADD_UPDATE configuration directive with
three possible values: always, never and delete. It will force or not
Ora2Pg to add "ON UPDATE CASCADE" on foreign keys declaration.
- Allow FORCE_OWNER to work with all exported objects. Thanks to BRGM for
the feature request.
- Add FKEY_OPTIONS to force foreign keys options. List of supported options
are: ON DELETE|UPDATE CASCADE|RESTRICT|NO ACTION. Thanks to the BRGM for
the feature request.
- Fix ambiguous column in view extraction. Thanks to Dominique Legendre for
the report.
- Fix replacement of TYPE:LEN by boolean, ex: REPLACE_AS_BOOLEAN CHAR:1.
Thanks to jwiechmann for the report.
- Fix error ORA-00942 where Ora2Pg try to export data from a view defined
in VIEW_AS_TABLE configuration directive.
- Update list of excluded Oracle schema to the documentation.
- Fix export of all views with comments when VIEW_AS_TABLE is set.
- Fixed some typos in the generated sample configuration file. Thanks to
Hal Deadman for the patch.
- Limit column information export to the type of object extracted.
- Remove call to MDSYS in SQL code. Thanks to Dominique Legendre for the
report.
- Add more Oracle schema to the exclusion list.
- Fully remove join on DBA_SEGMENTS to retrieve the list of tables, views
and comments. Replaced by ALL_OBJECTS. Thanks to Dominique Legendre for
the help.
- Exclude JAVA\$.* tables and fix tables list query to include newly created
tables with no segments. Thanks to Dominique Legendre for the fix.
- Fix regex that convert all x = NULL clauses to x IS NULL to not replace
:= NULL too.
- Autodetect unusual characters in owner name when extracting data and used
it embeded into double quote.
- Replace single return with return new in trigger code. Thanks to Dominique
Legendre for the report.
2014 11 12 - v14.1
This is a maintenance release only mainly to add patches that was not
been applied in previous major release.
- Remove ALLOW_CODE_BREAK, it is no more useful.
- Change output of SHOW_ENCODING to reflect change to default encoding.
- Comment ALLOW_PARTITION in default configuration file
- Add QUERY and KETTLE export type in configuration file comments.
2014 11 05 - v14.0
This major release adds full export of Oracle Locator or Spatial geometries into
PostGis, SDO_GEOM functions and SDO_OPERATOR are also translated. This export
adds the following features:
1. Basic and complex geometry types support
2. Geometry data conversion from Oracle to PostGIS
3. Spatial Index conversion
4. Geometry metadata / constraints support
5. Spatial functions conversion
For spatial data export, you have three choice, WKT to export data using
SDO_UTIL.TO_WKTGEOMETRY(), WKB to export data using SDO_UTIL.TO_WKBGEOMETRY()
and INTERNAL to export geometry using a Pure Perl library. Unlike the first
two methods, INTERNAL is fast and do not raise Out Of Memory. The export is
done in WKT format so that you can verify your geometry before importing to
PostgreSQL.
Other additional major features are:
- Parallel table processing.
- Auto generation of migration template with a complete project tree.
- Allow user defined queries to extract data from Oracle.
Parallel table processing is controlled by the -P or --parallel command line
options or the PARALLEL_TABLE configuration directive to set the number of
tables that will be processed in parallel for data extraction. The limit is
the number of cores on your machine. Ora2Pg will the open one connection to
Oracle database for each parallel table extraction. This directive, when upper
than 1, will invalidate ORACLE_COPIES but not JOBS, so the real number of
process that will be used is (PARALLEL_TABLES * JOBS).
The two options --project_base and --init_project when used indicate to Ora2Pg
to create a project template with a work tree, a generic configuration file
and a shell script to export all objects from the Oracle database. So that you
just have to define the Oracle database connection into the configuration file
and then execute the shell script called export_schema.sh to export your
Oracle database into files. Here a sample of the command and the project's tree.
ora2pg --project_base /tmp --init_project test_project
/tmp/test_project/
config/
ora2pg.conf
data/
export_schema.sh
reports/
schema/
fdws/ functions/ grants/ kettles/
mviews/ packages/ partitions/
procedures/ sequences/ tables/
tablespaces/ triggers/ types/ views/
sources/
functions/ mviews/ packages/
partitions/ procedures/ triggers/
types/ views/
It create a generic config file where you just have to define the Oracle
database connection and a shell script called export_schema.sh. The
sources/ directory will contains the Oracle code, the schema/ will
contains the code ported to PostgreSQL. The reports/ directory will
contains the html reports with the migration cost assessment.
Sometime you may want to extract data from an Oracle table but you need a
custom query for that. Not just a "SELECT * FROM table" like Ora2Pg do but
a more complex query. The new directive REPLACE_QUERY allow you to overwrite
the query used by Ora2Pg to extract data. The format is TABLENAME[SQL_QUERY].
If you have multiple table to extract by replacing the Ora2Pg query, you can
define multiple REPLACE_QUERY lines. For example:
REPLACE_QUERY EMPLOYEES[SELECT e.id,e.fisrtname,lastname FROM EMPLOYEES e
JOIN EMP_UPDT u ON (e.id=u.id AND u.cdate>'2014-08-01 00:00:00')]
Other new features are:
- Export of declaration of language C function. Previous version was
not exporting function with no code body like external C function.
- Export of COMMENT from views.
- Function to replace some call to SYS_CONTECT(USERENV, ...) by the
PostgreSQL equivalent.
- Add POSTGIS_SCHEMA configuration directive to add the dedicated
PostGis schema into the search_path.
- Add PG_SUPPORTS_IFEXISTS configuration directive to be able to suppress
IF EXISTS call in DDL statement generated by Ora2Pg.
- Triggers are now all excluded/allowed following the table names specified
in the ALLOW and EXCLUDED directives
- Allow automatic export of nested tables (TYPE+TABLE+COPY).
One change is not fully backward compatible: Ora2Pg now use UTF8 by default
on both side. On Oracle connection NLS_LANG is set to AMERICAN_AMERICA.AL32UTF8,
NLS_NCHAR to AL32UTF8. On PostgreSQL side CLIENT_ENCODING to UTF8. For export
that dump to files, Perl binmode is set to utf8. You can always change those
default setting in configuration file, but it is not recommanded.
Here is the full changelog of the release:
- Fix inline comments into function declaration. Thanks to Marcel Huber
for the report.
- Fix case where SELECT ... INTO was wrongly replaced by PERFORM.
- Fix DECODE() translation. Thanks to Dominique Legendre for the report.
- Add replacement of SDO_OPERATOR into PostGis relationships.
- Add replacement of SDO_GEOM spatial function to postgis ST_* functions.
- Add GEOMETRY_EXTRACT_TYPE configuration directive to specify the geometry
extracting mode: WKT (default), WKB and INTERNAL.
- Add a pure Perl library to export SDO_GEOMETRY as a WKT representation.
This is controlled by a new extraction type INTERNAL to use with the
GEOMETRY_EXTRACT_TYPE configuration directive.
- Remove USE_SC40_PACKAGE directive and any reference to this library,
it is not useful now that we have the INTERNAL geometry extraction mode.
- Fix replacement of varchar2 in PL/SQL function.
- Fix bug in type replacement when default values used function.
- Add export of declaration of language C function. Previous version was
not exporting function with no code body like external function.
- Fix create statement in export of view as table. Thanks to ntlis for the
report.
- Fix replacement of to_number without format.
- Add export of COMMENT from VIEWS.
- Add function to replace some call to SYS_CONTECT(USERENV, ...) by the
PostgreSQL equivalent.
- Fix parsing from file of tablespace.
- Fix wrong alias name in FROM clause when extracting XML data. Thanks
to Marc Sitges for the report.
- Fix export of comments in FDW export, might be COMMENT ON FOREIGN TABLE.
Thanks to David Fetter for the report.
- Fix broken export of function based indexes. Thanks to Floyd Brown for
the report.
- Fix sequence with negative minvalue/maxvalue and negative increment.
Thanks to jwiechmann for the report.
- Fix forced owner to schema to the value of FORCE_OWNER when it is set
to a user name.
- Fix create schema when FORCE_OWNER is enabled. Thanks to Dominique
Legendre for the report.
- Add POSTGIS_SCHEMA configuration directive to add a schema to the
search_path. Thanks to Dominique Legendre for the feature request.
- Returns NULL when a geometry is NULL instead of calling ST_AsText with
a null value. Thanks to Dominique Legendre for the report.
- Add more explanation about values of CONVERT_SID.
- Fix issue in DBMS_OUTPUT replacement.
- Fix exclusion of default objects from type export.
- When CONVERT_SRID is > 1 this value will be used to force the SRID value
on all export.
- Disable NULL_EQUAL_EMPTY in generic configuration when generating a project
tree.
- Add LOGMNR$ and RECAP$ in the exclusion objects list.
- Fix performance issue in extracting data from geometry column and add
AUDSYS,DVSYS and DVF to the list of schema to exclude.
- Prefix table name with schema name on queries for retrieving data to
avoid errors in multi schema export.
- Add SDO_* cost to migration report.
- Fix real number of Synonym that should be review.
- Fix wrong report of CTXSYS synonym.
- Enabled AUTODETECT_SPATIAL_TYPE by default.
- Remove KETTLE and FDW export from the auto generated project.
- Force the copy of /etc/ora2pg/ora2pg.conf.dist into the project directory
with no more look at the current ora2pg.conf. Force autodetection of
spatial type in the generic configuration.
- Huge performance gain on querying information about Spatial column. Thanks
to Dominique Legendre for the great help.
- Fix wrong use of table alias with SEGMENT_NAME.
- Add unified audit table (CLI_SWP$.*) from the exclusion list.
- Fix operator in check condition of range partitions. Thanks to Kaissa
Chellouche for the report.
- Add to the internal exclusion list tables generated by spatial indexes
MDRT_.*, sequences MDRS_.* and interMedia Text index DR$.*. Thanks to
Dominique Legendre for the report.
- Make REPLACE_TABLES and REPLACE_COLS work with VIEW. The view name and
the columns aliases will be replaced. Take care that the table name or
columns names in the statement will be kept untouched and need manual
rewriting. Thanks to Sven Medin for the feature request.
- Add PG_SUPPORTS_IFEXISTS configuration directive to be able to suppress
IF EXISTS call in DDL statement generated by Ora2Pg. PostgreSQL below
9.x do not support this keywords. Thanks to George Kowalski fot the
feature request.
- Fix wrong substitution in EXECUTE ... USING statement, where parameters
number was not prefixed by a $ sign. Thanks to Dominique Legendre for
the report.
- Fix document about KEEP_PKEY_NAMES that also affect unique key and not
only primary key as it was specified in the documentation. Thanks to
Dominique Legendre for the report.
- Add tables generated by statistics on spatial index (MDXT_.*) into the
internal exclusion list. This join the already excluded table generated
by partition logging (USLOG$_.*) and materialized view logs (MLOG$_.*,
RUPD$_.*)
- Add DEFAULT_SRID configuration direction to permit change of the internal
default EPSG srid 4326.
- Fix new line after search_path settings. Thanks to Dominique Legendre for
the report.
- Triggers are now all excluded/allowed following the table names specified
in the ALLOW and EXCLUDED directive, no more on there own name which had
little interest. Thanks to Dominique Legendre for the feature request.
- Add support to COPY export with Spatial objects. Thanks to Legendre
Dominique for the great help to solve this problem.
- Fix default SRID value when a NULL value is returned by Oracle, SRID 8307
and the corresponding EPSG SRID 4326.
- Update documentation on relation between PARALLEL_TABLES and FILE_PER_TABLE
- Add the -P or --parallel command line options and update documentation
about parallel table processing.
- Add PARALLEL_TABLES configuration directive to force ora2Pg to use on
process and one connection per table up to the number of CPU specified.
Thanks to menardorama for the feature request.
- Add PARALLEL_TABLES configuration directive to force ora2Pg to use on
process and one connection per table up to the number of CPU specified.
Thanks to menardorama for the feature request.
- Add --init_project and --project_base command line options to create a
migration template with a complete project tree, a generic configuration
file and script to automate export of all object in the project tree.
- Fix unwanted space before AND returned by limit_to_tables(). Thanks to
Alex Wang for the report.
- Add note about regex inclusion/exclusion not working with 8i database in
documentation
- Fix regex inclusion/exlusion of table that was not more working since the
inclusion of limit_to_tables() function. Thanks to alex wang for the patch
- Exclude dropped tables (those who are in the recycle bin) from export.
- When USER_GRANTS is disabled, aka login as dba user, force table list to
be checked against DBA_SEGMENTS with SEGMENT_TYPE of type table or table
partition. This could help solving some incomprehensible object found in
Oracle view ALL_TABLES.
- Fix query to retrieved list of tables, owner selection was set two time.
- Add support to automatic nested table export (TYPE+TABLE+COPY).
- Fix wrong export of materialized view log table. Thanks to Ronson Blossom
for the report.
- Update the SYSUSER array to exclude objects owned par those more users.
- Fix unwanted export of overflow table of an index-organized table. Thanks
to Ronson Blossom for the report.
- Update the SYSUSER array to exclude objects owned par those users.
- Display table owner in debug mode for SHOW_TABLE or SHOW_COLUMN.
- Add a section to give hint about converting Oracle outer join syntax to
ANSI. Thanks to Sven Medin for the links.
- Fix issue #82 again. Thanks to Sven Medin fro the report.
- Add first support to user defined queries to extract data from Oracle.
This feature add a new configuration directive named REPLACE_QUERY.
- Change program title when dump to file.
- Fix MODIFY_TYPE directive that was broken when using type with space
character. Thanks to Dmitry K. for the patch.
- Show missing view name in debug mode when exporting some views as table.
- Rewrite replace(a,b) with three arguments replace(a,b,'') for PostgreSQL.
Thanks to Dominique Legendre for the report.
- Convert all x <> NULL or x != NULL clauses to x IS NOT NULL. All x = NULL
are converted into x IS NULL. Thanks to Dominique Legendre for the report.
- Add warning at exit to signal when a OOM occurs. In that case, when a child
Ora2Pg process was silently killed by the OOM killer there was no information
that a failure occurs.
2014 06 02 - v13.0
This major release adds first support to export Oracle Spatial Objects to PostGis
Spatial objects. There's also a new configuration directive to allow logging of
statement failures to prevent Ora2Pg to abort and continue to load valid data.
The other main feature is the possibility to convert DDL files without needing an
Oracle database connection, until now this was reserved to files containing stored
procedures. There's also several bug fixes.
- Allow error logging during data import. This feature controlled by the
LOG_ON_ERROR directive allow you to not abort the data import process
when an error is encountered and to log to a file the COPY or INSERT
statement that generate the error. After fixing the statement you will
be able to load the missing data. Thanks to menardoram for the feature
request.
- Force export type to be INSERT when COPY is used and a table have a
GEOMETRY column. I can not find a solution to export as copy statement
for the moment. Thanks to Dominique Legendre and Vincent Picavet for
the help.
- Fix export of user defined type as object. Thanks to Shanshan Wang for
the report.
- Limit look up of objects to the ALLOW or EXCLUDE filter into the SQL
query instead of the Perl code to avoid retrieving huge list of objects
on such database. Thanks to menardorama for the feature request.
- Add support to spatial data export in INSERT mode. Still need some work
in COPY export mode if possible.
- Fix query to retrieve SRID that broken with patch on CONVERT_SRID.
- Fix wrong filter with ALLOW directive when getting list of partition.
- Add GRANT export read from an input file.
- Fix data type conversion when using input file and data type such
varchar2(10 BYTE).
- Add export of comment with TABLE and VIEW exports using an input file.
- Add extraction of TABLESPACE from an input file.
- Add support to SEQUENCE extraction from input file.
- Fix wrong filter with ALLOW directive when exporting partition. The
filter was done on partition name instead of table name, that mean
that setting ALLOW directive was resulting in no export at all. Thanks
to menardorama for the report.
- Add CONVERT_SRID configuration directive to control the automatic
conversion of SRID to standard EPSG using the Oracle SDO function
sdo_cs.map_oracle_srid_to_epsg() Oracle function. Thanks to Dominique
Legendre for the help.
- Fix a typo in the create index prefix on partitioned tables. Thanks
to menardorama for the patch.
- Fix non replacement of destination during SHOW_COLUMN and COPY export.
Using MODIFY_TYPE was only working in TABLE export.
- Force pl/sql conversion with TABLE export to replace advanced default
values. Fix code TRUNC(SYSDATE, MONTH) in default value and everywhere
that should be: date_trunc(month,LOCALTIMESTAMP). Thanks to menardorama
for the report.
- Fix code regarding unique partition index naming. Thanks to menardorama
for the report.
- Add PREFIX_PARTITION configuration directive. When enabled it will force
renaming all partition table name with the name of the parent table.
Thanks to menardoram for the feature request.
- Add AUTODETECT_SPATIAL_TYPE in configuration file and documentation
about this new directive.
- Add export of SDO_GEOMETRY column type. They are basically exported to
the non-constrained "geometry" type with SRID if defined. When the
configuration directive AUTODETECT_SPATIAL_TYPE is enable, Ora2Pg will
try to autodetect the geometry type, the dimension and the SRID used
to set a constrained geometry type. For example, in the first case
column shape with Oracle type SDO_GEOMETRY will be converted as:
shape geometry(GEOMETRY) or shape geometry(GEOMETRY, 4326)
and in the second case, with constrained geometry type:
shape geometry(POLIGONZ, 4326)
with a three dimensional polygon. Thanks to Vincent Picavet for the
feature request and specification.
- Add support to spatial index read from file.
- Add export of Oracle spatial index. For example, index:
CREATE INDEX cola_spatial_idx ON cola_markets(shape) INDEXTYPE IS MDSYS.SPATIAL_INDEX;
will be exported as
CREATE INDEX cola_spatial_idx ON cola_markets USING GIST(shape);
Thanks to Vincent Picavet / Oslandia for the feature request and explanations.
- Allow TRIGGER export to parse an input file with Oracle DML orders.
- Add PG_SUPPORTS_CHECKOPTION configuration directive to not remove
WITH CHECK OPTION in create view statement. It is supported in
PostgreSQL 9.4.
- Allow VIEW export to parse an input file with Oracle DML orders.
- Allow TABLE export to parse an input file with Oracle DML orders.
- Add SYNCHRONOUS_COMMIT configuration directive disabled by default.
This is the current behavior of Ora2Pg, it set synchronous_commit
to off before data import to PostgreSQL. This is only used when you
load data directly to PostgreSQL, the default is off to disable
synchronous commit to gain speed at writing data. Some modified or
old version of PostgreSQL, like Greenplum, do not have this setting.
- Add some useful information for Windows user in documentation. Thanks
to Roger Park for the report.
- Fix case when parentheses are omitted in index creation. Thanks to
Yuri Ushakov for the report.
- Fix export type PACKAGE when ALLOW is defined to extract only some
packages. Thanks to Maciej Bak for the report.
- Fix INSERT export where backslash should be escaped and single be
doubled in standard conforming string notation. Thanks to Yuri
Ushakov for the report.
- Add important note about LONGREADLEN and DATA_LIMIT that could need
to be adjusted to avoid out of memory. Thanks to Mike Kienenberger
for the patch.
- Fix case sensitivity issue with export of comment on column. Thanks
to Pierre Crumeyrolle for the report.
- Fix export of RAW data in COPY mode, was missing a backslash. Thanks
to jwiechmann for the report.
- Fix RAW data export in COPY and INSERT mode, RAW data type is returned
in hex by DBD::Oracle. Thanks to jwiechmann for the report.
- Fix one release 8i condition.
- Fix inexistent column USE_NO_INDEX with Oracle 8i and MVIEW export.
- Enclose call to utf8::encode and utf8::valid into eval.
- Fix export of constraint with Oracle 8i release.
- Fix unrecognized fatal error with 8i database. Thanks to UnvorherSeba
for the patch.
- Revert change level of error from fatal to error, when querying
materialized view.
- Change level of error from fatal to error, when querying materialized
view.
2014 01 28 - v12.1
This is a maintenance release with some minor bug fixes and a new configuration
directive, INDEXES_SUFFIX, to allow appending a suffix to indexes names.
- Fix example given for the WHERE configuration directive. Thanks to
Bob Treumann for the report.
- Add INDEXES_SUFFIX configuration option to allow append a suffix to
indexes names.
- Replace special charater ^M by \r as they are not supported by git.
- Fix IF EXISTS in alter table of sub _drop_foreign_keys. Thanks to
Francis Corriveau for the patch.
- Fix isolation level when exporting data. Thanks to Ludovic Penet for
the report.
- Fix regression when ora2pg tries to create foreign keys on tables or
to tables that are not selected for export. Thanks to Ludovic Penet.
- Add information about backslashed comma into directive MODIFY_TYPE
into Makefile.PL.
- Add missing MODIFY_TYPE definition in documentation.
- Allow backslashed comma into MODIFY_TYPE type redefinition. Example:
TABLE1:COL3:decimal(9\,6),TABLE1:COL4:decimal(9\,6).
Thanks to Mike Kienenberger for the report
- Fix missing single cote into create_materialized_view() call. Thanks
to Jacky Rigoreau for the patch.
- Fix some typo in documentation, thanks to Mike Kienenberger for the
report.
- Add a chapter about installing DBD::Oracle into documentation. Thanks
to Raghavendra for the patch.
- Fix case sensitivity on external table name with FDW export type.
Thanks to Guillaume Lelarge for the report.
- Fix export of materialized views when PG_SUPPORTS_MVIEW is disabled.
Thanks to Christian Bjornbak for the report.
- Update copyright.
2013 10 22 - v12.0
This release fixes lot of issues and three new features. Using REORDERING_COLUMNS
directive you will be able to reorder columns to minimized the footprint on disc,
so that more rows fit on a data page. The PG_SUPPORTS_MVIEW will allow you to
export materialized with native PostgreSQL 9.3 syntaxe. The USE_TABLESPACE variable
will allow you to export object using their original tablespace.
- Skip constraints on system internal columns (sys_nc...$) from export.
- Fix missing output directory in generic psql file for data loading.
- Add missing progress bar during TYPE and PARTITION export type.
- Remove duplicated message in debug mode during Oracle reconnection.
- Allow file input with create type declaration to use ora2pg converter.
Unsupported syntax is signaled into the output file.
- Exclude MLOG$.* and RUPD$.* table from export.
- Prevent export of indexes and constraints during FDW export type.
- Fix wrong total number of sequences shown in progress bar.
- Remove warning when PG_DSN is define during a export type that do not
support direct import into PostgreSQL.
- Auto switch prefix from DBA to ALL when error 942 is returned when
looking at tables informations. A hint is also displayed to ask for
activating USER_GRANTS or connect using a user with DBA privilege.
- Add REORDERING_COLUMNS configuration directive to allow reordering
columns during the TABLE export. This could help to minimized the
footprint on disc, so that more rows fit on a data page. Thanks to
Christian Bjornbak for the feature request.
- Fix call to unblessed reference at disconnect when direct import to
pg is not used. Thanks to Christian Bjornbak for the report.
- Fix regression in drop/create foreign keys and index during data
export. Thanks to Christian Bjornbak for the report.
- Fix truncate table error with parallel and direct data copy. Thanks
to keymaper for the report.
- Fix several other issues with parallel and direct data import.
- Fix trigger export on multi files when FILE_PER_FUNCTION is enabled.
- Fix issue on converting boolean values with non default values.
Thanks to Christian Bjornbak for the report.
- Fix boolean value for disabled key in default %BOOLEAN_MAP key/value.
- Fix case where INTO was wrongly replaced by INTO STRICT. Thanks to
Jacky Rigoreau for the report.
- Fix case where label after a END was not removed. Thanks to Jacky
Rigoreau for the report.
- Fix discard of input file parsing. Fix PERFORM replacement in PL/SQL
code wirh cursor. Thanks to Jacky Rigoreau for the report.
- Enable PG_SUPPORTS_MVIEW by default and update documentation.
- Replace DBA_DATA_FILES by USER_SEGMENTS to get database size to avoid
error ORA-00942. Thanks to Pierre Boizot for the report.
- Fix trigger conversion error. Thanks to Pierre Boizot for the report.
- Add support to PostgreSQL 9.3 materialized view syntaxe, this need a
new configuration directive PG_SUPPORTS_MVIEW to be enabled.
- Update default configuration file and documentation about USE_TABLESPACE.
- Add USE_TABLESPACE configuration directive to force ora2pg to use Oracle
tablespace name with table, constraints indexes and indexes if tablespace
in not in the default (TEMP, USERS, SYSTEM). Thanks to Rob Moolhuijsen
for the feature request.
- Allow DEFER_FKEY, when enabled during TABLE export, to create all foreign
keys as DEFERRABLE and INITIALLY DEFERRED. Thanks to David Greco for the patch.
- Fix non working ON_ERROR_STOP set to 0 during data export.
- Lot of code changes to fix dump to file in multiprocess mode. Ora2Pg will
also only drop/create constraints and indexes related to the allow/exclude
tables, thanks to Maciej Bak for the report.
- Force decimal character from Oracle output to be a dot. Thanks to Maciej Bak
for the report.
- Add default exclusion of Oracle recycle bin objects with name begining by BIN$.
- Fix escaping quote in table and column comments. Thanks to realyota for the report.
- Reduce DECODE migration cost from 2 to 1 unit.
- Reduce OUTER JOIN (+) migration cost from 3 to 1 unit.
- Add Time::HiRes to the requirement chapter for Perl <= 5.8. Thanks to
Mike Kienenberger for the report.
- Replace wrong use of --config instead of --conf into the documentation. Thanks
to Mike Kienenberger for the report.
- Fix regex used to rewrite CREATE VIEW code. Thanks to David Greco for
the patch.
- Fix an issue with oracle copies when primary key was negative. Thanks
to David Greco for the patch.
- Fix case sensitivity with SEQUENCE when preserve_case is enabled.
Thanks to Jean-Max Reymond for the report.
- Fix table COMMENT export when preserve_case is enabled. Thanks to
Jean-Max Reymond for the report.
2013 05 28 - v11.4
This release fixes others several major issues on migration cost assessment that
was not addressed in previous release, please upgrade.
- Fix other major issues in migration cost assessment.
- Redefine some migration cost values to be more precise.
2013 05 27 - v11.3
This release fixes several major issues on migration cost assessment, especialy
with stored procedures with lot of lines or if you have lot of comments in that
code. You may want to run your database evaluation again as the estimated times
can be up to tree time lower on huge PL/SQL code.
- Add full details about PL/SQL evaluation by ora2pg when --estimate_cost
or ESTIMATE_COST is enable. This will display cost units per keywords
detected in the function/package code.
- Fix wrong cost unit assessment on PL/SQL code size, this bug generated
very high migration cost assessment for functions/packages with lot of
lines. Please run your tests again, estimated times can be up to tree
time lower on huge code.
- Remove comments before code evalution.
- Fix file input parser for PL/SQL packages export when IS or AS was in
the next line than the CREATE PACKAGE BODY ...
- Exclude NOT NULL constraint from the count of CHECK constraints into
the TABLE report.
- Fix decimal precision in table migration assessment cost.
- Fix typo in changelog.
2013 05 01 - v11.2
This release fixes several major issues especially with direct import of data
into PostgreSQL and Windows port that was both broken.
- Update doc about Windows multiprocess issues and acknowledgements.
- Fix Windows OS issues using multiprocessing options by disabling
multiprocess support on this plateform. When -J or -j will be used a
warning will be displayed and Ora2Pg will simply run single process
like in previous 10.x versions. Thanks to Jean Marc Yao Adingra for
the report.
- Fix RAW and LONG RAW export to ByteA. Thanks to Prabhat Tripathi for
the report and testing.
- Fix patch regression on multiple TRUNCATE call for a single table.
Thanks to David Greco for the report.
- Placed calls to DB handle InactiveDestroy outside the forked process
to prevent fatal errors on Windows. Thanks to Jean Marc Adingra for
the report.
- Forked running processes are renamed into more readable name like
"ora2pg logger" for the progress bar, "ora2pg - querying Oracle" when
used with -J option and "ora2pg - sending to PostgreSQL" to better
know what is the current job of the process.
- Removed the use of /Y flag in Windows install script, this was causing
error "dmake: Error code 130, while making install_all". Thanks to
Jean-Marc Adingra for the report.
- Fix direct import to PostgreSQL that was just producing nothing. Thank
to David Greco for the patch.
- Fix ora2pg usage documentation.
- Add an underscore to CLIENT ENCODING in SHOW_ENCODING output to be the
same as the configuration directive.
UPGRADE: please reinstall all as most of the files have changed.
2013 04 07 - v11.1
This release adds partition data speed improvement by exporting data directly
from and into the destination partitioned table. There's also some bug fix on
RAW or LONG RAW data export and PL/SQL to PL/PGSQL code rewrite.
- Adjust cost assessment for indexes, tables and tables partition.
- Add comment to report of index partition about local index only.
- Fix position of TRUNCATE TABLE in output file.
- Fix export of data from RAW or LONG RAW columns, they was exported
as hex string. Now data are converted using utl_raw.cast_to_varchar2()
function before being escaped for insert into a bytea. Thanks to Alex
Delianis for the report.
- Fix issue with Oracle TIMESTAMP(0) data export that add a single
ending point, ex: "2008-08-09 00:00:00.", this ending character is
now removed by format_data_type(). Thanks to Pierre-Marie Petit for
the report.
- Fix typo on MODIFY_STRUCT description.
- Force DEBUG to off in default configuration file.
- Change range PARTITION operators in the check conditions, >= and <
replaced by > and <=, corresponding to Oracle VALUES LESS THAN.
- Add ALLOW_PARTITION to limit data export to a list of partition name.
- PLSQL: Fix wrong replacement of SELECT by PERFORM during VIEW export.
- Partitioned tables data is now imported directly into the destination
tables instead of inserted into the main table and dispatched by the
trigger. Ora2Pg will automatically detect the in/out table partition,
there's nothing to configure.
- PL/SQL: Do not allow decode() rewrite by case/when/else when there
is a function call in it.
- Fix Error when Compress::Zlib is not installed, this module is not
mandatory.
UPGRADE: please reinstall all as all files have changed.
2013 03 24 - v11.0
This is a new major release because it adds support to multiprocessing to export
data in parallel mode, this allow to improve speed during data import by more
than ten times. This multiprocessiing capabilities allow Ora2Pg to be closer than
the speed of any ETL. To compare speed or allow using Kettle for data import,
there's now a new export type to obtain Kettle XML transformation files. This
release adds also lot of work on speed improvement to scan Oracle database with
huge number of object.
- Add documentation about JOBS, ORACLE_COPIES, DEFINED_PK configuration
directive and informations about KETTLE export type.
- Add KETTLE export type to generate XML transformation file definition
for Penthatlo Data Integrator (Kettle). Thanks to Marc Cousin for the
work. Example of use:
ora2pg -c ora2pg.conf -t KETTLE -j 12 -J 4 -o loaddata.sh
- Fix major bug in export of auto generated named constraint. Thanks to
mrojasaquino fot the report.
- Show number of rows in the top largest tables.
- Add TOP_MAX description to the documentation.
- Add the TOP_MAX directive to default configuration file and update
documentation. Directive used to control the top N tables to show.
- Add top N of largest tables in SHOW_TABLE, SHOW_COLUMN and SHOW_REPORT
export type.
- Fix progressbar output when ora2pg is interrupted by ctrl+c.
- Add JOBS, ORACLE_COPIES and DEFINED_PK directives to configuration file.
JOBS replacing THREAD_COUNT but backward compatibility is preserve.
- Add 3 new command line options, -j | --jobs and -J | --copies, used to
set the number of connection to PostgreSQL and Oracle for parallel
processing. The third, -L | --limit is used to change DATA_LIMIT at
command line.
- Add multiprocess support on data export. With the help of Thomas Ogrisegg.
- Add more schema in SYSUSERS that should not be exported.
- Add full detailed information about SYNONYM in SHOW_REPORT.
- Add MODIFY_TYPE configuration directive to allow some table/column
type to be changed on PostgreSQL side during the export.
- Fix objects type count in progressbar of SHOW_REPORT.
- Restrict table and index in SHOW_REPORT to the tables defined in ALLOW
and EXCLUDE directives.
- Show total number of rows in SHOW_TABLE and SHOW_REPORT output.
- Add top 10 of tables sorted by number of rows in SHOW_TABLE and
SHOW_REPORT output.
- Fix typo in SYNONYM objects.
- Add report of top ten tables ordered y number of rows.
- Rewrite most of the Oracle schema storage information extraction for
speed improvement.
- Use Hash to store column informations.
- Fix %unique_keys declaration in _table() method.
- Remove call to _table_info() from SHOW_REPORT code as those informations
are already loaded with the _table() method.
- Fix missing column definition on TABLE export.
- Add progress bar during output generation following export type.
- Add STOP_ON_ERROR configuration directive to enable/disable the call to
ON_ERROR_STOP into generated SQL scripts. Thanks to Ludovic Penet for
the feature request.
- Huge speed improvement on columns informations retrieving.
- Fix progress bar to keep the total number of tables related to the ALLOW
or EXCLUDE configuration directives. Thanks to Ludovic Penet for the report.
- Change return type of function _table_info(), it now returns data instead
of the database handle.
- Improve speed on indexes and constraints extraction for database with huge
number of tables.
- Improve performance to retrieve columns information and comments.
- Remove report of column details during export in debug mode, use SHOW_COLUMN
instead.
- Remove call to upper() in objects owner condition to improve performance
with database with huge number of objects.
- Add a fix to not export foreign key for exclude tables. Thanks to Ludovic
Penet for the report.
- Fix Windows install issue with copying ora2pg.conf.dist. Thanks to
Dominique Fourdrinoy for the report.
- Increase the cost of Oracle function not converted to PG automatically.
UPGRADE: reinstall all is required to override the old installation, you may use the
new ora2pg.conf.dist file which included the new configuration directives.
2013 01 15 - v10.1
This release adds HTML report for migration cost assessment and some bug fix.
- Fix global where should not be overwritten. Thanks to Dan Harbin for
the patch.
- Fix bug/typo in boolean replacement, where a colon was used instead
of a single quote. Thanks to Alex Delianis for the patch.
- Update copyright.
- Add detection of additional Oracle functions for better migration
cost assessment.
- Update documentation.
- Force report detail in lowercase.
- Added information about the migration cost value to the reports.
- Add --dump_as_html command line option and DUMP_AS_HTML configuration
directive.
- Allow migration report to be generated as HTML.
- Separate report generation code from data collection code.
2012 12 12 - v10.0
This is the first version of Ora2Pg 10.x series, that is a major release.
Overall numerous improvements and bugs fixes there's now a new export type:
SHOW_REPORT that will output a report of all objects contained in your Oracle
database and some comments on how they will be exported. With this report you
can use a new directive ESTIMATE_COST to ask to Ora2Pg to evaluate the database
migration cost in terms of man days. There's also an other new configuration
directive EXTERNAL_TO_FDW, disable by default, to permit the export of all
Oracle external tables as file_fdw foreign tables.
The database content report and the migration cost estimation is a work in
progress so all feedback on these new features are welcome. Here is the complete
changelog:
- Update documentation about ora2pg usage and new feature.
- Fix quote escaping on table comments. Thanks to Sebastian Fischer.
- Fix some other issues with 8i databases, added database version auto-
detection to avoid printinf warning. Thanks to Sebastian Fischer for
the help.
- Allow null value in BFILE to the oar2pg_get_bfilename().
- Update documentation about BFILE export.
- Add drop function ora2pg_get_bfilename() when necessary.
- Add support to BFILE external path export by creating a function
ora2pg_get_bfilename( p_bfile IN BFILE ) to retrieve path from BFILE.
BFILE will be exported as text field with the full path to the file as
value. Note that this is the first time that Ora2Pg need write access
to the Oracle database, if you do not have BFILE or you have set the
corresponding PostgreSQL type asd bytea (the default) the function
will not be created.
- Fix a performance issue when extracting BLOB with a LongReadLen upper
than 1MB.
- Fix priviledge on schema created from Oracle package body. Thanks to
Dominique Legendre for the report.
- Add object type in comment before priviledge extraction.
- Order output of grant to groups grants by object types. This is useful
to quickly disable some SQL orders corresponding of not already loaded
objects. Thanks to Dominique Legendre for the feature request.
- Fix progress bar output.
- Fix priviledge on sequence, tablespace and schema.
- Fix backward compatibility with Oracle 8i, remove query with JOIN.
Thanks to Sebastian Fischer for the report.
- Fix backward compatibility with Oracle 8i on priviledge extraction.
Thanks to Sebastian Fischer for the report.
- Fix backward compatibility with Oracle 8i on index extraction. Thanks
to Sebastian Fischer for the report.
- Add more precision in cost estimation.
- Add somme other PL/SQL uncovered code detection.
- Add more debug information during data extraction.
- Removed progress bar when debug is enabled.
- Add report and estimate cost about CHECK constraint and function
based indexes.
- Update documentation about new export directives SHOW_REPORT and
ESTIMATE_COST.
- Add --estimate_cost and --cost_unit_value command line options.
- Add ESTIMATE_COST and COST_UNIT_VALUE to default configuration file.
- Rewritte and extend support to ROWNUM replacement.
- Remove incompatible grants between Oracle and the PortgreSQL export,
especially on views.
- Limit GRANT export to VALID object. Activate EXPORT_INVALID to enable
grants export on all object.
- Add export of VALID only views. To export all with INVALID ones you
must activate the EXPORT_INVALID directive. Thanks to Dominique
Legendre for the feature request.
- Fix issue in substr() pl/sql replacement, thanks to Dominique
Legendre for the report, plus add other code replacements in pl/sql.
- Fix issue with function name not on the same line as the create
statement - was affecting file input only.
- Add report of number of JOB object in the database (SHOW_REPORT).
- Add PL/SQL replacement of various form of EXEC function call.
- Remove creation of password with users that are not requiring
password. Thanks to Dominique Legendre for the feature request.
- A sql type and a precision can now be used in REPLACE_AS_BOOLEAN to
replace all filed with that type as a boolean, example:
NUMBER:1 will replace all field of type NUMBER(1) as a boolean.
- Fix grants on partition export, will now used all_ and user_ tables.
- Fix removing of newline in the DECLARE clause. Thanks to Dominique
Legendre for the report.
- PostgreSQL client_encoding is now forced to UTF8 when BINMODE is set
to utf8. Thanks to Dominique Legendre for the report.
- Replace DISABLE TRIGGER ALL by DISABLE TRIGGER USER following the value
if USER_GRANTS to avoid permission denied on constraint trigger when
data are load under a non PG superuser. Thanks to Dominique Legendre
for the report.
- Rename DISABLE_TABLE_TRIGGERS to DISABLE_TRIGGERS and set default value
to 0. Other values are USER or ALL following the connected user.
- Fix missing newline after comment in PL/SQL code. Thanks to Dominique
Legendre for the report.
- Fix report message on external table export.
- The export TYPE have been entirely rewritten to only export supported
user defined types. Exported are: Nested Tables, Object type, Type in
herited and Subtype, Varrays. Associative Arrays, Type Body and type
with member method are not supported.
- When FILE_PER_INDEX is enable, SQL order to move indexes in their
respective tablespace will be written into a dedicated file prefixed
by TBSP_INDEXES_.
- Fix location on external table export. Thanks to Thomas Reiss for
the help.
- PG_SUPPORTS_INSTEADOF is now activated by default, that mean that
migration should be done on PG >= 9.1.
- Remove obsolete --xtable commande line option, should be replaced by
--allow, backward compatibility is preserved.
- Add EXTERNAL_TO_FDW configuration directive, disable by default, to
export all Oracle external tables as file_fdw foreign tables.
- Fix an other case where user defined type were not exported with an
ending semi-colon. Thank to Dominique Legrendre for the report.
- Fix export of user defined type with extra ");" at end of the type
definition and remove system types from export. Thanks to Dominique
Legendre for the report.
- Add PLSQL replacemement of currval. Thanks to Thomas Reiss for the
patch.
- Add PLSQL replacement of PIPELINED and PIPE ROW by SETOF and RETURN
NEXT.
- Add rewrite of Oracle DETERMINISTIC function into PostgreSQL
IMMUTABLE function.
- Fix copy during install on MacOSx and add /Y option to windows
install copy to force overwrite existing files. Thanks to Dennis
Spaag for the report.
- Fix issue exporting rows with perl ARRAYS ref. Thanks to Sorin
Gheorghiu for the report.
- Add report of number of database link in SHOW_REPORT output.
- Fix major bug on export of NUMBER with precision, they was all
exported as bigint. Thanks to Dominique Legendre for the report.
- Add progress bar during SHOW_REPORT export.
- Add detailed report about index in SHOW_REPORT output.
- Fix data export when schema was given in lower case. Thanks to
Dominique Legendre for the report.
- Add SHOW_REPORT export type to display a full summary of the Oracle
database content.
- PLPGSQL: add the name keyword to XMLELEMENT call. Thanks to Thomas
Reiss for the hint.
- Add SHOW_VERSION export type to display the version of Oracle.
- Add COLLATION to the keyword list. Thanks to Dominique Legendre for
the report
- Change documentation to add more detail on exporting Oracle views as
PostgreSQL tables based on the new VIEW_AS_TABLE directive.
- Add -a | --allow option and --view_as_table to ora2pg script.
- Add VIEW_AS_TABLE configuration option to allow export of view as
table and permit the additional use of the ALLOW or/and EXCLUDE
directive. Thanks to Dominique Legendre for the feature request.
- Removed conflict with transaction when DEFER_FKEY was enabled and
allow DEFER_FKEY and DROP_FKEY to be enabled both. Before, only
DEFER_FKEY was used in this case, now both are used and of course
DEFER_FKEY is wasted. Thanks to Dominique Legendre for the report.
- Directives ALLOW and EXCLUDE are now usable with all kind of object
following the export type.
- Rename TABLES directive as ALLOW to be less confusing, backward
compatibility is preserved.
- Thanks to Dominique Legendre for the feature request.
- Remove auto ordering of table export following the foreign keys to
fix an infinite loop. Thanks to Siva Janamanchi for the report.
- Rewrite the view as table export to reuse the same code as table
export, old code was resulting in issues with disable triggers and
deferring constraints.
- Remove alter session to set NLS_NCHAR that was returning error on some
installation. Thanks to Dominique Legendre for the report.
- Fix replacement of IS SELECT wrongly replaced by IS PERFORM in some
case. Thanks to Dominique Legendre fot the report.
UPGRADE: Almost all files have changed so a new installation is required.
2012 10 07 - v9.3
- Add auto detection of Oracle character set and the corresponding
PostgreSQL client encoding to use. NLS_LANG and CLIENT_ENCODING
configuration directives can be leaved commented, Ora2Pg will set
their values automatically.
- Add PL/SQL replacement of CURSOR IS SELECT by CURSOR FOR SELECT and
IS REF CURSOR by REFCURSOR. Thanks to Dominique Legendre for the
report.
- Fix missing set client_encoding orders into fonction or procedure
export file. Thanks to Dominique Legendre for the report.
- Fix not working SKIP configuration directive. Thanks to Siva
Janamanchi for the report.
- Add configuration directive NULL_EQUAL_EMPTY to disable the IS NULL
and IS NOT NULL replacement into PL/SQL code. Enabled by default.
Thanks to Dominique Legendre for the feature request.
- Remove exclusion of object names with the dollar sign. Thanks to
Konrad Beiske for the suggestion.
- Fix timestamp with time zone when microsecond is enabled. Thanks
to Siva Janamanchi for the report.
- Fix extra semi-column when PKEY_IN_CREATE is enabled. Thanks to
Siva Janamanchi for the report.
- Update configuration about boolean replacement.
- Allow any column type replacement as a boolean in PostgreSQL, values
will be converted as well. Thanks to Konrad Beiske for the feature
request.
- Add REPLACE_AS_BOOLEAN and BOOLEAN_VALUES configuration directives
to allow type replacement with a boolean. Thanks to Konrad Beiske
for the feature request.
- Add new configuration directive PKEY_IN_CREATE to add primary keys
definition in CREATE TABLE statement instead of creating them after
with an ALTER TABLE statement. For Greenplum database, primary key
must be created with the CREATE TABLE statement so you may want to
enable this configuration directive. Thanks to Siva Janamanchi for
the feature request.
- Add new configuration directive USE_RESERVED_WORDS to force Ora2Pg to
auto-detect PostgreSQL reserved words in Oracle object's names and
automatically double quote them. Thanks to Siva Janamanchi for the
feature request.
- SHOW_TABLE and SHOW_COLUMN will now display a warning when Oracle
object's name is a PG reserved words. Those names will need to be
renamed or double-quoted (see USE_RESERVED_WORDS).
- Add TIMESTAMP WITH LOCAL TIME ZONE Oracle type conversion to timestamp
and force timestamp with time zone format to use TZH:TZM. Thanks to
Siva Janamanchi for the report.
- Fix table and column replacement issues introduced with path that
removed double-quote when PRESERVE_CASE is disabled. Thanks to Steve
DeLong for the report.
- PLPGSQL convertion: Fix SELECT replacement with PERFORM in VIEW
declaration. Thanks to Thierry Capitaine for the report.
- Add display Ora2Pg type conversion map between Oracle originals types
and PostgreSQL's types when using export type SHOW_COLUMN. Thanks to
Thierry Capitaine for the feature request.
- Reorder command line options in ora2pg script usage and documentation.
- Add call to quote_ident() and quote_literal() into materialized
functions to secure parameters.
- Fix major issue in pl/sql to pl/pgsql conversion with multiple package
declaration in the same code record. Thanks to Marc Cousin for the
report.
- Add data type TIMESTAMP WITH TIME ZONE. Thanks to Siva Janamanchi for
the report.
- Add new export type: MVIEW to allow export of all materialized views
as snapshot materialized view (fully reload of the view).
- Add -e | --exclude option to Perl script ora2pg to exclude some given
objects from the export. It will override any value of the EXCLUDE
directive. The value is a coma separated list of object name or regex.
- Update domumentation about the EXCLUDE directive change.
- Allow exclusion from export of functions, procedures and functions in
package by specifying a list of name or regex in EXCLUDE directive.
Thanks to Keith Fiske from Omniti for the feature request.
UPGRADE: Almost all files have changed so a new installation is required.
2012 09 05 - v9.2
- In plpgsql conversion, SELECT without INTO becomes PERFORM.
- In plpgsql conversion, EXECUTE IMMEDIATE replaced by EXECUTE.
- Fix DATA_TYPE value in configuration file.
- Fix case sensitivity on data export using COPY mode.
- Directive XML_PRETTY is now disabled by default as it is better to
use getClobVal() to get the XML data out of an xmltype column.
- Add documentation about regex usage int EXCLUDE and TABLES directives.
- Remove all double-quote around object name when CASE_SENSITIVY is
disabled. Thanks to Dominique Legendre for the suggestion.
- Rename CASE_SENSITIVE by PRESERVE_CASE to be less confusing, backward
compatibility preserved. Thanks to Dominique Legendre for the request.
- Add support to user defined type data export. Before it will simply
export an array reference ARRAY(0xa555fb8), now the array is explored
and inserted as ROW(col1,col2,...). Thanks to Mathieu Wingel for the
feature request.
- Fix bug in direct data import in postgresql with COPY: pg_putcopydata
can only be called directly after issuing a COPY FROM command. Thanks
to Steve Delong for the report.
- Add warning at any debug level before abort when a data file already
exist during data export.
- Fix issue with oracle sensitivity when exporting data.
- Fix search_path on package export, indexed and constraints files on
TABLE export.
- Remove obsolete ORA_SENSITIVE configuration directive, thanks to
Dominique Legendre it is no more used.
- Force automatic conversion of PL/SQL when entry is an input file.
- Fix errors in main file for package loader with FILE_PER_FUNCTION
enabled.
- Fix case where package body where not exported.
- Add missing EXPORT_INVALID directive into default configuration file.
- Fix replacement of END followed by the name of the function, the semi-
colon was removed.
- Fix case sensitivity issue in INDEX creation.
- Fix case sensitivity issue in CHECK constraint and a typo in a
progress bar variable.
- Replace old DATA export type by INSERT in configuration file.
- Fix case sensitivity issue in ALTER TABLE ... ADD CONSTRAINT. Thanks
to David Greco for the report.
- Add set client_encoding before table output to handle character
encoding into comments and possibly objects names.
- Fix some case sensitivity issue with schema name. Thanks to Dominique
Legendre for the report.
- Do not display warning message about direct import if no connection
to a PostgreSQL database is defined.
- Allow multiple export type to be specified into the ora2pg -t command
line option.
- Dump progress bar to stderr instead of stdout to separate logs.
- Add new -q | --quiet option to perl script ora2pg to disable progress
bar.
2012 08 19 - 9.1
- Add progress bar to show data export progression.
- Add -q | --quiet option to ora2pg perl script to disable progress bar.
- Change documention about tnsnames.ora to mark it is not necessary.
- Add progress bar during data export, per table and globaly.
- Replace export type DATA by INSERT to mark the difference with COPY
and avoid confusion. Documentation is updated and full backward
compatibility preserved.
- Improve Oracle case sensitivity detection on column and update
documentation about ORA_SENSITIVE directive
- Direct import for COPY statement now used DBD::Pg and pg_putcopydata()
instead of a pipe to psql command.
- Fix case sentitivity issue on disabling/enabling all triggers.
- Add autodetection of case sensitivity with column name.
- Move trunc() to data_truc() convertion into the ALLOW_CODE_BREAK part.
- Update comment about FILE_PER_FUNCTION in configuration file.
- Fix NOT NULL constraint add twice, the first time in the column
definition and the second time in an ALTER TABLE ... ADD CONSTRAINT
... CHECK ( ... NOT NULL). Reported by Dominique Legendre.
- Add support to direct CALL of stored procedures in trigger definition.
Reported by Dominique Legendre.
- Remove index creation on primary and unique key autogenerated by
PostgreSQL.
- Fix PL/SQL to PLPGSQL automatic convertion on index when exporting
data with DROP_INDEX activated.
- Fix DROP_INDEX to only delete indexes that will be created at end.
- Fix search path when exporting data with EXPORT_SCHEMA disabled
- Add missing documentation about the LOGFILE directive
- Fix case sensitivity on sequence export. They will now always be
insensitive as in PostgreSQL its called is converted between quotes:
nextval('seq_name'). Reported by Dominique Legendre.
- Limit export of primary and unique key if KEEP_PKEY_NAMES is enabled
to those who are not autogenerated by Oracle. Reported by Dominique
Legendre.
- Trigger export is now limited to those belonging to table that are not
excluded from the export (see TABLES and EXCLUDE directives). Reported
by Dominique Legendre
- Fix case sensitivity on trigger export.
- Fix data export failure in table with column name with accent.
Reported by Dominique Legendre.
- Fix set client_encoding syntax. Reported by Dominique Legendre
- Add automatic try with oracle sensitivity when an error occurs during
retreving table information. This additionaly also fixes an error when
table has accent on his name.
- Fix replacement of user defined data type with directive DATA_TYPE.
Reported by Dominique Legendre.
- Fix function or procedure detection with external input file. Reported
by Arul Shaji.
- Update documentation about Windows installation and ActiveState Perl
distribution. Thanks to Stephan Hilb for the report.
- Fix date format issue by forcing NLS_DATE_FORMAT to format:
YYYY-MM-DD HH24:MI:SS. Thanks to Stephan Hilb for the report.
- Remove obsolete pod documentation in Ora2Pg.pm.
- Add new configuration directive CREATE_SCHEMA to disable the sql
command of schema creation at top of the output file during TABLE
export type. Patch by David Greco.
- Added converting INSERTING/UPDATING/DELETING to PG_OP=INSERT, etc.
Patch by David Greco.
- Fix parsing leading ':' on triggers, as they generally have :NEW and
:OLD to denote the new and old recordsets. Patch by David Greco
- Add new PG_INTEGER_TYPE configuration directive activated by default,
to limit conversion into postgresql integer or bigint of Oracle number
without scale - NUMBER(p), PG_NUMERIC_TYPE is now reserved to convert
NUMBER(p,s). Patch by David Greco.
- Limit numeric with precision <= 9 to be converted as integer, numeric
with precision >= 10 will be converted to bigint to handle integer
above 2147483647. Patch by David Greco.
- Add plsql to plpgsql automatic conversion on check constraints. Patch
by David Greco.
- Add plpgsql replacement, patch by David Greco:
REGEX_LIKE( string, pattern ) => string ~ pattern
- Update documentation about NOESCAPE and STANDARD_CONFORMING_STRING
- Change place of the ENABLE_MICROSECOND into the documentation.
- Fix forgot to add documentation about encryption with Oracle server.
- Add missing DISABLE_COMMENT configuraton directive in default
configuration file and update documentation
2012 07 15 - 9.0
- Remove call to obsolete BINDIR and MANDIR variables into packaging
scripts to reflect the changes in Makefile.PL.
- Update documentation about installation of Ora2Pg under Windows.
- Automatically set LONGREADLEN to ORA_PIECE_SIZE length if the last one
is larger, for CLOB export.
- Change Makefile.PL and source tree to fully support installation under
Windows OSes.
- Change double quote by single in Makefile.PL perl replacement call.
- Replace double quote by single one in $CONFIG_FILE default setting to
simplify automatic replacement at install.
- Fix CLOB export that was limited to 64Kb even with LONGREADLEN defined
to an upper value. Patch use the ora_piece_size DBD::Oracle prepare
attribute. Patch by Mohamed Gargouri. See here for more detail:
http://search.cpan.org/~pythian/DBD-Oracle-1.46/lib/DBD/Oracle.pm#Piecewise_Fetch_with_Polling
- Add a note into documentation about encrypted communication between
Ora2Pg and Oracle. Note by Jenny Palomino.
- Change documentation to reflect change to the format of the Oracle
timestamp with millisecond. This format is now enabled by default in
configuration file.
- Fix bug with LONGREADLEN and LONGTRUNCOK when exporting LOB that was
not applied even after change into the configuration file. Reported
by Mohamed Gargouri
- Fix microsecond format FF3 not compatible with Oracle 8i. Set to FF.
- Add a warning to stderr when a table export need that ORA_SENSITIVE
be enabled.
- Fix case where Oracle indexes with same name as a constraint was not
exported - Rodrigo
The following are old patches that was not applied to v8.10 and the git
repository:
- Fix creation of bad constraint for each indexes.
- Add DISABLE_COMMENT configuration directive to remove comments from
TABLE export type. Comments are exported by default.
- Fix a bug in removing function name repetion at end
- Add PL/SQL to PLGPSQL replacement of the to_number function
- Fix PL/SQL to PLGPSQL replacement of substr into substring
- Add replacement of specials IEEE 754 values BINARY_(FLOAT|DOUBLE)_NAN
and BINARY_(FLOAT|DOUBLE)_INFINITY by NaN and Infinity on PLPGSQL
conversion and on data export - Thanks to Daniel Lyons.
- Fix return type of function with just OUT or INOUT params. Thanks to
Krasi Zlatev for the patch.
- Add schema name on functions or procedures export when EXPORT_SCHEMA
is activated. Thanks to Krasi Zlatev for the patch.
- Fix case sensitivity issue with schema on partition export.
- Fix case sensitivity issue with --xtable option.
- Fix issues with case sensitivity on the schema owner set into the
SCHEMA configuration directive.
- Add default search_path on schema for contraints, index and data
export when EXPORT_SCHEMA is activated.
- Fix case sensitivity issue in search_path.
- Force Oracle datetime format to be YYYY-MM-DD HH24:MI:SS.FF in client
session to prevent other defined output format. Thanks to Aaron Culich
for the patch.
- Add export/import of table and column comment. Thanks to Krasi Zlatev
for the patch.
2012 06 26 - 8.13
- Fix broken export with missing single quote in Oracle timestamp export
formating with to_char(timestampcolumn, YYYY-MM-DD HH24:mi:ss). Thanks
to Steve Delong for the report.
2012 06 22 - 8.12
- Add nex configuration directive ENABLE_MICROSECOND to allow timestamp
to be exported with millisecond precision. Thanks to Patrick King for
the feature request.
- Fix multiple quote on foreign keys column names. Thanks to Vitaliy for
the report.
- Add new export type FDW to allow table export as foreign table for
oracle_fdw. Thanks to David Fetter for the feature request.
- Fix typo in LongTruncOk variable name. Thanks to Magnus Hagander for
the patch.
- Add XML_PRETTY configuration directive to replace getStringVal() by
getClobVal() when extracting XML data. Thanks to Magnus Hagander for
the patch.
- Fix case sensitivity issue in ALTER TABLE and TRUNCATE statement.
Thanks to Magnus Hagander for the patch.
UPGRADE: Ora2Pg.pm and ora2pg perl scripts have changed as well as configuration
file. Documentation has been updated too so you'd better install all again.
2012 04 30 - 8.11
- Fix an error when running ora2pg directly against PG, index and
constraints are created against PG instead of being written to
the output file. Thanks to David Greco for the report.
- Ora2Pg will now output a warning message when direct import to PG
is set with other import type than COPY and DATA.
- Fix NUL character removing on LOB to bytea export. Thanks to info31
on PostgresqlFr for the report.
2012 03 11 - 8.10
- Add two configuration directives to control the BLOB export.
LONGREADLEN to set the database handle's 'LongReadLen' attribute
to a value that will be the larger than the expected size of the
LOB. LONGTRUNKOK to bypass the 'ORA-24345: A Truncation' error.
Thanks to Dirk Treger for the report.
- Fix install problem on non-threaded Perl and the threads package.
Replace use() by require() call. Thanks to Ian Sillitoe for the patch.
- Fix strange Oracle behaviour where binary_double infinity is exported
from Oracle as '~'. Replaced by 'inf'. Thanks to Daniel Lyons for the
report.
UPGRADE: only Ora2Pg.pm have changed so you can just override it. See also
documentation for new configuration directives: LONGREADLEN and LONGTRUNKOK.
2011 11 07 - 8.9
- Fix double quote into file name of package function export when case
sensitivity is preserved.
- Add support to XMLType data extraction. Thanks to Aaron Culich for
the report. Before this release, xml data was exported as a Perl
array reference.
- Fix bug during foreign key export when foreign keys have different
owners. Thanks to Krasi Zlatev for the patch.
- Add support to plpgsql conversion during index extraction as many
index use some Oracle function on their declaration. Thanks to
Sriram Chandrasekaran fot the feature request.
- PLSQL: Add replacement of Oracle subtr() by PostgreSQL substring().
Thanks to Sriram Chandrasekaran fot the feature request.
- PLSQL: Add replacement of Oracle decode() by PostgreSQL CASE/THEN/ELSE.
Thanks to Sriram Chandrasekaran fot the feature request.
Note that this two replacement was not implemented because they could
break the code if there's complex subqueries inside their declaration.
This is why you can enable it by setting ALLOW_CODE_BREAK to 1 (new).
In later release this directive will be enable by default.
- Add output ordering on some object name so that results between two
runs can be compared. Thanks to Henk Enting for the patch.
- Fix misshandling of all cases of RAISE_APPLICATION_ERROR rewrite into
RAISE EXCEPTION concatenations. Thanks to Krasi Zlatev for the report.
UPGRADE: only Ora2Pg.pm and Ora2Pg/PSQL.pm have changed so you can just override them
if you dont want to reinstall all.
2011 10 13 - 8.8
- Before that release when you upgraded Ora2Pg using Makefile, the old
ora2pg.conf was renamed as ora2pg.old. This can lead to lost previous
configuration, the old ora2pg.conf is now untouched and the new one is
installed as ora2pg.conf.new.
- Renamed ora2pg.pl into ora2pg_pl in the package before installation
to avoid the copy of the perl script into the site Perl install dir.
It is still installed as ora2pg in /usr/local/bin by default.
- Fix errors that appeared to be due to no quoting on the field names
when ORA_SENSITIVE is enabled. Thank to Sam Nelson for the patch.
- Limit case sensitivity on check constraints to column names only,
before that if there was a value between double quote into the check
constraint, it was wrongly changed to lower case.
- Fix broken case sensitivity at data export when disabling/enabling
triggers and truncating tables with copy or insert statement.
- Change Ora2Pg version in packaging files that was still in 8.5.
UPGRADE: only Ora2Pg.pm have changed so you can just override it.
2011 09 07 - 8.7
- The escape_bytea() function has been rewritten using a prebuild array
to gain twice of performances. Thanks to Marc Cousin from Dalibo for
the patch.
- Improve speed of bulkload data by disabling autocommit by issuing a
BEGIN at the start and COMMIT at the end.
- Add multi-threading support. It is only used to do the escaping to
convert LOBs to byteas, as it is very cpu hungry. There's a lot of
CPU-waste here. The threads number is controlled by a new configuration
directive: THREAD_COUNT. Putting 6 threads will only triple your
throughput, if your machine has enough cores. If zero (default value),
do not use threads, do not waste CPU, but be slower with bytea.
Performance seems to peak at 5 threads, if you have enough cores, and
triples throughput on tables having LOB. Another important thing:
because of the way threading works in perl, threads consume a lot of
memory. Put a low (5000 for instance) DATA_LIMIT if you activate
threading. Many thanks to Marc Cousin for this great patch.
- Fix standard_conforming_string usage on export as INSERT statement.
- Fix an issue with importing Oracle NULL character (\0 or char(0)) with
bytea and character data with UTF8 encoding. Now whatever is the data
type or the encoding, this character is simply removed to prevent the
well known 'ERROR: invalid byte sequence for encoding "UTF8": 0x00.'
Thanks to Jean-Paul Argudo from Dalibo for the report.
- Fix an incorrect syntax for "for each statement" triggers.
Thanks to Henk Enting for the report.
- Add comment at end of line to signal on which cursor the replacement
on " EXIT WHEN (...)%NOTFOUND " is done. This will return something
like "IF NOT FOUND THEN EXIT; END IF; -- apply on $1". Thanks to
jehan Guillaume De Rorthais from Dalibo for the report this help a lot
during Pl/Pgsql code review.
- Fix table/column name replacement on constraint creation and dropping
when REPLACE_TABLES/REPLACE_COLS is set during DATA/COPY export.
- Fix table/column name replacement on indexes creation and dropping
when REPLACE_TABLES/REPLACE_COLS is set during DATA/COPY export.
- Remove unused table name parameter in _drop_indexes() function.
- Add support to REPLACE_TABLES/REPLACE_COLS during schema export.
Before this release those replacements were only applied to DATA or
COPY export. You can now use it in schema export, it will replace
table and/or column names in the TABLE/INDEX/CONSTRAINT schema export.
MODIFY_STRUCT is still limited to DATA or COPY export as it have no
sense outside this export. Unfortunately those replacements can not be
done easilly in other export type like TRIGGER, FUNCTION, etc. so you
must still edit this code by hand.
- Use the bundled Perl Config module to detect if Perl is compiled with
useithread. This mean that the old local defined %Config hash has been
replaced by %AConfig.
- SKIP indices is now obsolete and must be replaced with SKIP indexes.
backward compatibility is preserved.
- The file generated when FILE_PER_INDEX is activated has been renamed
into INDEXES_... instead of INDICES_...
- Add a warning on tablespace export when Oracle user is not a dba.
- Fix fatal error when dumping to one file per function with double
directory output.
- Fix double print of FATAL messages and dirty database disconnect on
fatal errors.
- Add setting of client_encoding into each export type as defined in
the configuration file.
- Update web site documentation.
UPGRADE: Ora2Pg.pm, Ora2Pg/PGSQL.pm and ora2pg have changed so they must be
overwritten. There's also changes in the configuration file and documentation
has changed as well. Backward compatibility is fully preserved.
2011 07 07 - 8.6
- Remove "use strict" from head of Ora2Pg.pm that breaks view export.
This is usually removed before public release, but not this time.
Thanks to Jehan Guillaume de Rorthais from Dalibo for the report.
- Add a new configuration directive called COMPILE_SCHEMA that force
Oracle to compile the PL/SQL before code extraction to validate
code that was invalidate for any reason before. If you set it to 1,
you will force compiling of the user session schema, but you can
specify the name of the schema to compile as the value too. Thanks
to Jean-Paul Argudo from Dalibo for the solution.
- Add new configuration directive EXPORT_INVALID to allow export of all
PL/SQL code even if it is marked as invalid status. The 'VALID' or
'INVALID' status applies to functions, procedures, packages and user
defined types.
- Excluded from export all tables, types, views, functions and packages
that contains a $ character. Most of the time they don't need to be
exported.
- PLSQL: add automatic conversion of Oracle SYS_REFURSOR as PostgreSQL
REFCURSOR.
- Rewrite entirely the parser of DBMS_OUTPUT du to concatenation errors.
- PLSQL: add automatic replacement of some Oracle exception errors:
INVALID_CURSOR=>INVALID_CURSOR_STATE, ZERO_DIVIDE=>DIVISION_BY_ZERO,
STORAGE_ERROR=>OUT_OF_MEMORY.
UPGRADE: Ora2Pg.pm and Ora2Pg/PGSQL.pm have changed so they must be overwritten.
There's also changes in the configuration file and documentation has changed as
well. Backward compatibility is fully preserved.
2011 07 01 - 8.5
- When FILE_PER_FUNCTION is activated and export type is PACKAGE, Ora2Pg
will now save all functions/procedures of a package body into a
directory named as the package name and into different files. This
will allow to load each function separatly or load them all with the
OUTPUT SQL script generated by Ora2Pg.
- Fix Oracle package body parsing failure when a procedure is declared
inside an other.
- Add new configuration options FILE_PER_CONSTRAINT and FILE_PER_INDEX
to generate three files during the schema extraction. One for the
'CREATE TABLE' statements, one for the constraints (primary keys,
foreign keys, etc.) and the last one for indices. Thanks to Daniel
Scott for the feature request.
- Allow to process PL/SQL Oracle code from file instead of a database
to apply Ora2Pg code conversion. Thank to Mindy Markowitz for the
feature request. See -i or --input_file command line option to ora2pg
perl script or INPUT_FILE new configuration option.
- Add new configuration directive STANDARD_CONFORMING_STRINGS that is
used only during DATA export type to build INSERT statements. This
should avoid 'WARNING: nonstandard use of \\ in a string literal'.
Please check that this behavior is backward compatible with your
PostgreSQL usage as this is enabled by default now.
UPGRADE: The new features has changed Ora2Pg.pm and ora2pg.pl so that they must
be overwritten. There's also changes in the configuration file and documentation
has changed as well. Take care of backward compatibility with escaped strings in
DATA export type and the new behavior on PACKAGE export.
2011 06 07 - 8.4
- Moves Ora2Pg to SourceForge.net.
- Fix an issue on setting owner in "ALTER SEQUENCE ... SET OWNER TO".
Thanks to Herve Girres for the report.
- Bugfix on lower case convertion for check constraints extraction.
Thanks to Alexander Korotkov for the patch.
UPGRADE: There's no new functionality, this is a bug fix release.
2011 05 11 - 8.3
- Fix issue on inherited user defined types converted to inherited tables.
Add comment on unsupported inherited type in PostgreSQL too. Thanks to
Mathieu Wingel for the report.
- Fix issue on column default values. Oracle all this kind of strange
syntax: counter NUMBER(4) default '' not null, that was translated to
counter smallint DEFAULT '' by Ora2Pg. Thanks to Mathieu Wingel this is
now rewritten as DEFAULT NOT NULL.
- Fix case sensitivity on create view when there was double quote on the
column name statement part. Thanks to Mathieu Wingel or the report.
- Fix bad patch applied on column name case sensitivity issue during check
constraint export. Thanks to Philippe Rimbault for the report.
- Fix bug on package export introduced into version v8.2. The issue was
related to end of package procedure detection. hanks to Mathieu Wingel
or the report.
UPGRADE: There's no new functionality, this is a bug fix release and every one
should upgrade to it.
2011 05 01 - 8.2
- PLSQL: automatic replacement of EXIT WHEN cursor%NOTFOUND; by Pg
synthax: IF NOT FOUND THEN EXIT; END IF;. Works with additional
condition too.
- PLSQL: Automatic replacement of SQL%NOTFOUND by NOT FOUND.
- PLSQL: Add detection of TOO_MANY_ROW to NO_DATA_FOUND to add STRICT.
- Completely rewrite the parsing of Oracle package body to handle all
cases and especially prodedure declared into an other procedure.
Those procedure are renamed INTERNAL_FUNCTION and must be rewritten.
- Fix type usage of ora2pg Perl script.
- Add a new directive FORCE_OWNER. By default the owner of the database
objects is the one you're using to connect to PostgreSQL. If you use
an other user (postgres for exemple) you can force Ora2Pg to set the
object owner to be the one used in the Oracle database by setting the
directive to 1, or to a completely different username by setting the
directive value to that username. Thanks to Herve Girres from Meteo
France for the suggestion and patch.
- Add --forceowner or -f command line option to ora2pg program.
- Add SHOW_ENCODING extract type to return the Oracle session encoding.
For example it can return: NLS_LANG AMERICAN_AMERICA.AL32UTF8
- Remove SYS_EXTRACT_UTC from index creation as Pg always stores them
in UTC. Thanks to Daniel Scott for the patch.
- In PLSQL code SYS_EXTRACT_UTC is replaced by the Pg syntaxe:
field AT TIME ZONE 'UTC'.
- Fix a pending problem with "Wide character in print at" on COPY mode.
Thanks to Bernd Helmle from Credativ GmbH for the patch.
- PLSQL: Add automatic rewrite of FOR ... IN REVERSE ... into Pg synthax
- Fix column name case sensitivity issue during check constraint export.
Thanks to Daniel Berger for the report.
- Remove the possibility to add comment after a configuration directive
it may not be used and it was generating an issue with the passwords
configuration directives for examples. Thanks to Daniel Berger for the
report.
- Complete rewrite of user defined type extraction. Add support of inherited
type using Oracle UNDER keyword as well as better support to custom type
with BODY. Thanks to Mathieu Wingel for the report.
- Fix case sensitivity on user defined types. Thanks to Mathieu Wingel for
the report.
UPGRADE: All files have changed so you need a fresh install/upgrade.
Previous release used to remove any string starting from a # in the config file,
this was to allow comments after a configuration directive. This possibility
have been removed in this release so you can no more add comments after a
configuration directive.
2011 03 28 - 8.1
- Prevent Ora2PG to export twice datas when using FILE_PER_TABLE and
the data output file exist. This is useful in case of export failure
and you don't want to export all data again. This also mean that if
you want to get new data you have to remove the old files before.
- Fix parsing of procedure/function into pl/sql Oracle package.
- Fix bug in IS NULL/IS NOT NULL replacement. Thanks to Jean-Paul Argudo
from Dalibo for the report.
- Add CREATE OR REPLACE on RULE creation.
- Add DROP TRIGGER IF EXISTS before trigger creation.
- Replace Oracle date "0000-00-00" by NULL.
- Fix infinite loop in package/fonction type replacement.
- Add one file per package creation if FILE_PER_FUNCTION is enabled.
- Fix double quote in name of custom type extraction.
- Add extraction of custom type IS VARRAY as an custom type of table
array. Thank to Jean-Paul Argudo from Dalibo for the patch.
- Fix multiple double quote in name of create index definition.
- Apply excluded and limited table to tablespace extraction.
- Fix function and procedure detection/parsing on package content.
- Fix schema prefix in function name declaration in package export.
- PLSQL: Replace some way of extracting date part of a date :
TO_NUMBER(TO_CHAR(...)) rewritten into TO_CHAR(...)::integer when
TO_NUMBER just have one argument.
- Fix Makefile.pl error when trying to modify file ora2pg now renamed
into ora2pg.pl
- Add 3 new export types SHOW_SCHEMA, SHOW_TABLE and SHOW_COLUMN. Those
new extraction keyword are use to only display the requested information
and exit. This allow you to quickly know on what you are going to work.
The SHOW_COLUMN allow a new ora2pg command line option: '--xtable relname'
or '-x relname' to limit the displayed information to the given table.
- Add type replacement for BINARY_INTEGER and PLS_INTEGER as integer.
UPGRADE: Please make a full upgrade asap to this release.
2011 03 15 - 8.0
This major release simplify and improve Oracle to PostgreSQL export. Ora2Pg v8.x
now assume that you have a modern PostgreSQL release to take full advantage of
the Oracle compatibility effort of the PostgreSQL development team. Ora2Pg since
v8.x release will only be compatible with Pg >= 8.4.
- Remove addition of AS for alias as with modern PG version this can
be optional (Pg >= 8.4).
- Fix CREATE with missing USER/ROLE for grant extraction. Thanks to
Herve Girres for the report.
- Apply missing psql_pgsql converter to view definition.
- PLSQL : Normalize HAVING ... GROUP BY into GROUP BY ... HAVING clause
- PLSQL : Convert call to Oracle function add_months() in Pg syntax
- PLSQL : Convert call to Oracle function add_years() in Pg syntax
- Apply missing psql_pgsql converter to triggers WHEN clause.
- Fix DECLARE CURSOR rewrite.
- Allow one file per function / procedure / package exported with a new
configuration option FILE_PER_FUNCTION. Useful to editing and testing.
Thank to Jean-Paul Argudo from DALIBO for the feature request.
- The FILE_PER_TABLE configuration option is now also applied to views.
- Remove obsolete PG_SUPPORTS_INOUT as it is supported by with modern
PG version (Pg >= 8.4).
- Remove obsolete PG_SUPPORTS_DEFAULT as it is supported by with modern
PG version (Pg >= 8.4).
- Allow to adjust PostgreSQL client encoding with a new configuration
directive: CLIENT_ENCODING.
- Add TRUNCATE_TABLE configuration directive to add TRUNCATE TABLE
instruction before loading data.
- Add type conversion of Oracle XMLTYPE into PostgreSQL xml type.
- PLSQL: SYSDATE is now replaced by LOCALTIMESTAMP to not use timezone.
Thanks to Jean-Paul Argudo from DALIBO for the report.
- Use 'CREATE OR REPLACE' on create trigger function instruction.
- Fix prefixing by OUTPUT_DIR when file per table/function is enabled.
- Use 'CREATE OR REPLACE' on create view.
- PLSQL_PGSQL is now enabled by default. If you want to export Oracle
original function/procedure/package code, disable it.
- PLSQL: WHERE|AND ROWNUM = N; is automatically replaced by LIMIT N;
- PLSQL: Rewrite comment in CASE between WHEN and THEN that makes Pg
parser unhappy.
- PLSQL: Replace SQLCODE by SQLSTATE
UPGRADE: You must reinstall all and review your configuration file
2011 02 14 - 7.3
- Remove PG_SUPPORTS_INOUT, now Ora2Pg assumes the PostgreSQL database
destination support it (Pg > v8.1).
- Remove PG_SUPPORT_ROLES, now Ora2Pg assumes the PostgreSQL database
destination support it (Pg > v8.1).
- Complete rewrite of the GRANT (user/role/grant) export type. It now
should be only related to the current Oracle database. Note that do
not try to import rights asis as you may import have errors or worse
miss handling of the rights! Just remember that for example in Oracle
a schema is nothing else than a user so it must not be imported like
this.
- Fix multiple errors in partitionning definition. Thank to Reto Buchli
for the report.
- PLSQL: reordering cursor Oracle declaration "DECLARE CURSOR name" into
"DECLARE name CURSOR". Thank to Reto Buchli (WSL IT) for the report.
- Fix miss handling of DEFAULT parameters value in convert_function().
Thanks to Leonardo Cezar for the patch.
- Fix Oracle tablespace export where Pg tablespace location was based on
Oracle filename. This fix extract the path and replace the filename
with tablespace name. Thank to Reto Buchli (WSL IT) for the report.
- Fix parsing of ending function code. Thanks to Leonardo Cezar for the
patch.
- Fix call to _convert_procedure() that is in fact the same function as
_convert_function(). Thanks to Leonardo Cezar for the report.
- Fix multiple call on tablespace alter index on the same object. Thank
to Reto Buchli (WSL IT) for the report.
- PSQL: Rewrite RAISE EXCEPTION concatenations. Double pipe (||) are
replaced by % and value is set as parameter a la sprintf. Thank to
Reto Buchli (WSL IT) for the report.
- Add missing comment of PARTITION export type into configutation file.
- Complete rewrite of the table partition export part has it was not
handling all case and was really buggy.
- PLSQL: add normalisation of the to_date() function.
- Ora2Pg now warns during grant export when it is not connected as an
Oracle DBA user. GRANT export need rights of Oracle DBA or it fail.
- Fix install of changelog into Makefile.PL, name was wrong. Thanks to
Julian Moreno Patino for the patch.
2011 01 12 - 7.2
- Fix escaping of BLOB/RAW to bytea data import causing import to crash.
Thanks to Ozmen Emre Demirkol for the report.
- Add support to default value into functions parameter (PG >= 8.4).
Can be activated with a new configuration directive: PG_SUPPORTS_DEFAULT.
Default is 1, activated.
- Fix bad ending of exported function: remove trailing chars after END.
- Add support to WHEN clause on triggers (PG >= 9.0), can be activated
with a new configuration directive: PG_SUPPORTS_WHEN.
- Add support to INSTEAD OF usage on triggers (incoming PG >= 9.1). Can
be activated with a new configuration directive: PG_SUPPORTS_INSTEADOF.
- Fix error using SKIP directive. Thanks to Laurent Renard from Cap Gemini
for the report.
- Fix missing perl object instance in format_data() function.
- Fix duplicate procedure or function when export type use both FUNCTION
and PROCEDURE.
2010 12 04 - 7.1
- Improve direct DBD::Pg data export/import speed by 10.
- Add --section=3 in pod2man call into Makefile.PL. Thanks to Julian
Moreno Patino for the report.
- Renamed ChangeLog into changelog to avoid upstream warning with Debian
package. Thanks to Julian Moreno Patino for the suggestion.
- Fix some spelling mistakes in doc/Ora2Pg.pod. Thanks to Julian Moreno
Patino for the fix.
- Fix release version into Ora2Pg.pm and PLSQL.pm, was still in 6.5.
- Fix direct data export/import using DBD::Pg. Thanks to Laurent Renard
from Cap Gemini for the report.
- Fix drop/create contraints and index during direct data export/import
using DBD::Pg. Thanks to Thierry Grasland from Cap Gemini for the report.
2010 11 23 - 7.0
- Rename ora2pg perl script into ora2pg.pl in sources because Windows
users can't extract the tarball. During install it is renamed into
ora2pg. Thanks to Andrew Marlow for the report.
- Fix doinst.sh for SlackWare Slackbuid packaging.
- The DEFER_FKEY configuration directive has been fixed as it only
works in a transaction. Note that foreign keys must have been created
as DEFERRABLE or it also will not works. Thanks to Igor Gelman for the
report.
- Add DROP_FKEY configuration directive to force deletion of foreign keys
before the import and recreate them and the end of the import. This may
help if DEFER_FKEY not works for you.
- Add DROP_INDEX configuration directive to force deletion of all indexes
except the automatic index (primary keys) before data import and to
recreate them at end. This can be used to gain speed during import.
- Add TSMSYS, FLOWS_020100 and FLOWS_FILES to the owners exclude list.
This concern the SRS$ table and all tables begining with 'WWV_FLOW_'
- Change the way DATA_LIMIT is working. It must be used now to set the
bulk size of tuples return at once. Default is 10000.
- Improve data export speed by 6! The data export code has been entierly
rewritten and the speed gain is really fun.
- Add OUTPUT_DIR configuration directive to set a base directory where all
dumped files must be written. Default: current directory.
- Change value of default numeric(x) type from float to bigint and change
default numeric(x,y) type to double precision.
- Change conversion type for BFILE from text to bytea.
2010 09 10 - 6.4
- Configuration directives SHOWTABLEID, MIN and MAX are now obsolete
and has been definitively removed. They were never used and add too
much confusion.
- Fix bug in column name replacement where table name was also replaced.
Thank to Jean-Paul Argudo from DALIBO for the report.
- Fix case sensitive errata in PG schema search path. Thank to Jean-Paul
Argudo from DALIBO for the report.
- Remove double \n at end of debug message.
- Fix debug mode not activated if the DEBUG directive is enable and
the -d command line is not present.
- Add unbuffered output for debug message.
UPGRADE: simply override the Ora2Pg.pm Perl module where it is installed.
2010 07 22 - 6.3
- Fix Oracle 8i compatibility error during schema extraction complaining
that column CHAR_LENGTH doesn't exist. Thanks to Philippe Rimbault for
the report. Note that the error message is still displayed but tagged
as WARNING only.
- Fix error using the IMPORT option on a read_conf method call. Thanks
to Diogo Biazus for the report.
- Fix export of sequences that does not handle maxvalue well and can be
lower than minvalue. Thanks to Nathalie Doremieux for the report.
UPGRADE: Just override Ora2Pg.pm
2010 06 15 - 6.2
- Change default transaction isolation level from READ ONLY to
SERIALIZABLE to ensure consistency during data export. Thanks to
Hans-Jurgen Schonig from postgresql-support.de
- Add the TRANSACTION configuration directive to allow overriding of
the isolation level. Value can be readonly, readwrite, committed and
serializable. The last is the default.
2010 05 07 - 6.1
- Fix error on partition export following schema definition.
- Add first support to export Oracle user defined types.
- Add CTXSYS,XDB,WMSYS,SYSMAN,SQLTXPLAIN,MDSYS,EXFSYS,ORDSYS,DMSYS,
OLAPSYS to the sysuser default exclusion list.
- PLSQL.pm: Add automatic translation of Oracle raise_application_error
and dup_val_on_index to PG RAISE EXCEPTION and UNIQUE_VIOLATION.
- Change/fallback to a lower case package name (ora2pg-6.x.tar.gz).
- Change default convert type for 'LONG RAW' to bytea.
- Add PG_SCHEMA configuration directive to defined a coma delimited
list of schema to use in SET search_path PostgreSQL command.
2010 02 28 - 6.0
- Publish a dedicated site to Ora2Pg at http://ora2pg.darold.net/
- Add export of Oracle table partitoning. See export type PARTITION.
- Add command line arguments to perl script ora2pg. See --help for a
full listing of these option. The most interesting is --type to change
the export type directly at command execution without needing to edit
the configuration file, --plsql to directly enable PLSQL to PLPSQL
code conversion and --source, --user --password to set Oracle data
source. There's also --namespace to set the Oracle schema.
- Create all file for standard Perl module install. Install is now done
with: perl Makefile.PL && make && make install
- Move Ora2Pg license from Perl Artistics to GPLv3.
- Move PLSQL package as Ora2Pg::PLSQL for standard Perl module install.
- Remove use of Perl module String::Random.
- Rename program ora2pg.pl into ora2pg for standard usage.
- Fix extra double quote on column name of index export. Thanks to
Guillaume Lelarge for the patch.
- Add packaging facilities to build RPM, SlackBuild and Debian packages.
- Fix miss handling of Ora2Pg.pm options at object instance init.
- Configuration file ora2pg.conf is now generated by Makefile.PL
2009 12 18 - 5.5
- Fix CONSTANT declaration in Procedure/Function/Package export.
- Fix length of char and varchar on multibyte like UTF8 encoding. Thanks
to Ali Pouya for the patch.
- Fix view export where alias to column in Oracle not use 'AS' and
PostgreSQL required it. Thanks to Ali Pouya for the report.
- Add type replacement of sql variable in PLSQL code (PLSQL.pm). Thanks
to vijay for the patch.
2009 07 15 - 5.4
- Fix bug introduced in multiple output file feature. This bug force
Ora2pg to crach after the first table export when output is wanted in
a single file. Thanks to Leo Mannhart for the report.
- Fix debug filename output on multiple export file. Thanks to Leo
Mannhart for the report.
2009 07 07 - version 5.3
- Fix wrong escaping of data named as column during view export. Thank
to Andrea Agosti for the patch.
- Allow export of datas into one file per table. See FILE_PER_TABLE
configuration directive. Thanks to Alexandre - Aldeia Digital for the
idea.
2009 06 19 - version 5.2
- Fix order of the column name of the view which was not preserved. Now
ordered by COLUMN_ID. Thank to Andrea Agosti for the report.
- Fix case sensitivity in VIEW extraction. Thank to Andrea Agosti for
the patch.
2009 03 06 - version 5.1
- Fix missing -U username at Pg connection. Thanks to Brendan Richards.
- Fix $ENV{ORACLE_HOME} and $ENV{NLS_LANG} to not being overwritten
by configuration settings if they are already defined in environment.
- Fix typo in ora2pg.pl where keep_pkey_names was replaced by
keep_pkeys_name and so prevent use of KEEP_PKEY_NAMES in configuration.
Thanks to Olivier Mazain for the report.
- Configuration file directives are now case insensitive.
- Force $type parameter given to _sql_type() to be uppercase in that
methode instead of during function call. Thanks to Ali Pouya for the
report.
- Modify ora2pg.pl to remove the obsolete call to export_data(). Use
only export_schema() now.
- Modify ora2pg.pl to simplify it. Reading configuration is now done
internally by Ora2Pg.pm as well as all other initialization process.
You can always overwrite all configuration options into call to new
Ora2Pg(). Now ora2pg.pl can be as simple as:
use Ora2Pg;
my $schema = new Ora2Pg('config' => "/etc/ora2pg.conf");
$schema->export_schema();
exit(0);
This will be less confusing. You can upgrade Ora2Pg.pm without carring
about that, backward compatibility with previous version is preserved.
- Review entire documentation with the great help of Ali Pouya.
- Add type BOOLEAN converted to boolean.
- PG_SUPPORTS_INOUT is now enabled by default in the configuration file
- SQL and PL/SQL to PLPGSQL converter:
.Replace MINUS call to EXCEPT
.Replace DBMS_OUTPUT.put, DBMS_OUTPUT.put_line, DBMS_OUTPUT.new_line
by the PLPGSQL equivalent: RAISE NOTICE
.Rewrite function/procedure/package convertion functions.
This Oracle SQL converter for function/procedure/package is now only
applied if the configuration directive PLSQL_PGSQL is enable, else
these Oracle code are exported as is. Thanks to Ali Pouya for the help.
- Reserved call to sql transaction only for DATA export type. Others
export type now use \set ON_ERROR_STOP ON. Thanks to Ali Pouya.
- Fix tablespace creation into schema (missing search_path). Thanks to
Olivier Mazain.
- Fix the type returned by the _sql_type() method in the case of a
numeric with null len and pg_numeric_type is set. Thanks to Ali Pouya.
- Change function body delimiter to $body$ to allow use of $$ into the
body as quote replacement. Thanks to Ali Pouya.
- Fix returns type from function. If multiple OUT parameters: RECORD,
if only one OUT parameter, return his type. If no out parameter: return
VOID. Thanks to Ali Pouya.
- Fix export DATA when the name of a column in the table match COMMENT,
AUDIT or any other defined reserved words. These reserved words are
defined in a new configuration variable ORA_RESERVED_WORDS. It accept
a list of comma separated reserved words. Thanks to Andrea Agosti for
the report.
- Fix configuration parser that omit custom SYSUSERS definition.
2009 02 13 - version 5.0
- Fix places where $self->{prefix} where not used. This prefix is
used to replace DBA_... objects into ALL_... objects. Thanks to Daniel
Scott report and patch.
- Fix some problem on trigger export (missing ending semicolon, return
opaque replaced by return trigger, add missing return new value, single
quote for delimitating the function body hits against quotes inside the
function). Thanks to Luca DallOlio for reports and patches.
- Add first attempt to rewrite plsql code to plpgsql code (see function
plsql_to_plpgsql in new perl module PLSQL.pm). There's a configuration
option named PLSQL_PGSQL to activate the convertion.
2008 12 16 - version 4.11
- Fix Ora2Pg failure on Oracle database with case sensitive tables.
Thanks to Marc Cousin for report and patch.
- Fix missing schema name in query when extract views as tables.
2008 12 04 - version 4.10
- Fix missing replacement of table name on disable triggers when
required.
- Fix some malformed debug output messages.
- Add the capability to extract data from view as if it was a table.
This is usefull if you want to export/import data from an Oracle
view into a Pg table. There's nothing special to do, just to give
the view name into the TABLES configuration directive and set TYPE
to DATA or COPY. If views are not specified in the TABLES directive
there's not view export but only table data.
- Add capability to extract views structure as table schema. There's
nothing special to do, just to give the view name into the TABLES
configuration directive and set TYPE to TABLE. This will not extract
constraints or other table tuning from table used in the view. Thanks
to Groupe SAMSE for the feature request.
2008 10 27 - version 4.9
- Modify the DISABLE_TABLE_TRIGGERS configuration option. Should be now
replaced by DISABLE_TRIGGERS, but compatibility is preserved.
- Add DISABLE_SEQUENCE configuration option to not export alter
sequence after COPY or DATA export.
- Fix extraction of function based index that appears as SYS_NC....
Thanks to Bozkurt Erkut from SONY for the report
2008 09 04 - version 4.8
- Add SYSUSERS configuration option that allow you to specify a coma
separated list of Oracle System user/schema to exclude from extracted
object. By default it only exclude user SYS,SYSTEM,DBSNMP,OUTLN and
PERFSTAT
- Add support to other binary mode output than ':raw' to avoid the Perl
error message:"Wide character in print". See the BINMODE configuration
directive. This will help a lot if you have UTF-8 records.
Thank to Guillaume Demillecamps for the report.
- Fix double escaping of special character.
Thank to Guillaume Demillecamps for the report.
2008 01 25 - version 4.7
- Add support to regular expressions in the exclusion list. Thanks to
Peter Eisentraut
- Fix misformatted SQL string in function _extract_sequence_info.
Thanks to Bernd Helmle.
- Add escaping of backslash on COPY output. Thanks to Peter Eisentraut
2008 01 03 - version 4.6
- Applied a patch to add ALTER SEQUENCE statements to the dump to
adjust the sequence to the correct values after DATA and COPY dumps.
Thanks to Bernd Helmle.
- Applied a patch which fixes problems with broken COPY output when
extracting data from Orace databases with embedded tabs, carriage
returns and line feeds. Thanks to Bernd Helmle.
- Move the project to PgFoundry
2007 06 20 - version 4.5
- Fix columns order in index extraction. Thanks to Ugo Brunel from BULL.
2007 04 30 - version 4.4
- Fix missing single quote in role extraction.
- Add configuration directive NOESCAPE to enable/disable
escaping characters during data extraction. Default is enabled.
- Add TIMESTAMP, BINARY_FLOAT and BINARY_DOUBLE data type translation.
- Add DATA_TYPE configuration directive to allow user defined data type
translation.
- Add NLS_LANG configuration directive to set Oracle database encoding
and enforce a default language-setting in ora2pg.pl. Thanks to Lars
Weber
2007 04 03 - version 4.3
- Fix duplicate view export. Add schema selector to views. Thank to
Ugo BRUNEL from BULL for the fix.
- Remove 'use strict' to prevent failure on certain condition.
Thank to Andrea Schnabl for the report.
2006 06 08 - version 4.2
- Fix a miss taping on constraint type search that convert unique key
to primary key. Thank to Ugo BRUNEL (BULL) for the patch.
- Fix case sensitivity on CHECK constraint that could cause problem when
check value is uppercase. Thank to Ugo BRUNEL (BULL) for the patch.
2006 03 28 - version 4.1
- Fix a problem when using data_limit and where clause. Thank to
Rene Bentzen for the patch.
- Add enable/disable trigger on data import. Thank to Bernd Helmle.
- Fix escaping of chr(13) MS crashing data import into PG. Thank
to Ugo Brunel (BULL).
2006 03 22 - version 4.0
- Add validation of the requested schema in the database before all.
Thanks to Max Walton for the idea.
- Add multiple export type at the same time. Thanks to Max Walton
for the idea.
- Add support for in/out/inout function parameter. See PG_SUPPORTS_INOUT
configuration option. Thanks to Bernd Helmle for this great
contribution/patch.
- Add support for ROLES with Pg v8.1+. See PG_SUPPORTS_ROLE configure
option.
2006 02 10 - version 3.4
This release add better support to Oracle grant, function and grant
extraction. Great thanks to the Pg team!
- Add preservation of oracle primary key names. See KEEP_PKEY_NAMES
configuration option. Thanks to Antonios Christofides for this patch.
- Fix bug in case insensitive check constrainte. Thanks to Wojciech
Szenajch for the patch.
- Fix saving data to files correctly (binmod) when the oracle database
contains utf8 chars. Thanks to Richard Chen for the report.
- Fix bug on view extraction when a column contains the word WITH.
Thanks to Richard Chen for the patch.
- Fix wrong mapping between tge data type in Oracle "number(10)" and
Postgresql, which should be "integer" and not "bigint". Thanks to
Sergio Freire for the patch.
- Fix bug in EXCLUDE configuration directive parsing. Thanks to Matt
Miller for the patch.
2005 02 22 - version 3.3
- Fix bug "Modification of a read-only value attempted"
2005 02 11 - version 3.2
- Fix patch error on column position sort
- Replace 'now' by CURRENT_TIMESTAMP on SYSDATE replacement
- Fix bytea type that was not quoted.
2005 02 10 - version 3.1
- Fix bug on deferrable constraint. Thanks to Antonios Christofide for
the patch.
- Fix problem on defer_fkey that should be in a transaction. Thanks to
Antonios Christofide for the patch.
- Add sort by column position during schema extraction.
- Add support to SYSDATE. Thanks to David Cotter-Alatto Technologies Ltd
2004 12 24 - version 3.0
- Add 'TABLESPACE' extraction type to create PostgreSQL v8 tablespace.
2004 12 24 - version 2.9
- Debuging output rewrite. Thanks to Antonios Christofide for help.
- Add 'PG_NUMERIC_TYPE' configuration option to replace portable
numeric type into PostgreSQL internal type (smallint, integer,
bigint, real and float).
2004 12 24 - version 2.8
- Fix/add support to data export of type BLOB, RAW and LONG RAW.
Thanks to Antonios Christofide for help.
2004 12 23 - version 2.7
- Add 'FKEY_DEFERRABLE' configuration option to force foreign key
constraints to be exported as deferrable. Thanks to Antonios
Christofide for help.
- Add 'DEFER_FKEY' configuration option to defer all foreign key
constraints during data export. Thanks to Antonios Christofide
for help.
2004 12 23 - version 2.6
- Fix duplicate output during export. Thanks to Adriano Bonat for the
report.
- Fix data limit infinite loop during data extraction. Thanks to Thomas
REISS for the report.
- Add 'GEN_USER_PWD' configuration option allowing to generate a random
password. Thanks to Antonios Christofide for help.
(Require String::Random from CPAN).
- Fix USER/ROLES/GRANT extraction problem. Now all users are dumped.
All roles are translated to PostgreSQL groups. All grants are
exported. YOU MUST EDIT the output file to rewrite real privilege
and match your needs. Thanks to Antonios Christofide for help.
- Fix split COPY export into multiple transaction for large data export.
The number of row per transaction is set to 'DATA_LIMIT' value. A
value of O mean all in a single transaction.
2004 10 13 - version 2.5
- Fix extraction problem when the connection to Oracle DB is not as DBA.
2004 08 22 - version 2.4
- Fix bug in DBI errstr call.
- Add CASE_SENSITIVE configuration option to allow case sensitivity on
Add a new configuration directive 'USER_GRANTS' to do that. Thanks to
Octavi Fors for the report.
object name. Thanks to Thomas Wegner.
- Fix major bug in unique keys extraction. Thanks to Andreas Haumer and
Marco Lombardo for their great help.
- Add CHECK constraint extration. Thanks again to Andreas Haumer.
- Add IMPORT configuration option to include common configuration file
throught multiple configuration files.Thanks to Adam Sah and Zedo Inc.
- Add SKIP configuration option to turning off extraction of certain
- schema features. Thanks to Adam Sah and Zedo Inc.
- Fix bug in excluded tables
- Fix backslash escaping. Thanks to Adam Sah and Zedo Inc.
- Add REPLACE_TABLES configuration option to change table name during
data extraction.
- Add REPLACE_COLS configuration option to change columns name during
data extraction.
- Add WHERE configuration option to add where clause to each table or
specific tables during extraction. Usefull for replication. Thanks
to Adam Sah and Zedo Inc.
- Add progress indicators (per 1000 rows) and performance results
during data extraction in debug mod. Thanks to Adam Sah and Zedo Inc.
- Add Gzip and Bzip2 compress to output file if extension .gz or .bz2.
Gzip compress require perl module Compress::Zlib from CPAN. Thanks
to Adam Sah for the idea.
2004 04 13 - Version 2.3
- Fix bug in date/time conversion when using data export limit. Thanks
to Andreas Haumer.
- Add sort order when extracting tables and data to respect the TABLES
limited extraction array write order. Usefull if you have foreign key
constraints. Thanks to Andreas Haumer for the idea.
2004 04 13 - Version 2.2
- Add EXCLUDE configuration option to allow table exclusion
from all extraction.
- Fix a bug in escaping single quote on data export.
2004 03 09 - Version 2.1
- Fix COPY output by replacing special character.
- Add configuration file usefull for people who don't have Perl in mind
Thank's to Tanya Krasnokutsky to force me to do that :-)
- Fix other minor problem.
2002 12 26 - Version 2.0
- Clean code.
- Fix COPY output on column value with EOL and add column naming.
- Add support to the PostgreSQL 7.3 schema. So Oracle schema can now be
exported. (see export_schema init option)
- Remove data extraction limit (old default: 10) so each tuple will be
dump by default.
2002 12 03 - Version 1.12
I have fixed 2 bugs when using it against Oracle 817R3 on linux.
- Fix problem regarding RI constraints, the owner name was not
getting into the sql statement. Thank to Ian Boston.
- Moved all the RI constraints out of the create table statement.
Thank to Ian Boston for this contribution. This was a major request
from Ora2pg users.
2002 09 27 - Version 1.11
- Fix a problem when retrieving package+package body. Thanks to Mike
WILHELM-HILTZ.
- Set LongReadLen to 100000 when exporting table information. Many
users reports this kind of error: A-01406 LongReadLen too small and/or
LongTruncOk not set. This should fix the problem else you must
increase the value.
- Filtering by owner for better performance when retreiving database
schema. Thanks to Jefferson MEDEIROS.
2002 07 29 - Version 1.10
- Fix a problem with local settings regarding decimal separator (all ,
are changed to .) Thank to Jan Kester.
2002 06 04 - Version 1.9
- Fix a problem on exporting data which fill NULL instead of 0 or
empty string. Thanks to Jan Kester.
- Add time + date when export data [ tochar('YYYY-MM-DD HH24:MI:SS') ].
Thanks to Paolo Mattioli.
2002 03 05 - Version 1.8
- Add Oracle type FLOAT conversion to float8.
- Add column alias extraction on view.
Thanks to Jean-Francois RIPOUTEAU
- Add PACKAGE extraction (type => DATA).
2002 02 14 - Version 1.7
- Remove export of OUTLINE object type. Thanks to Jean-Paul ARGUDO.
2002 01 07 - Version 1.6
- Fix problem exporting NULL value. Thanks to Stephane Schildknecht.
2001 12 28 - Version 1.5
- Fix LongReadLen problem when exporting Oracle data on LONG and LOB
types. Thanks to Stephane Schildknecht for report and test.
- Add more precision on NUMBER type conversion
- Add conversion of type LONG, LOB, FILE
- Fix a problem when extracting data, sometime table could need to be
prefixed by the schema name.
- Fix output of Oracle data extraction. It now require a call to
function export_data().
2001 06 27 - Version 1.4
- Add online Oracle data extraction and insertion into PG database.
- Data export as insert statement (type => DATA)
- Data export as copy from stdin statement (type => COPY)
2001 06 20 - Version 1.3
- Grant/privilege extraction are now done separatly with option
type=>'GRANT'
- Sequence extraction with the option type=>'SEQUENCE'
- Trigger extraction with the option type=>'TRIGGER'
- Function extraction with the option type=>'FUNCTION' and
type=>'PROCEDURE'
- Complete rewrite of the foreign key extraction
- Fix incorrect type translation and many other bug fix
- Add schema only extraction by option schema => 'MYSCHEM'
2001 05 11 - Version 1.2
- Views extraction is now really done with the option type=>'VIEW'
- Add indexes extraction on tables.
- Changes name of constraints, default is now used.
- Add debug printing to see that the process is running :-)
- Add extraction of only required tablename.
- Add extraction of only n to n table indice. Indices of extraction
can be obtained with the option showtableid set to 1.
- Fix print of NOT NULL field.
- Complete rewrite of the grant extraction
- Complete rewrite of most things
2001 05 09 - Version 1.1
- Add table grant extraction based on group.
Oracle ROLES are exported as groups in PG
2001 05 09 - Initial version 1.0
------------------------------------------------------------------------------
Special thanks to Ali Pouya for documentation review. All my recognition
to Ali Pouya and Olivier Mazain for their great work in the package and
function export. Thanks to Jean-Paul Argudo for the time spent to heavily
testing Ora2Pg.
Special thanks to Josian Larcheveque and Stephane Silly as Oracle DBA
and their "patience".
Special Thanks to Dominique Legendre for his help on Spatial support and
all the tests performed.
Many thanks for all congratulation message, idea and bug report+fix I received.
Very special thanks to Jean-Paul Argudo that represent Ora2Pg at Linux Solution Paris 2005.
Gilles DAROLD <gilles (at) darold (dot) net>
|