1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854
|
TODO
* libfdata next search keep going on a key match?
* fix flaky test for System_Search_AutoSummary (one byte off?)
* pyesedb
- null value support
- long value: add data size support
* refactor libesedb_record_get_value_binary_data_size and
libesedb_record_get_value_binary_data
* complete libfdata btree refactor
- libesedb_values_tree.c replace by fdata functions?
- fix is_flexible_match for xp_search.edb indexes
* record flags
- add indication of corrupt records
* format notes:
- value of un-initialized page checksum: 0xdeadbeefbaadf00d
"ESE Super ECCXORChecksum"
* create tool to analyze cache usage, hits and misses
* move read of data definition into leaf value callback
- create separate callback for long values?
- this will clean up data definition
* refactor:
- libesedb_multi_value_get_value what about value_type ?
- clean up multi value calls to libfvalue
- libesedb_long_value_get_segment_data
* deprecate:
- libesedb_record_get_value
- libesedb_multi_value_get_value
* complete libfvalue update integration - multi value
* windows search add support for windows 8 SystemIndex_PropertyStore
* libesedb check esedbtools_libX header files for _LIBESEDB_LIBX_ defs
* handle compressed values inside the library
- compressed binary data
* esedbexport -t /root creates /root.export ?
* esedbexport skip table on error
* check Win2K ntds.dit support
* clean up and refactor info handle
* export handle: fix creation of indexes directory
API:
* add function to retrieve database signature
- can be used to match up EDB/STM pairs
* add long value functions (string and binary data?)
* remove
- libesedb_record_get_value
- libesedb_long_value_get_segment_data
- libesedb_multi_value_get_value
* windows search
- multi long value segment decryption/compression support
* export error in more test files: nibble count value exceeds maximum
* code clean up
- change page value index and number of values into int
- clean up TODO's
- refactor info handle
* combine libesedb_io_handle_read_page with libesedb_page_read
- need libesedb_page_read for recovery ?
* page tree on demand
- check sanity of page value 0
* clean up
- libesedb_data_type_definition remove offset
* handle default value
- need database which uses default value
* parse variable types even when they are not part of the table catalog definition
* add support for build in table number 1
* implement cloning of items for improved asynchronous and multi threaded usage
libesedb:
* add get_table_by_object_identifier function
* ECC checksum
- non 4 byte modulus buffer support
- add alignment optimization
- error correction
- Windows 7 extended checksum ?
* template table
- what about non-tagged data types
* multi value
- what about multi value 0x18
* database header
- compare primary and secondary
* catalog
- compare primary and secondary
- limit the size of the catalog ?
- support for type 0x0006
- support for type 0x0007 (line 3457 in libesedb_page_tree.c)
* long values
- support for non-linear stored long value segments - current solution does a sanity check
* default values
- apply to data definition
Error tolerance
* probe test for the page size using the backup database header
* use backup catalog
Debug
* is function libesedb_debug_print_table_group_of_bits needed?
* print value flags
* print part of keys as strings
Tools
* esedbinfo
- page summary (scan for uninitialized, empty, etc. pages)
- file header summary (check backup file header)
- summary of catalog (check backup catalog)
- print database state
* esedbexport
- add export mode
Windows search:
* improve windows_search_decompress_byte_indexed_compressed_data
File format
* where is the root object defined? or is it implied to be object 1?
* compressed table, column, index names ?
* page flags
- 0x0400, 0x0800, 0x8000
* tagged values
- flag 0x4000
* multi value offsets
- flag 0x8000
* page flags
* page keys
* (revision 0x11) extended page header
* what about the first byte in 7-bit ascii compressed data
* codepage 1200 string in Exchange EDB starts with 0xff 0xff
* streaming file
- only consists of ESEDB header with paged streamed data,
how to reconstruct streams ?
Deployment:
* update configure
- improve libfdata detection
- improve libfguid detection
- improve libfwnt detection
* update manuals
- add esedbexport.1
Performance
* create data definition tree in page tree for faster lookup of keys
keep list for sequential access (B+-tree)
* create a UTF-8 version of run-length compressed UTF-16 string decompression function
* use register alignment for key comparison
* optimize libesedb_checksum_calculate_little_endian_ecc32
Test suite:
* windows search: work on index-based compression
- create test program to validate output
Notes
* create a root page table instead?
- no
* read catalog data type defintions and convert them into column definitions?
- no
20160902
* worked on tests
20160110
* worked on Python 3 support
20160107
* 2016 update
20151219
* changes for deployment
20151213
* worked on support for newly discovered unknown page flag
* removed LIBESEDB_FLAG_READ and LIBESEDB_FLAG_WRITE
20151205
* worked on Python bindings
20150905
* updated dependencies
20150822
* worked on setup.py
20150704
* code clean up
20150409
* changes for long value look up in dirty databases
20150405
* changes for handling tagged data types in version 0x620 revision 0x14
database
* changes for long value key compare
20150322
* worked on Python bindings
* worked on long value support
* worked on long value error tolerance
* worked on NULL value support (JET_coltypNil)
20150321
* worked on libfdata btree refactor
* changes for long value key compare
20150105
* 2015 update
20141224
* worked on Python 3 support
* worked on tests
20141129
* code clean up
20141110
* code clean up
20141109
* worked on reading backup catalog
* additional checks for out of bound page tags value offset and size values
* worked on libfdata btree refactor
20141026
* changes for deployment
20141019
* changes for deployment
20141014
* worked on utility scripts
20141004
* update Python-bindings tests
* additional checks for out of bound page tags value offset and size values
20141003
* removed README.macosx
* changes for project site move
20140915
* worked on libfdata btree refactor
20140803
* bug fix in Python-bindings
20140726
* updated dependencies
20130406
* worked on Python-bindings
* updated msvscpp files
20130317
* worked on setup.py
20130305
* worked on Python-bindings
* worked on multi value
20130304
* worked on Python-bindings
20130302
* worked on Python-bindings
20130301
* fix in dpkg files
* worked on Python-bindings
20130217
* worked on Python-bindings
20130216
* 2014 update
* updated dependencies
* worked on Python-bindings
* worked on libfdata update
* worked on libfvalue update
20130421
* worked on libfvalue update
20130420
* worked on libfvalue update
20130415
* worked on libfvalue update
20130414
* updated dependencies
* updated macosx files
* textual changes
* changes for libfdata update
* worked on libfvalue update
20130303
* updated dependencies
* 2013 update
* changes for libfdata update
20120826
* updated dependencies
* project move
20120517
* updated dependencies
20120423
* fixes for deployment
20120414
* updated dependencies
* code clean up
20120411
* code clean up
20120102
* 2012 update
20111210
* updated libcstring, libsystem
* esedbexport:
- added table index nr to filename for duplicate table names
- added microseconds to filetimes
20111127
* updated libfdata
* added libfcache
20111120
* code clean up
* updated array type, list type
* updated libcstring, libsystem, libfdata, libfdatetime, libfguid, libfvalue,
libfmapi, libfwnt
* removed restriction that page tree header should consists of 0-byte values
20111031
* updated README.macosx
* updated libcstring, libsystem, libuna, libbfio, libfdatetime, libfvalue
20111030
* updated configure.ac and m4 files
* updated libfdatetime, libfvalue
* changes for libfvalue
20111008
* updated configure.ac and m4 files
* updated README files
* updated pc and spec file
* worked on compressed binary data support
* updated common, libsystem, libfdatetime
20111002
* improved space tree leaf page entry support
* worked on compression support
- added XPRESS support
- improved 7-bit support
* bug fix for tagged data value offset > 0x4000 in certain 0x620 revision 0x11
databases
20110927
* updated libfmapi
20110925
* updated libuna, libbfio, libfdatetime
* added 7bit ASCII compressed strings support
* added 7bit ASCII compressed multi value strings support
20110920
* code clean up
20110919
* updated configure
20110904
* updated configure
20110828
* updated configure, common, libsystem, libbfio, libfvalue,
* added support to ignore empty pages
* windows search added support for compression type 0x06
20110713
* updated msvscpp files
20110712
* fixes for export handle and windows search ASCII codepage
* windows search: fix for cut-off of 'compressed' UTF-16 stream in the surrogate range
* code clean up
* updated libfdata
* fixed unused variables warnings in libfdata and libesedb
20110711
* changes for: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call detected in body
* updated common, libcstring, log_handle.[ch], libuna, libbfio, libfdatetime,
libfmapi, libfvalue
* refactored export handle and esedbexport
* refactored esedbinfo to use info handle
* code clean up
20110509
* updated libesedb_libfdata.h
20110413
* updated libuna - initial windows 936 codepage support
* updated libcstring, libfdata
* updated msvscpp files
* worked on windows search codepage support
* code clean up
20110321
* updated msvsccp files
20110320
* updated configure.ac, libcstring, libsystem, libuna, libbfio, libfdata,
libfguid, libfmapi
* fix for revision 11 and < 16K paged files
20110122
* updated libfdata
20110121
* updated libfdata
20110120
* updated exchange_export_record_value_guid to handle non 16 byte values
20110114
* updated common, libcstring, libsystem, libuna, libbfio, libfdata, libfmapi,
libfvalue
* esedbtools: code clean
* optimization for non existent key lookup
* fix for long value segment key
20110107
* updated configure.ac
* updated libsystem, libfvalue
* some changes to exchange output conversion
20110103
* updated libuna, libbfio, libfdatetime, libfguid, libfwnt, libfmapi
* 2011 update
20101231
* updated configure.ac
* updated common, libcstring, liberror, libnotify, libsystem, libuna, libbfio,
libfdata, libfdatetime, libfvalue
* updated array type, list type
* code clean up
* fix for generation of libesedb.pc
20101119
* updated configure.ac
* updated libcstring, liberror, libbfio
* updated msvscpp - added missing libfmapi
20101113
* worked on streaming file support
* updated libfmapi
* textual changes
20101112
* updated libfvalue: optimization for codepage 1200 handling
* worked on support for windows search compression type 2
* fixed issue in debug output
20101111
* improved multi value support
* updated libfvalue
* esedbexport: worked on support for exchange
* fixed issue regarding incorrect long value segment offset
* esedbexport: reformats text with some escaped special characters
20101110
* updated configure.ac
* updated include/error.h
* updated include/types.h
* updated array type
* updated common, liberror, libnotify, libsystem, libbfio, libfdata, libfmapi
libfvalue
* worked on support for multi value with emtpy values
* added libesedb.rc for msvscpp build
20100919
* worked on Windows Security database support
20100916
* added support for handling space tree page: 0
* updated libcstring, libsystem, libuna, libbfio, libfdata, libfdatetime,
libfguid, libfmapi, libfvalue, libfwnt
20100903
* code clean up
20100807
* changed ThumbnailCacheId to hexadecimal representation
20100803
* updated libfdata
20100802
* updated libuna, libbfio
* improved codepage 1200 support
20100730
* worked on esedbexport export mode
* worked on gettext support
20100728
* worked on esedbexport export mode
20100727
* moved value debug print to libfvalue
* updated libfdatetime
* worked on index support
20100726
* worked on libfvalue integration
* small performance increase due to cache tuning
* updated msvscpp files
20100725
* updated common
* updated include/error.h and liberror
* updated libuna, libbfio, libfdata, libfdatetime
* split column definition and record value
* worked on libfvalue integration
20100715
* updated libuna
20100714
* added get_table_by_name function
* updated libuna
* fix for open/close/open
20100713
* configure.ac added stat check
* updated common, libsystem, libuna, libbfio, libfdata, libfdatetime,
libfmapi, libfwnt
* updated array and list type
* changes for updates
* fixed FDP mismatch issue
* updated msvscpp files
20100628
* update for libfmapi
20100627
* updated libuna, libbfio
* code clean up
* added libfmapi
20100626
* worked on big-endian support for aligned XOR-32
* updated macosx files
20100625
* code clean up
- removed API codepage functions, not needed
- worked on windows UTF-16 support
- added several UTF-16 API functions
- removed page tree values, data definition and tree type
- create catalog object
- create database object
* updated msvscpp files
* updated libfdatetime
20100624
* added multi value offset 0x8000 flag support
* esedbexport: added option to export specific table
* updated libfdatetime
* worked on libfwnt
20100623
* worked on libfdata tree integration
* added memory alignment support to XOR32 calculation
20100622
* worked on libfdata tree integration
20100621
* updated libfdata, libbfio
* updated array type
* worked on libfdata fixed list
20100529
* changed libdatablock into libfdata
* worked on libfdata
* fixes for non-debug output
* remove attach/detach functions
20100528
* worked on libdatablock
20100527
* worked on libdatablock
20100526
* worked on reading backup database (file) header
* fixed issue in page key lookup function
* support for additional Windows Search compressed values
* changes for handling root page flag
* changes for handling database with empty catalog
20100525
* fixed issues in page key lookup function
* added error tolerance for non-supported catalog types
20100524
* worked on storing page tree values as a hierarchy
* moved bfio handle to item layer
20100522
* added support for default value
- read from catalog
* clean up of long value page tree code
* worked on storing page tree values as a hierarchy
20100521
* worked on template table support
* fixed memory leak in libesedb_page_tree_read_leaf_page_values
* worked on large database support
- branch pages
20100520
* worked on large database support
- changed page number to 64-bit value
* fixed checksum calculation for new format, old page
* updated msvscpp files
* worked on template table support
20100519
* worked on Windows Vista Search support
* worked on long value support
20100518
* worked on long value support
20100517
* worked on long value support
20100516
* worked on Windows Search index-based compression
* worked on long value support
20100515
* worked on Windows Search index-based compression
* worked on long value support
20100514
* updated common, libcstring, liberror, libnotify, libsystem, libuna
libbfio, libfdatetime
* renamed libfwintype to libfguid
* changed amount of in number of
* moved bfio handle from io handle to file
* worked on long value support
20100407
* worked on refactoring file io handle out of io handle
20100402
* fixed non 4 byte modulus sized buffers issue in XOR32
20100328
* added libcstring
* updated liberror, libnotify, libsystem, libfdatetime, libuna, libbfio
* removed common/narrow_string.h common/wide_string.h
* worked on libcstring support
20100302
* worked on windows 7 engine support
20100301
* worked on windows 7 engine support
20100227
* worked on windows 7 engine support
* enlarged filetime string to 24
20100224
* worked on windows 7 engine support
20100223
* worked on windows 7 engine support
20100222
* 2010 update and email change
* updated common, liberror, libnotify, libuna, libbfio, libsystem
* updated array, list and tree type
* worked on libnotify update
* worked on windows 7 engine support
20091130
* worked on long value support
20091129
* worked on Windows Search
* worked on multi value support
* worked on long value support
* worked on file format
20091127
* worked on Windows Search
20091125
* worked on file format
20091124
* worked on file format
- data type definition with fixed and variable size values
without corresponding definition
- worked on long value support
* fix for reading files without debug mode
20091123
* worked on file format
* changes for MSVSCPP compilation
20091121
* worked on support of Windows Search Vista database
- added ASCII 7-bit compression support
* fixed error regarding variable size data type size
20091120
* worked on file format
* fixed ECC for 32-bit architectures
* worked on support of Windows Search Vista database
* fixed memory leak in clean up of index page tree
20091119
* worked on file format
* worked on ECC
* fixed possible memory leak on error
* worked on handling ASCII 7-bit compression
- additional findings regarding leading byte 0xa0
- now handles extended ascii strings with leading byte 0x01
20091118
* worked on file format
20091116
* worked on file format
20091115
* updated common, libuna, libbfio, liberror, libnotify, libsystem
libdatetime
* removed All rights reserved
* updated codepage header
* removed item reference lists and list elements
* replaces endian.h by byte_stream.h
* updated float conversions
* worked on support for callback catalog type
20091013
* some textual corrections
20091012
* change for version 0x620 revision 0x0b
20091009
* worked on EDB revision 2 version of tagged data type definitions
20091008
* added support for empty space tree header
* added support for default value of tagged data type definition
* improved column identifier check in data type definition
* worked on EWF revision 2 version of tagged data type definitions
20091007
* added column get type
* changed output of esedbinfo to print column type
20091006
* worked on page tree support
20091005
* added support for parent pages
20091004
* added missing libesedb_types.h to dist
20091003
* updated libbfio
* corrected license in several of the esedbtool handles
* updated acinclude
20091002
* worked on exporting Windows Search database
* worked on record API
20091001
* worked on page values and data definition types (dynamic version)
* worked on esedbexport
* worked on record API
20090930
* fixed possible memory leak of libbfio handle
* worked on page values and data definition types (dynamic version)
* worked on esedbexport
* worked on record API
20090927
* updated libbfio and libsystem
20090925
* worked on page values and data definition types
20090922
* worked on page values and data definition types
20090921
* worked on page values and data definition types
20090920
* worked on page values and data definition types
20090919
* worked on page values and data definition types
* worked on reading the catalog
* esedbinfo now shows
- version, revision
- creation version, revision
- page size
- table names
- index names
20090918
* worked on page values and data definition types
20090917
* worked on page trees
* worked on page values and data definition types
* refactored creation of array out of libesedb_page_read_tags
20090916
* worked on unknown values in file header
* worked on page values and data definition types
20090915
* worked on table support
* implement XOR-32 checksum support
* worked on unknown values in file header
* worked on page values and data definition types
20090914
* worked on page trees
* worked on page values and data definition types
20090913
* initial version based on liblnk
|