$Id: ChangeLog,v 1.50 2005/04/18 02:25:45 hexwab Exp $

OpenFT 0.2.1.6:

   * Fixed a bug where full parents could trigger an assertion failure.

   * Index tokens database by array index instead of by pointer
     (should improve space efficiency, especially on 64-bit platforms).

   * Clean up the database properly.  Not only does this avoid leaving
     huge temporary files lying around, but potentially allows the db to be
     shutdown and restarted at will.

   * Avoid unaligned memory accesses, which cause problems on some
     RISC platforms.

   * Don't die on malformed URLs.

OpenFT 0.2.1.5:

   * Major search database rethink: shares are keyed by host plus a
     unique identifier instead of by hash.  Hopefully this will improve
     space efficiency (and reduce I/O), and also eliminate the (non-)handling
     of duplicate hashes that existed previously.

   * New tokenization API.

   * Removed support for hidden searches due to new improved hashing algorithm.

   * New searching functionality: exclusion tokens and quoted queries.

   * Moved local shares into search database.  This should reduce CPU usage
     with a large number of shares.

   * Improved documentation about search node configuration.

   * Fixed two bugs in stats initialization.

   * Added Bloom filter support, which is currently unused but may eventually
     be used to optimize query routing.

   * Temporarily backed out of cleaning up the database environment on exit,
     because it would try to sync the entire database to disk.  This will be
     fixed properly later when individual children are tracked within
     ft_search_db.c.

   * Implemented automatic compressed stream handling for unbounded
     data (currently search queries and forwarded search results).  This
     reduces bandwidth usage significantly, sometimes by as much as 50%.

   * Increased maximum packet size, although this is currently unused for
     compatibility reasons.
	
OpenFT 0.2.1.4:

   * Fixed a signedness bug that could cause problems with non-ASCII characters
     on some platforms.

   * Fixed a packet handling bug that could cause an infinite loop.

   * Minor networking tweaks.

   * Fixed clipping of the nodes cache.

OpenFT 0.2.1.3:

   * Fixed a potentially fatal problem when parsing an HTTP request header
     which fails to provide a request method.

   * Fixed a bug that could result in a remote crash when supplying a relative
     path in HTTP GET and HEAD request methods.

   * Rewrote the ZLib decompression code to [hopefully] fix a bug which
     can potentially leave OpenFT in an infinite loop (very rare).

   * Fixed a problem when requesting additional nodelists from actively
     connected peers which would produce an invalid request packet, and thus
     never receive nodes.

   * Corrected an invalid assumption when determining the "indirect" status of
     remote peers that expected both port_openft and port_http to be positive
     integers.  The HTTP port is not transmitted by nodelist responses, and
     thus will be initialized to 0.  This subsequently prevents connection
     attempts to these peers.

   * Fixed ft_netorg_length cache corruption.

   * Fixed search db filename problem which caused SEARCH nodes to be unusable
     on Windows.

OpenFT 0.2.1.2:

   * Fixed a problem that would improperly cast the result of recv() to size_t
     before testing for a negative return value.

   * Improved URL encoding correctness.  This code is likely to be phased out
     entirely with the introduction of giFT 0.12.x, *sigh*.

   * Added HEAD request method support.  This is a server-side only change and
     is not currently being utilized by this code base.

   * Minor internal cleanups regarding the use of `FT_SELF' as a valid
     FTNode object.

   * Major logic improvements regarding class changes.

   * Turned off incoming search verification by default.

   * Massive search database redesign and internal code cleanups.  Added
     an additional non-persistent index for efficient removal (similar to
     what was used before but greatly cluttered the memory pool cache and
     persisted for the life of the child connection).  This improvement
     manifests itself in the form of a drop of n file descriptors actively
     opened where n is the number of children the search node is currently
     managing.

   * Added a simple test-case system for benchmarking and testing
     ft_search_db.c tweaks.

   * Search node defaults have been adjusted to better facilitate the
     increased search node distribution and lower the system requirements for
     more casual search nodes.

   * Minor search node optimization which avoids port verification for nodes
     that can't possibly become our child (even if they think they can).  This
     optimization should eventually be built into the network by allowing
     users to dynamically adjust their firewalled status based on the
     verification results from remote search peers.

   * Automatically detect and adjust the fd limit of the process space
     OpenFT is running in using getrlimit/setrlimit when available.  This
     should greatly improve some operating system's ability to function as
     a search node.

   * Avoid a fatal assertion when giFT [mistakenly] calls p->download_start
     consecutively, with no p->download_stop in-between.

   * Major win32 portability improvements.  Thanks pretender :-)

OpenFT 0.2.1.1:

   * Move libgiftproto version requirements up to 0.11.4 for the changes to
     the p->upload_auth system.

   * Improved default nodes file from 0.2.1.0.

   * Optimized ft_netorg_length to use a constant iteration of 31 (total
     distinct possibilities for the klass fields) instead of moving over every
     node that currently exists in the state set.

   * Added a feature to clamp the maximum number of OpenFT connections that
     will be allowed.  By default, this feature is only active for search
     nodes and it is intended to keep them away from the default maximum fd
     limit of 1024 on most systems.

   * Fixed a small bug which would allow search nodes to release peers when
     the purpose value was not properly assigned.

OpenFT 0.2.1.0:

   * Deliver the proper nodelist sentinel after the complete command has been
     sent.  Should we maybe use a generalized stream here?

   * Fixed a typo which would result in acceptance of a new parent despite
     the precense of a previous one.  The remote peer is sent a rejected
     response while the local node adds a parent class, thus creating an
     inconsistency.  Eventually this leads to the local node dropping
     legitimate parent nodes and then seeking out new parents to start the
     process all over again.

   * Improved search result filtering to firewalled users, specifically when
     the search result would not be accessible to the user receiving the
     results.  You could reason that this will reduce search result traffic by
     about 25% due to the frequency of searches from firewalled users and also
     the number of firewalled users on the network.  Although, I don't believe
     this to be entirely accurate.  Still, it's an improvement *g*.

   * Fixed a small memory leak which would occur for each HTTP request which
     returned the 503 HTTP response (no available upload positions, generally).

   * Handle the new UPLOAD_AUTH_HIDDEN condition from p->upload_auth.  Also
     improved the logic which responds to 503 messages so that only
     UPLOAD_AUTH_MAX will send the queue position.

   * Fixed a severe search database bug which would cause all single share
     removals (such as when resyncing shares) to fail to be removed from the
     secondary token index which resulted in the need for the SEARCH_DB_HACK
     switch.  Lingering references to the share database would always exist
     and the size of the secondary token index would grow indefinitely,
     causing performance and resource management issues.

OpenFT 0.2.0.0:

   * Internal improvements which allow idle connections to be cleaned up on
     the client side more gracefully.  This system will need to be implemented
     on a case-by-case basis, and is currently in use only by a small number
     of systems.

   * Modified the PUSH forward request/response such that a response is
     always sent back, even when successful.  This way we can be sure to
     terminate the session as soon as we can be assured that the packet was
     delivered.

   * Fixed several bugs which would cause regular user nodes to enter a state
     of desperation when searching for nodes, even when they have all the
     necessary connectivity.

   * Automagically clamp the number of nodes written out to the cache to avoid
     ridiculous startup times trying to read and sort them back in.

   * Fixed a fatal assertion failure when responding with a node list that
     exceeds 2048 bytes, triggered on search nodes when the external
     OpenFT spider runs.

   * Merged nodelist request and response into a single command for
     efficiency.

   * Fixed a fatal error with libdb3 >= 3.2.x.

OpenFT 0.1.0.5:

   * Fixed a bug where search nodes would try a new outgoing connection
     when a push forward request was made.  The user must be connected for
     this operation to have any meaning.

   * Reduced the amount of stale PUSH sources that are allowed by killing the
     source when the search node parent that is handling the forwarding says
     that the user is no longer available as a child.

   * Fixed a resource mismanagement bug which would request a nodelist from
     remote peers even when all connection types are satisfied.

   * Fixed a fatal bug when searching and no parent nodes are available.

OpenFT 0.1.0.4:

   * Fixed a problem where old nodes would stll prefer outgoing connections to
     newer incompatible versions, effectively flooding them with erroneous
     connection attempts.

   * Reduce the amount of nodelists that are requested from remote peers in
     an effort to reduce search node bandwidth requirements.

   * First official version change which uses the a.b.c.d version identifier,
     instead of a.b.c-d, to reduce confusion with the Debian packages.

OpenFT 0.1.0-3:

   * Added a special share (GET /nodes...) which will dynamically construct a
     file containing references to all active OpenFT connections.

OpenFT 0.1.0-2:

   * Completely rewritten HTTP transfer system.  During this rewrite OpenFT's
     nodepage was put to rest.

   * Minor non-backwards compatible change to the PUSH system used by
     firewalled nodes during uploading.

OpenFT 0.1.0-1:

   * No real changes, only a version change for this release candidate.

OpenFT 0.0.10-1:

   * Completely eliminated the need for FTSHost, moving the few required
     facilities it provided to ft_search_db.h:FTSearchDB.

   * Major search database optimizations resulting from direct storage of the
     FTSearchDB pointer instead of relying on a lookup by IP in order to
     complete the result set.  Needless to say these are highly experimental.

   * Rearranged protocol commands related to sharing and stats.  Specifically,
     sync messages are used to indicate when modifications to the shares will
     be made, this replaces one part of the modshare_request command.  Added
     FT_CHILD_PROP message to adjust availability and hopefully other child
     properties in the future.  Divided FT_STATS_REQUEST such that it no
     longer implemented any form of sub-commands, added
     FT_STATS_DIGEST_{ADD,REMOVE} to replace the functionality.

   * Improved the stats semantics such that parent search nodes will archive
     previous index node replies and deliver them to children nodes who
     request. This means that index nodes will no longer be bound to the base
     user class, opting instead for a better defined hierarchy.

   * Many legacy cleanups.

   * Take advantage of the new FT->share_lookup (FT, ...) command.  In the
     very near future a new command to access the entire share index (for
     ft_share_local_submit) will be added as well.

OpenFT 0.0.9-7:

   * Actually, nothing.  It's a trick, but update anyway *g*.

OpenFT 0.0.9-6:

   * Improved FT_NODELIST_REQUEST so that specific classes may be filtered
     by the remote node in order to produce visual graphs by spidering the
     network.  Please see the openft-map module from this CVSROOT if you'd
     like to know exactly how we are doing this...

OpenFT 0.0.9-5:

   * Major search efficiency improvements.  Temporarily removed MD5 search
     forwarding.

   * Reintroduced the concept of a search termination sentinel.  Each node
     in the network will wait on all forwarded nodes to respond before it
     delivers the sentinel so that the original searching node may elegantly
     terminate the search before the timeout hits.  This feature depends on
     all other nodes sending the sentinel in the proper order, and as such
     is only moderately backwards compat to 0.0.9-4.

OpenFT 0.0.9-4:

   * Redesigned push requests and modified search results to properly maintain
     the original search parent for push purposes.

OpenFT 0.0.9-3:

   * Modified the search response headers to track the original parent.  This
     will be used to fix the push request problem introduced with the virtual
     cluster design.

   * Modified FT_NODELIST_{REQUEST,RESPONSE} so that only one packet is
     exchanged for each direction (instead of one per node exchanged).

OpenFT 0.0.9-2:

   * Search enhancement using an id that can be considered unique for all
     nodes operating on the search.  This is used for a stronger (and simpler)
     system of search forwarding that can preserve as much of the original
     data as possible, as well as detect "loops" in the query system.

OpenFT 0.0.9-1:

   * Major high-level redesign begun.  Initial changes include the formation
     of virtual clusters through a configurable number of static peer search
     node connections.  Searches are broadcasted to all peers, which then
     rebroadcast the search until TTL expires, or the maximum result set is
     reached.  Several optimizations are planned to improve efficiency in a
     large scale network.

   * Migrated the code over to the new logging facilities provided through the
     Protocol structure.

OpenFT 0.0.8-2:

   * More improvements to the search database design.  We are now utilizing a
     single database environment and all child shares are inserted into the
     same physical file on disk.

   * Tested and [almost] finished direct browsing.  Currently,
     ft_packet_sendto is limited to only being able to deliver packets to
     non-firewalled users.  Hopefully this will be addressed soon.

OpenFT 0.0.8-1:

   * Massive scalability improvements related to the search node database code
     which organizes and indexes all child shares.  For the more technical of
     our audience, we have moved to a database design utilizing two master
     databases for all searching requirements (greatly increasing insert and
     removal overhead) while reducing the number of file descriptors open at
     any one time and total memory overhead.

   * Rewrote node, network organization, and connection subsystems for OpenFT.
     While it has little effect on the network structure directly, I will
     begin to utilize these new APIs through implementation of node
     clustering.  Honestly, it's coming soon :)

   * Untested "fix" for direct user browsing.  See 0.0.7-8's entry for more
     details on this.

OpenFT 0.0.7-8:

   * Direct browsing implemented.  Please note that this code will currently
     only work if you already have an OpenFT connection with the user you are
     browsing.  This limitation is strictly client-side and should be
     considered a "bug".  Will be fixed ASAP.

OpenFT 0.0.7-7:

   * Fixed session handshaking...I know, shut up.

OpenFT 0.0.7-6:

   * Fixed session handshaking for real this time...I promise.

OpenFT 0.0.7-5:

   * Fixed a bug which ignored the need to test supplied ports.  This caused
     nodes which have been improperly configured (port = non-zero yet it is
     not remotely accessable, likely because of a firewall) to report as
     non-firewalled users and reply with a URL as such.  It would thus be
     impossible to download from these users.

   * Fixed a somewhat related bug which caused session handshaking to
     prematurely complete.  Fixing this bug causes a minor disruption in
     handshaking with 0.0.7-4, which worsens the race condition in 0.0.7-4.

OpenFT 0.0.7-4:

   * Deliver node aliases over HTTP and display appropriately for uploads
     registered this way.

OpenFT 0.0.7-3:

   * Fixed a minor endianness problem when communicating IPv4 addresses over
     OpenFT.

OpenFT 0.0.7-2:

   * Public "release" of OpenFT 0.0.7.

OpenFT 0.0.7-1:

   * Transparent stream compression over OpenFT.  This system introduces a
     a bandwidth optimization which greatly reduces the number of OpenFT
     packets required for writing (by blocking multiple writes into one packet)
     as well as transparently compressing the writes with ZLIB if available.
     Currently, the non-zlib code is somewhat broken so ZLIB is a requirement.
     This will be lifted soon.

   * Strict protocol "handshaking" stage requiring all nodes deliver the
     required information/packet responses before the stage increases and the
     true OpenFT session begins.

   * Completely removed old method of browsing due to a severe waste of
     resources by design.  A new direct browse method which operates over
     OpenFT using stream compression is expected to be completed shortly.

OpenFT <= 0.0.6:

   * Information lost.