Package: future.batchtools
==========================

Version: 0.10.0 [2021-01-02]

SIGNIFICANT CHANGES:

 * Lazy batchtools futures only creates the internal batchtools registry when
   the future is launched.

 * Removed S3 generic functions await(), finished(), and status(), which were
   functions that were used for internal purposes.

DOCUMENTATION:

 * Document option 'future.delete' and clarify option 'future.cache.path'
   in help("future.batchtools.options").

BUG FIXES:

 * If run() was called twice for a BatchtoolsFuture, it would not produce
   a FutureError but only a regular non-classed error.

DEPRECATED AND DEFUNCT:

 * Removed S3 generic functions await(), finished(), and status(), which were
   functions that were used for internal purposes.


Version: 0.9.0 [2020-04-14]

SIGNIFICANT CHANGES:

 * The default number of workers on HPC environments is now 100. To revert
   to the previous default of +Inf, see below news entry.
 
NEW FEATURES:

 * It is now possible to configure the default number of workers on the job
   queue of an HPC scheduler via either R option 'future.batchtools.workers'
   or environment variable 'R_FUTURE_BATCHTOOLS_WORKERS'.

 * It is now possible to configure the batchtools registries that are used by
   batchtools futures via new argument 'registry' to plan().  This argument
   should be a named list of parameters recognized by the 'batchtools' package,
   e.g. plan(batchtools_sge, registry = list(...)).  For notable example, see
   below news entries.

 * The default working directory for batchtools futures is the current working
   directory of R _when_ the batchtools future is created.  This corresponds to
   specifying plan(batchtools_nnn, registry = list(work.dir = NULL).  Sometimes
   it is useful to use a explicit working directory that is guaranteed to be
   available on all workers on a shared file system, e.g. plan(batchtools_nnn,
   registry = list(work.dir = "~")).

 * It is possible to control if and how batchtools should use file compression
   for exported globals and results by specifying batchtools registry parameter
   'compress'.  For example, to turn off file compression, use
   plan(batchtools_nnn, registry = list(compress = FALSE)).

 * The default location of the '.future' folder can be controlled by R option
   'future.cache.path' or environment variable 'R_FUTURE_CACHE_PATH'.

 * batchtools_custom() and BatchtoolsFuture gained argument 'conf.file'.
   Using plan(batchtools_custom) will now use any batchtools configuration
   file (an R script) found on the batchtools::findConfFile() search path.

DOCUMENTATION:

 * Add help("future.batchtools.options") which descriptions R options and
   environment variables used specifically by the future.batchtools package.
   

Version: 0.8.1 [2019-09-30]

BUG FIXES:

 * print() for BatchtoolsFuture would produce an error if the underlying
   batchtools Registry was incomplete.


Version: 0.8.0 [2019-05-04]

NEW FEATURES:

 * Setting option 'future.delete' to FALSE will now prevent removal of the
   batchtools registry folders.

 * When a batchtools job expires, for instance when the scheduler terminates
   it because the job was running out of its allocated resources, then a
   BatchtoolsFutureError is produced which by default outputs the tail of the
   output logged by batchtools.  The default number of lines displayed from
   the end is now increased from six to 48 - a number which now can be set
   via option 'future.batchtools.expiration.tail'.
   
 * Now a more informative error message is produced if a batchtools *.tmpl
   template file was not found.

 * Debug messages are now prepended with a timestamp.

BUG FIXES:

 * Argument 'workers' could not be a function.

 * Argument 'workers' of type character was silently accepted and effectively
   interpreted as workers = length(workers).


Version: 0.7.2 [2018-12-03]

DOCUMENTATION:

 * Add a simple example(future_custom).

FIXES:

 * Made internal code agile to upcoming changes in the future package on
   how a captured error is represented.

SOFTWARE QUALITY:

 * FYI: Every release is tested against one Torque/PBS and one SGE scheduler.
 
BUG FIXES:

 * resolve() on a lazy batchtools future would stall and never return.
 

Version: 0.7.1 [2018-07-18]

NEW FEATURES:

 * The batchtools_* backends support the handling of the standard output as
   implemented in future (>= 1.9.0).
 
BUG FIXES:

 * A bug was introduced in future.batchtools 0.7.0 that could result in "Error 
   in readLog(id, reg = reg) : Log file for job with id 1 not available" when
   using one of the batchtools backends.  It occurred when the value was
   queried.  It was observed using 'batchtools_torque' but not when using
   'batchtools_local'.  This bug was missed because the 0.7.0 release was not
   tested on an TORQUE/PBS HPC scheduler as it should have.


Version: 0.7.0 [2018-05-03]

NEW FEATURES:

 * Argument 'workers' of future strategies may now also be a function, which
   is called without argument when the future strategy is set up and used as
   is.  For instance, plan(callr, workers = halfCores) where
   halfCores <- function() { max(1, round(availableCores() / 2)) } will use
   half of the number of available cores.  This is useful when using nested
   future strategies with remote machines.

CODE REFACTORING:

 * Preparing for futures to gather a richer set of results from batchtools
   backends.


Version: 0.6.0 [2017-09-10]

NEW FEATURES:

 * If the built-in attempts of batchtools for finding a default template file
   fails, then system("templates", package = "future.batchtools") is searched
   for template files as well.  Currently, there exists a `torque.tmpl` file.

 * A job's name in the scheduler is now set as the future's label (requires
   batchtools 0.9.4 or newer).  If no label is specified, the default job name
   is controlled by batchtools.

 * The period between each poll of the scheduler to check whether a future
   (job) is finished or not now increases geometrically as a function of number
   of polls.  This lowers the load on the scheduler for long running jobs.

 * The error message for expired batchtools futures now include the last few
   lines of the logged output, which sometimes includes clues on why the future
   expired.  For instance, if a TORQUE/PBS job use more than the allocated
   amount of memory it might be terminated by the scheduler leaving the message
   "PBS: job killed: vmem 1234000 exceeded limit 1048576" in the output.
   
 * print() for BatchtoolsFuture returns the object invisibly.

BUG FIXES:

 * Calling future_lapply() with functions containing globals part of non-default
   packages would when using batchtools futures give an error complaining that
   the global is missing. This was due to updates in future (>= 1.4.0) that
   broke this package.
 
 * loggedOutput() for BatchtoolsFuture would always return NULL unless an error
   had occurred.


Version: 0.5.0 [2017-06-02]

 * First version submitted to CRAN.
 
SOFTWARE QUALITY:

 * Added more tests; test coverage now at 93%.
 

Version: 0.4.0 [2017-05-16]

NEW FEATURES:

 * Added batchtools_custom() for specifying batchtools futures using any type
   of batchtools cluster functions.
   
 * batchtools_template(pathname = NULL, type = <type>) now relies on the
   batchtools package for locating the <type> template file.
   
 * nbrOfWorkers() for batchtools futures now defaults to +Inf unless the
   evaluator's 'workers' or 'cluster.functions' specify something else.

 * Renamed argument 'pathname' to 'template' for batchtools_<tmpl>() functions.

BUG FIXES:
   
 * Under plan(batchtools_*), when being created futures would produce an error
   on "all(is.finite(workers)) is not TRUE" due to an outdated sanity check.

SOFTWARE QUALITY:

 * TESTS: Added test of future_lapply() for batchtools backends.

 * TESTS: Added optional tests for batchtools_* HPC schedulers listed in
   environment variable 'R_FUTURE_TESTS_STRATEGIES'.

CODE REFACTORING:

 * CLEANUP: Package no longer depends on R.utils.

 
Version: 0.3.0 [2017-03-19]

NEW FEATURES:

 * The number of jobs one can add to the queues of HPC schedulers is in
   principle unlimited, which is why the number of available workers for such
   batchtools_* backends is reported as +Inf.  However, as the number of
   workers is used by future_lapply() to decide how many futures should be used
   to best partition the elements, this means that future_lapply() will always
   use one future per element.  Because of this, it is now possible to specify
   plan(batchtools_*, workers = n) where 'n' is the target number of workers.


Version: 0.2.0 [2017-02-23]

GLOBALS:

 * batchtools (>= 0.9.2) now supports exporting objects with any type of names
   (previously only possible if they mapped to strictly valid filenames).
   This allowed me to avoid lots of internal workaround code encoding and
   decoding globals.
   

Version: 0.1.0 [2017-02-11]

 * Package created by porting the code of future.BatchJobs. This version passes
   'R CMD check --as-cran' with all OK after a minimal amount of adjustments
   to the ported code.