NEWS about R package opt

TODO
      o trig1507.R issue of bounds to avoid multiple minima, note ttrig8b lbfgsb3 etc.
      o Put counters in efn, egr
      o General issue: nlminb and possibly nlm not working as Newton with 
	explicit Hessian i.e., may "work", but seemingly not like Newton
      o Should add examples, esp. of use of summary.
      o Rationalize tests, examples, unit tests and demos.
      o can we add bounds and masks easily to Newton methods?
      o Drop nmkb from methods for bounds if start is on bound
      o Add nloptr methods -- there are a number and we need to choose how
	to do so. Seems unable to handle parameter scaling or dot arguments.
        nloptr difficult to incorporate -- for now left outside of optimr()
        Troubles with NAMESPACE collisions NOT resolved for nloptr
      o Consider how the hyperbolic transformation (transfinite fn)
        could be used in all unconstrained methods
        How can transfinite be made to work when start is on bounds?
	To work when end is on bounds?
      o Rcgmin seems to report reset of parameters to bounds when parameter
        is apparently on the bound. Could have rounding issues, and should
        avoid this warning if it is really not needed.
      o Do NOT add ipoptr until non-standard install is fixed up.
      o Check all failures reported below are fixed! (As at 2018-03-27
	JN believes they are.)

===================================================================================
NEWS from the optimx package

Some comments apply to deprecated packages optimr on CRAN and optimrx on R-forge

VERSION 2020-04-02
      o Repair maximization error in optimr()

VERSION 2020-01-18
      o This version does not call some non-CRAN packages.

VERSION 2019-12-02
      o Replaced all instances of tests on class() with inherits()
      o Replaced calls to lbfgsb3 package with those to lbfgsb3c,
        and removed calls to the lbfgsb3() function
      o Needed to set fval to NA in fnchk when length > 1

VERSION 2018-07-10
      o Added proptimr() compact display of result of optimr
      o Some minor changes to vignettes and to message (msg) output for 
        line ending

VERSION 2018-07-06
      o Changed setup of control vector using ctrldefault to allow for use of
	individual method defaults

VERSION 2018-04-17
      o Comments in tests etc. to explain why methods fail
        Add hessian to simple test
      o Split optim() call in optimx() based on have.bounds. Note glitch in 
	optim() for vector vs scalar infinite bounds.
      o Fix Rvmmin for n=1 infinite loop      

VERSION 2018-03-30
      o Cleanup of output from various solvers and fix some minor omissions
	and errors in the returned information
      o Conditional output of nlm diagnostic info.
      o Changed Rvmmin call. Rvmminb and Rvmminu NOT now called separately,
	as user may inadvertently supply inadmissible parameters and get wrong
	answer.
      o Changed optimr() so nlminb() can use ehess() function
      o Replace convcode in optimr() with convergence to align with optim() result
      o axsearch() example added to bdstest.R in tests.
      o Merged optextras package into optimx
      o rationalized  optimz-package.R, optimx-package.R into one set of code
      o Tests for grchk, kktchk, fnchk included
      o Fixed unconditional printout of maxit
      o savehess saves (approximate!) hessian with nlminb, call with 'hess' not 'hessian'

VERSION 2018-03-18
      o added 
           mcontrol$fnscale <- control$fnscale
        in optim() section of optimr for correction of maximization 
        Thanks to Carlo Lapid cmlapid@up.edu.ph
      o Thanks to playing with examples from Hans Werner Borchers for Nelder-Mead
        style codes, found that optimr() did not transfer control$maxit to the
        optim() functions. Fixed.

2018-03-03
  o update snewton vignette and tests
  o sorted out optimr nlminb failure (used hessian vs hess in optimr call)

VERSION 2017-12-15
      o merging optimr(x) into optimx
      o some documentation of differences between opm() and optimx()
      o Use pkg::fn style calls in optimx.run.R
      o add bib references to vignette
      o checked method="ALL" and control(all.methods=TRUE) are equivalent for opm()
      o set trace=0 in all tests
      o partially simplify tests


VERSION 2017-11-25 optextras
   o    Changed kktchk argument maxfn to maximize to agree with 
        optimr and other packages


VERSION 2017-10-05
      o Ravi Varadhan as maintainer (reverted later)
      o Warning about combining options.

VERSION 2017-09-24
      o comments include 2017-09-18 version
      o brought optimx solvers into optimr and added check that packages 
        are available
      o Fix for lbfgsb3 which was changed at same time to correct and
        simplify some control parameters

2017-04-20
  o unconstrained method is working. Several tests, but Hobbs
    gives a problem from 1, 1, 1

VERSION 2017-04-04
      o allowed hessian in opm() and optimr() (for nlm)
      o Thanks to Marie Boehnstedt for catching error in creation of
        the composite function and gradient for calling nlm() method.

2017-04-10
  o Initial setup of project with trial codes. No manuals

VERSION 2017-02-15
      o added subplex method

VERSION 2016-09-16
      o Harry Joe reported troubles with numerical gradient approximations. Found
           some glitches and conflicts to fix. Also documentation is less than ideal.
        Scaling spar to par was INSIDE control$trace

VERSION 2016-08-26
      o Thanks to Henrik Singmann for spotting a bug in optimx.run.R
        for the case where nlminb fails. I had missed resetting some
        elements of the solution object in this case. ??check optimr()/opm()

VERSION 2016-07-10
      o hessian=TRUE is NOT properly executed ??
      o Test function hobbs15b illustrates different scalings across all methods
      o Appropriate lists of "ALL", "bounded" and "masked" methods are defined 
        in the ctrldefault() function.
      o jonesrun L-BFGS-B fails with numerical gradient approximation.
      o dfoptim hjkb was fixed by Ravi Varadhan

VERSION 2016-07-03
      o Move allmeth, bdmeth, maskmeth into ctrldefault so we have just in
	one place. Note that this will change with optimr/optimrx
      o Masks for hjn(), and incorporate hjn in optimr
      o time (xtimes) removed from optimr, but timing is included in opm()
      o polyalgorithm wrapper polyopt() included and documented with example
      o multistart wrapper multistart() included and documented with example
      o renamed coef.optimx to coef.opm

VERSION optextras 2016-06-30
   o    Streamlined the package to get rid of ufn, ugr, uhess, 
        ugHgenb that are so far unused. 
   o    The optsp environment is only currently used
        with the simple numerical gradient approximations.
   o    kktc renamed to kktchk. scalecheck renamed to scalechk
        for consistency with other naming.

VERSION 2016-06-28
      o Checked that controls do not conflict, e.g., maximize and fnscale. Note that
        maximize is NOT in optim() call.
      o Checked and fixed scaling of bounds in optimr. However, hjkb seems to fail
      o Creates scbdtest.R,  a test function that is simple but bounded and scaled badly
      o Get routines from optextras rather than have duplicates in optimr
      o rename package to optimr for optimR(eplacement)
      o Problems with lbfgs package (unconstrained) resolved
      o test parameter scaling (but this could be improved)
      o Demos still refer to optimx -- must be changed -- 160628 moved 
        off to Roptimtest/current/


VERSION 2016-05-30
      o Crude setting of uobyqa and newuoa for control rhobeg.
        Especially when parameters all 0, or bounds undefined, program 
	will fail so we need to set this. May also be needed for rhoend.

VERSION 2016-01-07
      o realized control rhobeg needed to be set or bobyqa would fail.

VERSION 2015-06-03
      o Added index to bstate in optimx.check.R to avoid unwanted 
        elements in reporting bounds when trace > 0

VERSION 2015-04-23
      o removed condition that made solution NA if gradient or function
        limits exceeded. Now returns parameters and function at the point
	where limit reached.

VERSION 2015-01-21
      o Added lbfgsb3, the 2011 version of this code.

VERSION 2013-10-27 optextras
   o    Inserted missing index in bmchk (bdmsk)

VERSION 2013-9-11 optextras
   o    Fixed some elements of manual. Alias optextras-package
        to optextras.

VERSION 2013-8-20 optextras
   o    optsp scratchpad environment added. Note that it
        is aliased in grfwd to avoid a warning in R CMD
        check.
   o    Can change this in numerical derivative routines
        in case of parallel running.

VERSION 2013-08-08
      o Attempt to sort out Rvmmin and Rcgmin no gradient failures. 
        Resolved by REQUIRING a gradient routine. In optimr() can provide
        an approximation via quoted gradient function name.
      o Fix for nlm calls which sometimes failed due to inappropriate
        control settings and failure to clear (NULL) temporary settings.

VERSION 2013-08-07
      o Moved optimizer packages to Imports and removed require() calls to
	satisfy R 3.1.2.
      o This version is artificial! It is the version of 2013-08-06 with an 
	updated CITATION and minor fixes to satisfy NAMESPACE requirements.
	A new optimx is in process with quite sweeping changes that may or
	may not result in changed behaviour and usage. (JN 2014-09-04).
      o Failure to have the following packages installed may give rise to 
	diagnostic messages that can be confusing: Rvmmin, Rcgmin, ucminf,
	minqa, BB, and dfoptim.

VERSION 2013-08-02

      o Change startup to avoid warning messages for missing methods.
        Add function to check which methods available.

      o UOBYQA is present in optimx.run.R, but is not included in the list of
        "all.methods" in optimx.setup.R. It should be possible to run UOBYQA
        by including "uobyqa" in the list of methods in a direct call to optimx.run.R.

VERSION 2013-07-09

      o patch for missing "details" behaviour

      o point to example of use of "follow.on" control 

VERSION 2013-07-03 

      o added control usenumDeriv

VERSION 2013-06-04 

      o improved coef() function to extract parameters

VERSION 2013-05-03

      o removed optansout.R, get.result.R, get.best.R (no longer needed)

VERSION 2013-04-30

      o added optimx<-.coef

VERSION 2013-04-29

      o removed help pages documenting obsolete features

VERSION 2013-04-06

      o nmkb and hjkb had no ... arguments in call. Fixed.

      o L-BFGS-B (and possibly other optim() methods) return no gevals (count[[2]])
        on failure. Forced to NA in case of try-error.

VERSION 2013-04-05

      o Fixed maximize of function without gradient function. Also glitch
        (not tested or encountered) with user supplied Hessian function when
        maximizing.

      o Separate .Rd files for coef() and summary()

      o Use have.bounds to select bounds and unconstrained methods where there
        is a choice to avoid duplication of tests. This has been done for 
	hjk, nmk, Rcgmin, Rvmmin

      o Revision of output description in optimx.Rd; some simplifications.

      o Parameter names preserved from starting vector, or "p1", "p2", etc. used.
        There is a test for these in the ox demo().

VERSION 2013-04-03

      o in summary.optimx no re-ordering is now specified using order = NULL

      o Fixup of examples -- simplified, corrected call to details

VERSION 2013-04-02

      o in summary.optimx the order= argument may now be an unevaluated 
        expression (as before), a string representation of that expression or 
        a character vector of names, e.g. these all work:
           order = value # unevaluated expression
           order = "value" # character string containing column name
	   order = list(fevals, gevals) # unevaluated list
           order = c("fevals", "gevals") # character vector of column names
           order = list(round(value,3), fevals) # unevaluated list
           order = "list(round(value,3),fevals)" # expression as string

VERSION 2013-03-29

      o Note that asking for a method that is NOT in the results will return
        a row with rowname (the method name) as "NA". Note that this will NOT
        test TRUE with is.na() on the rowname. Use instead a test like

               (row.names(ans8missmeth["Rvmmin",])[[1]] == "NA")

      o kkt1 and kkt2 in results now returned TRUE/FALSE as in 2012 versions

      o Fixed error in kkt2 test (using old name evnhatend rather than hev)

      o Results are no longer sorted by optimx, but can be using the 'summary()'
        function. This can be called in a variety of ways. ?? see examples in??

      o The 'details' are always kept in a matrix form, even when only one
        method has been used.

      o 'convcode' is used in place of 'convergence' in result structure

      o There is a directory inst/interactive-test that contains ox.R test
        script that pauses so user can see interim results. 

      o As a result of several issues, the result structure is changed from
        that of optimx 2012 versions. It is now a data frame with a `details'
        attribute. Also an 'npar' attribute to give the number of parameters,
        and a 'maximize' attribute that is TRUE when the function is to be
        maximized.


VERSION 2013-03-28

      o print.optimx dropped. Not needed as print defaults to print.data.frame.

      o added summary.optimx which has order= and par.select= arguments

      o order= is an expression or list of expressions using the column names 
        of the "optimx" object as variables (plus the variable rownames).

      o par.select = FALSE now has same meaning as par.select = 0 (rather than 
        being an error)

      o [.optimx now subsets details attribute too

      o method column from details attribute dropped and moved to row names


VERSION 2013-03-27

      o created unit test framework with 2 unit tests in 
          inst/unitTests/runit.all.R.  
        To run: demo("unitTests")

VERSION 2013-03-25

      o [.optimx and as.data.frame.optimx added

      o coef.optimx fixed to reflect new ans.ret

VERSION 2013-03-22


      o ans.ret structure changed

VERSION 2013-03-21

      o maximize works with print(x, best.only=TRUE)

      o KKT checks can be switched off. Decided to NOT separate the code from
        optimx.run  UNTESTED

      o optimx.setup, optimx.run, optimx.check are NOT exported in NAMESPACE,
        but a knowledgeable user could use these codes directly, either from
        the source code or by modifying the package locally.

VERSION 2013-03-18

      o removed method= and columns= arguments from print.optimx and added
        best.only

      o removed print.optimx help page and moved it to the optimx help page

      o the row names of the optimx output now show the method and the method
        column has been removed

VERSION 2013-03-18

      o Removed all user accessible functions except optimx and added
        print.optimx.

VERSION 2013-03-16
      o  get.result now emits a warning if method requested is not in
         optimx solution
      o  attempt 2013-3-17 to include summary() in methods to replace
         trimpars()

VERSION 2012-07-18 optextras
   o    Extracted helper routines from optplus to 
        create optextras package to put on CRAN.
        Formerly in optplus or optfntools on R-forge,
        but difficulties with dependencies indicated
        that the support routines need to be on CRAN.
        Renamed to avoid name confusion.

VERSION 2012-07-09 optextras
   o    Suppress warnings in ugHgenb -- too much
        unhelpful output
   o    Fixed optansout to print parameters

VERSION 2012-06-07 optextras
   o    Merged optfntools into optplus to ensure 
        R-forge build.

VERSION 2012-05-26 optextras
   o    Added check for "on a bound" in bmchk to aid
	in use of tanh() transformation as in nmkb().


##### OPEN ISSUES: (any date order)
# 091220 -- ?? kkt1 tolerance. Do we want to let this be an input control?
# 090612 -- ?? (Similar to above) Better choices for the tolerances in tests of equality for gradient / kkt tests?
# 091018 -- ?? masks?
# 090929 -- ?? control structure could use simplification 
# 090729 -- ?? simplify structure of answers -- avoid repetition and make easier to access
# 090531 -- ?? like SNewton back in to keep Ramsay and Co. on side
# 090601 -- ?? Do we want hessevals to count Hessian evaluations?

##### IMPLEMENTED: (reverse date order)
# 110212 -- & to && and | to || for controls
# 110212 -- Hessian changed from NULL to FALSE default
# 100328 -- check if maximize works for Powell (minqa) routines -- 100415
# 100329 -- make sure spg fixed on CRAN -- 100415
# 100329 -- maximize tested for all but minqa, though spg needs fixup on CRAN.
# 100215 -- Add newuoa and uobyqa to test minqa
# 100212 -- ?? setting scaletol in controls ??
# 091018 -- new control "starttest" so we can skip them DONE 091220
# 091218 -- scaling checks (before computing rhobeg) and warning
# 091215 -- mcontrol omission in nlm() call fixed
# 091026 -- Remove SANN because no real conv indicator
# 091018 - 090531 Use function code joint with funtest and funcheck in initial checks -- not fully possible
# 091018 -- decided not to Put optimx.dev into package to provide for local source packages etc.
# 090923 -- add bobyqa (from minqa)
# 090923 -- add Rcgmin for large scale problems
# 090729 -- put hessian argument back as an alternate for kkt
# 090729 -- should have kkt only carried out for n<50 (no gradient), n<500 (with gradient and Jacobian
#            of gradient to get Hessian)
# 090531 -- KKT stuff
# 090531 -- control for keeping failed runs (save.failures)
# 090531 Decided to omit 'trust' from package methods for the time being.
# 090527 -- added follow.on
# 090511 What should be default method(s)? 
#         090601: to provide compatibility with optim(), Nelder-Mead is put first. 
#                 The choice of BFGS second is open to discussion. JN
#  A wrapper function to integrate major optimization packages in R
#  For now, we only consider algorithms for unconstrained and box-constrained optimization
#  This function can implement "multiple" algorithms
#
# Input:
#  par = a single vector of starting values
#  fn = objective function (assumed to be sufficeintly differentiable)
#  gr = name of a function to compute the (analytic) gradient of the objective function
#  hess = name of a function to compute the (analytic) Hessian of the objective function
#         Editorial note: Capitalize Hessian after the name of Otto Hesse. JN
#  method = a character vector denoting all the algorithms to be executed (in the specified order)
#      Complete names are not needed. A partial matching is attempted.
#  hessian = logical variable that, if present, is equivalent to control$kkt. If TRUE, it causes
#      optimx to try to compute an approximation to the Hessian matrix at the final set of parameters.
#  control = list of control information, in particular
#      trace = an integer controlling output (note different methods may use logicals
#         trace = 0 gives no output, positive numbers give more output for greater values
#      follow.on = TRUE or FALSE. If TRUE, and there are multiple methods, then the last set of 
#         parameters from one method is used as the starting set for the next. 
#      save.failures = TRUE if we wish to keep "answers" from runs where the method does not 
#         return conv==0. FALSE otherwise (default).
#      maximize = TRUE if we want to maximize rather than minimize a function. (Default FALSE)
#         090601: Not yet implemented for nlm, nlminb, ucminf. However, there is a check to avoid
#                 usage of these codes when maximize is TRUE.
#      all.methods = TRUE if we want to use all available (and suitable) methods
#      sort.result=TRUE, that is, we sort the results in decreasing order of the final function value
#      kkt=TRUE to run Kuhn, Karush, Tucker tests of results unless problem large
#      kkttol=0.001 (was .Machine$double.eps^(1/4)) Default value to check for small gradient and negative
#               Hessian eigenvalues
#      kkt2tol=1E-6 (default WAS 10* default for kkttol) Tolerance for eigenvalue ratio in KKT test of 
#               positive definite Hessian
#      all.methods=FALSE By default we do NOT run all methods
#      starttests=TRUE  By default we run tests of the function and parameters: feasibility relative to
#               bounds, analytic vs numerical gradient, scaling tests) before we try optimization methods
#      dowarn=TRUE By default we leave warnings generated by optimx.
#      badval=(0.5)*.Machine$double.xmax The value to set for the function value when try(fn()) fails.
#      scaletol=3 To check parameters or their bounds we take logs of absolute values and find the range 
#               i.e., max - min. This should not exceed scaletol. A value of 3 gives magnitudes between 
#               1 and 20 approximately.  
#
# Output:
# ans = an object containing two sets of information:
# essential output = a data frame of converged parameters, function value at convergence, 
#    name of algorithm, convergence indicator, and function evaluation and gradient evaluation counts
# detailed output = this is a list with one component for each algorithm, and contains parameters, 
#    function values, convergence code, number of function and gradient evals, numerical gradient 
#    and hessian at convergence, eigenvalues of that hessian approximation, cpu time, and other 
#    information returned by algorithms, and name of the algorithm.
# detailed output can be accessed via the attribute called `details'
#
#  Authors:  Ravi Varadhan & John Nash
#  Date:  February 17, 2008
#  Changes: Ravi Varadhan - Date: May 29, 2009, John Nash - Latest: Oct 18, 2009
#
===================================================================================

Changes in the optextras package

Changes before 2012-5-26 not recorded 

TODOS
   o scalechk should take account of masks

Changes

2018-03-28
   o    Examples all functions added, checked, revised.

2017-11-25
   o    Changed kktchk argument maxfn to maximize to agree with 
        optimr and other packages

2016-6-30
   o    Streamlined the package to get rid of ufn, ugr, uhess, 
        ugHgenb that are so far unused. 
   o    The optsp environment is only currently used
        with the simple numerical gradient approximations.
   o    kktc renamed to kktchk. scalecheck renamed to scalechk
        for consistency with other naming.

2013-10-27
   o    Inserted missing index in bmchk (bdmsk)

2013-9-11
   o    Fixed some elements of manual. Alias optextras-package
        to optextras.

2013-8-20
   o    optsp scratchpad environment added. Note that it
        is aliased in grfwd to avoid a warning in R CMD
        check.

   o    Can change this in numerical derivative routines
        in case of parallel running.

2012-7-18
   o    Extracted helper routines from optplus to 
        create optextras package to put on CRAN.
        Formerly in optplus or optfntools on R-forge,
        but difficulties with dependencies indicated
        that the support routines need to be on CRAN.
        Renamed to avoid name confusion.

2012-7-09
   o    Suppress warnings in ugHgenb -- too much
        unhelpful output
   o    Fixed optansout to print parameters

2012-6-07
   o    Merged optfntools into optplus to ensure 
        R-forge build.

2012-5-26
   o    Added check for "on a bound" in bmchk to aid
	in use of tanh() transformation as in nmkb().

===================================================================================

Changes in Rcgmin

Changes before 2012-4-30 not recorded 

TODOS
   o    Eventually include different line searches, possibly in a
        separate routine

2014-7-17
   o    Changed default of control$checkgrad to FALSE. Changed error message
	that mentions Rvmmin to Rcgmin.

2013-9-24 
   o    Warning that parameter moved to bound suppressed when start
        is on the bound.

2013-8-20

   o    Allow quoted gradient names to bring in different gradient 
        routines. Also to allow gradient approximation step to be
	adjusted.

   o    Dependency on optextras rather than numDeriv

2013-2-20

   o    Removed fail, setstep (not used), ceps from Rcgminu (not used)

2012-8-08

   o    Minor edits and addition of references.

2012-6-09

   o    User-tolerance for gradient size.

2012-5-21
   o	Separation of constrained and unconstrained versions.

===================================================================================

Changes in Rtnmin

TODOS
   o    20171007 -- make lmqnbc printout conditional on trace??
   o    Simplification and streamlining of code to mesh
	better with other R optimization tools.

   o    Put maxit, maxfun in control list somehow.

   o    Setup trace as integer to work properly


2018-03-28
   o Try ! is.numeric(g) vs is.null(g)
   o trace changed to integer

??2016-09-22 account for null gradient 

2016-06-28

   o    Removed development files from inst, and inst directory

2013-09-12

   o    Modlnp fix from SGN implemented so jones function works.

2013-09-03

   o    Some fixes, particularly to dot-arguments missing
	from calls to user function.

   o    Changed returned user fgfun to f with g as attribute.

2013-06-03

   o    Properly packaged version of Rtnmin. 

2013-11-11 
   
   o    First version of Rtn.


===================================================================================

Changes in Rvmmin

Changes before 2012-4-30 not recorded 

TODOS
   o    Saving of final Hessian approx as in Rvmminx

   o    Eventual incorporation of line searches, possibly in a
        separate routine

2017-11-25
   o    Tidy tests
   o    Some cleanup of vignette

2017-7-18
   o    Simplify example
   o    Added tests

2016-11-30
   o    Vignette added (incomplete)

2015-11-30
   o    Changed control "stopbadupdate" to TRUE from FALSE.

2015-06-03
   o    Had "lower" instead of "upper" in one of the bounds checks.
        Fixed.

2015-05-03
   o    Check gradient count ig > 2 when doing convergence test.
	Possible failure for test fn = sum(x*x) noted. However, 
	other fns fail with this change. This is an issue in the
	program logic rather than an algorithmic failure. 

2015-01-07

   o    comments in codes Rvmminb.R and Rvmminu.R aligned so that the
        differences between bounds constrained version and unconstrained
        more easily visualized
        
   o    control keepinputpar explained. It is used when user does NOT want
        out-of-bounds parameters on input moved to nearest bound. Program
        will stop in this case.
        
   o    Minor changes to maxit and maxfeval defaults so that they are 
        increased with increasing numbers of parameters.

2014-12-21 

   o    As per CRAN policies, edited DESCRIPTION and NAMESPACE files. 
	Because there is no change to code, the date of the package 
	advanced by only 1 day.

2013-10-27 
   
   o    Added maxfevals to control. (It had been there some time.)

2013-8-19

   o    Numerical gradient from optextras, remove bmchk from package,
	put bmchk call in Rvmmin, not Rvmminb

2013-7-20
   o    Minor improvements to timing by use of crossprod() in 
        computing y and D2

2012-7-9
   o    Removed optansout to avoid conflicts with optplus/optimx

2012-5-21
   o	Separation of constrained and unconstrained versions.


===================================================================================