# Changes: 3.14

```{rubric} General:
```

- Deprecate PetscIgnoreErrorHandler(), use PetscReturnErrorHandler()
- Replace -debugger_nodes with -debugger_ranks
- Change PETSCABORT() to abort instead of MPI_Abort if run under
  -start_in_debugger
- Add PETSC_MPI_THREAD_REQUIRED to control the requested threading
  level for MPI_Init
- Add CUDA-11 support, but with CUDA-11,
  -mat_cusparse_storage_format {ELL, HYB} are not supported anymore.
  Only CSR is supported
- Add CUDA-11 option -mat_cusparse_spmv_alg {MV_ALG_DEFAULT,
  CSRMV_ALG1 (default), CSRMV_ALG2} for users to select cuSPARSE
  SpMV algorithms
- Add CUDA-11 option -mat_cusparse_spmm_alg {ALG_DEFAULT, CSR_ALG1
  (default), CSR_ALG2} for users to select cuSPARSE SpMM algorithms
- Add CUDA-11 option -mat_cusparse_csr2csc_alg {ALG1 (default),
  ALG2} for users to select cuSPARSE CSR to CSC conversion
  algorithms
- Remove option -cuda_initialize, whose functionality is succeeded
  by -cuda_device xxx
- Change -cuda_set_device to -cuda_device, which can now accept
  NONE, PETSC_DEFAULT, PETSC_DECIDE in addition to non-negative
  integers
- Change PetscCUDAInitialize(comm) to PetscCUDAInitialize(comm,dev)
- Add PetscCUDAInitializeCheck() to do lazy CUDA initialization
- Add -hip_device, -hip_view, -hip_synchronize,
  PetscHIPInitialize(comm,dev) and PetscHIPInitializeCheck(). Their
  usage is similar to their CUDA counterpart
- Add PetscOptionsInsertStringYAML() and -options_string_yaml for
  YAML-formatted options on the command line
- Add PETSC_OPTIONS_YAML environment variable for setting options in
  YAML format
- Add PetscDetermineInitialFPTrap(); fix so that when Linux or macOS
  Fortran linker enables catching floating point divide by zero the
  trapping is disabled for LAPACK routines that generate divide by
  zero, for example, the reference implementation of ieeeck()
- Add floating point exception handling support for freebsd and
  Windows
- Consistently set exception handling for divide by zero, invalid,
  underflow, and overflow for all systems when possible
- -options_monitor and -options_monitor_cancel have immediate global
  effect, see PetscInitialize() for details
- Remove PetscOptionsSetFromOptions()
- Remove PetscOptionsMonitorCancel()
- Remove -h and -v options. Use -help and -version instead. The
  short options -h and -v can now be used within user codes
- Import petsc4py sources into PETSc source tree. Continue to use
  --download-petsc4py to build petsc4py
- Add an experimental Kokkos backend for PETSc GPU operations. For
  example, one can use '--download-kokkos --download-kokkos-kernels
  --with-kokkos-cuda-arch=TURING75' to build PETSc with a Kokkos
  CUDA backend, and then use -vec_type kokkos -mat_type aijkokkos.
  With that, vector and matrix operations on GPUs are done through
  Kokkos kernels. Currently, VECKOKKOS supports all vector
  operations, but MATAIJKOKKOS only supports MatMult() and its
  variants. More complete support is coming

```{rubric} Configure/Build:
```

- Change --with-matlabengine-lib= to --with-matlab-engine-lib= to
  match --with-matlab-engine, print error message for deprecated
  form
- Change --download-mpich default for optimized build to ch3:nemesis
  and keep ch3:sock for debug build
- On macOS, --with-macos-firewall-rules can be used to automate
  addition of firewall rules during testing to prevent firewall
  popup windows

```{rubric} IS:
```

```{rubric} PetscDraw:
```

```{rubric} VecScatter / PetscSF:
```

- Add a Kokkos backend to SF. Previously, SF could only handle CUDA
  devices. Now it can handle other devices that Kokkos supports when
  PETSc is configured with Kokkos. The command line option is:
  -sf_backend cuda | kokkos

```{rubric} PF:
```

```{rubric} Vec:
```

- Fix memory leaks when requesting -vec_type
  {standard|cuda|viennacl} when the vector is already of the desired
  type
- Add VecViennaCLGetCL{Context|Queue|Mem} for VECVIENNACL to access
  the CL objects underlying the PETSc Vecs
- Add VecCreate{Seq|MPI}ViennaCLWithArray and
  VecViennaCL{Place|Reset}Array
- Add VecCreate{Seq|MPI}CUDAWithArrays to create VECCUDA sharing the
  CPU and/or GPU memory spaces
- Add VecCreate{Seq|MPI}ViennaCLWithArrays to create VECVIENNACL
  sharing the CPU and/or GPU memory spaces
- Add an experimental vector type VECKOKKOS
- Add VecGetOffloadMask to query a Vec's offload mask

```{rubric} PetscSection:
```

- PetscSectionGetClosurePermutation(),
  PetscSectionSetClosurePermutation(), and
  PetscSectionGetClosureInversePermutation() all require a new
  argument depth and the getters require closure size to be
  specified by the caller. This allows multiple closure permutations
  to be specified, e.g., for mixed topology meshes and boundary
  faces and for variable-degree spaces. The previous behavior only
  applied to points at height zero

```{rubric} PetscPartitioner:
```

```{rubric} Mat:
```

- Add MatSetLayouts()
- Add MatSeqAIJSetTotalPreallocation(Mat,PetscInt) for efficient row
  by row setting of a matrix without requiring preallocating for
  each row
- Add full support for MKL sparse matrix-matrix products in
  MATSEQAIJMKL
- Fix few bugs for MATSEQSBAIJ when missing diagonal entries
- Fix few bugs when trying to reuse matrices within MatMat
  operations
- Deprecate MatFreeIntermediateDataStructures() in favor of
  MatProductClear()
- Add MatShellSetMatProductOperation() to allow users specify
  symbolic and numeric phases for MatMat operations with MATSHELL
  matrices
- Add support for distributed dense matrices on GPUs
  (MATMPIDENSECUDA)
- Add few missing get/set/replace array operations for MATDENSE and
  MATDENSECUDA matrices
- Add MatDense{Get|Restore}ColumnVec to access memory of a dense
  matrix as a Vec, together with read-only and write-only variants
- Add MatDense{Get|Restore}SubMatrix to access memory of a
  contiguous subset of columns of a dense matrix as a Mat
- Deprecate MatSeqDenseSetLDA in favor of MatDenseSetLDA
- Add support for A\*B and A^t\*B operations with A = AIJCUSPARSE and
  B = DENSECUDA matrices
- Add basic support for MATPRODUCT_AB (resp. MATPRODUCT_AtB) for any
  matrices with mult (multtranspose) operation defined and B dense
- Add MATSCALAPACK, a new Mat type that wraps a ScaLAPACK matrix
- Add support for MUMPS-5.3.0 distributed right-hand side
- Add support for MatMultHermitianTranspose with SEQAIJCUSPARSE
- Remove default generation of explicit matrix for MatMultTranspose
  operations with SEQAIJCUSPARSE. Users can still require it via
  MatSeqAIJCUSPARSESetGenerateTranspose
- Add MatOrderingType external returns a NULL ordering to allow
  solver types MATSOLVERUMFPACK and MATSOLVERCHOLMOD to use their
  orderings
- Add an experimental matrix type MATAIJKOKKOS

```{rubric} PC:
```

- Fix bugs related with reusing PCILU/PCICC/PCLU/PCCHOLESKY
  preconditioners with SEQAIJCUSPARSE matrices
- GAMG uses MAT_SPD to default to CG for the eigen estimate in
  Chebyshev smoothers
- Add PCMatApply() for applying a preconditioner to a block of
  vectors
- Add -pc_factor_mat_ordering_type external to use ordering methods
  of MATSOLVERUMFPACK and MATSOLVERCHOLMOD
- PCSetUp_LU,ILU,Cholesky,ICC() no longer compute an ordering if it
  is not to be used by the factorization (optimization)

```{rubric} KSP:
```

- Add KSPGMRESSetBreakdownTolerance() along with option
  -ksp_gmres_breakdown_tolerance to 3.14.3
- Change KSPReasonView() to KSPConvergenceReasonView()
- Change KSPReasonViewFromOptions() to
  KSPConvergedReasonViewFromOptions()
- Add KSPConvergedDefaultSetConvergedMaxits() to declare convergence
  when the maximum number of iterations is reached
- Fix many KSP implementations to actually perform the number of
  iterations requested
- Add KSPMatSolve() for solving iteratively (currently only with
  KSPHPDDM and KSPPREONLY) systems with multiple right-hand sides,
  and KSP{Set|Get}MatSolveBlockSize() to set a block size limit
- Chebyshev uses MAT_SPD to default to CG for the eigen estimate
- Add KSPPIPECG2, a pipelined solver that reduces the number of
  allreduces to one per two iterations and overlaps it with two PCs
  and SPMVs using non-blocking allreduce

```{rubric} SNES:
```

- Change SNESReasonView() to SNESConvergedReasonView()
- Change SNESReasonViewFromOptions() to
  SNESConvergedReasonViewFromOptions()

```{rubric} SNESLineSearch:
```

```{rubric} TS:
```

- Fix examples using automatic differentiation. One can use
  '--download-adolc --download-colpack' to install the AD tool
- Improve shift handling in TSComputeXXXJacobian()
- Update TSTrajectory (type memory) to preallocate a checkpoint pool
  to be reused across multiple TS runs

```{rubric} TAO:
```

- Add lm regularizer to TAOBRGN. This regularizer turns BRGN into a
  Levenberg-Marquardt algorithm. TAOBRGNGetDamping() vector returns
  the damping vector used by this regularizer

```{rubric} DM/DA:
```

- Change DMComputeExactSolution() to also compute the time
  derivative of the exact solution
- Add time derivative of the solution argument to DMAddBoundary(),
  DMGetBoundary(), PetscDSAddBoundary(), PetscDSUpdateBoundary(),
  PetscDSGetBoundary()

```{rubric} DMPlex:
```

- Deprecate DMPlexCreateFromCellList\[Parallel\]() in favor of
  DMPlexCreateFromCellList[Parallel]Petsc() which accept PETSc
  datatypes (PetscInt, PetscReal)
- Expose DMPlexBuildFromCellList(),
  DMPlexBuildFromCellListParallel(),
  DMPlexBuildCoordinatesFromCellList(),
  DMPlexBuildCoordinatesFromCellListParallel(). They now accept
  PETSc datatypes
- Add DMPlexMatSetClosureGeneral() for different row and column
  layouts
- DMPlexGet/RestoreClosureIndices() now take argument for ignoring
  the closure permutation and for modifying the input values for
  SetClosure()
- DMPlexComputeInterpolatorNested() now takes a flag allowing nested
  interpolation between different spaces on the same mesh
- Add DMPlexInsertBoundaryValuesEssentialBdField() to insert
  boundary values using a field only supported on the boundary
- Change DMPlexCreateSubpointIS() to DMPlexGetSubpointIS()
- Add PetscDSGet/SetBdJacobianPreconditioner() to assembly a PC for
  the boundary Jacobian
- Add DMSetRegionNumDS() to directly set the DS for a given region
- Add PetscDSGetQuadrature() to get the quadrature shared by all
  fields in the DS
- Add several refinement methods for Plex
- Add DMPlexGet/SetActivePoint() to allow user to see which mesh
  point is being handled by projection
- Add DMPlexComputeOrthogonalQuality() to compute cell-wise
  orthogonality quality mesh statistic
- Change DMPlexSetClosurePermutationTensor() to set tensor
  permutations at every depth, instead of just height 0
- Add DMComputeExactSolution() which uses PetscDS information
- Change DMSNESCheckFromOptions() and DMTSCheckFromOptions() to get
  exact solution from PetscDS
- Change DMPlexSNESGetGeometryFVM() to DMPlexGetGeometryFVM()
- Change DMPlexSNESGetGradientDM() to DMPlexGetGradientDM()
- Change DMPlexCreateSphereMesh() to take a radius
- Add DMPlexCreateBallMesh()
- Change DMSNESCheckDiscretization() to also take the time
- Add argument to DMPlexExtrude() to allow setting normal and add
  options for inputs
- Add DMPlexInsertTimeDerivativeBoundaryValues()
- Add field number argument to DMPlexCreateRigidBody()

```{rubric} DT:
```

- Add PetscDTJacobiNorm() for the weighted L2 norm of Jacobi
  polynomials
- Add PetscDTJacobiEvalJet() and PetscDTPKDEvalJet() for evaluating
  the derivatives of orthogonal polynomials on the segment (Jacobi)
  and simplex (PKD)
- Add PetscDTIndexToGradedOrder() and PetscDTGradedOrderToIndex()
  for indexing multivariate monomials and derivatives in a linear
  order
- Add PetscSpaceType "sum" for constructing FE spaces as the sum or
  concatenation of other spaces
- Add PetscDSGet/SetExactSolutionTimeDerivative()
- Add PetscDSSelectDiscretizations()
- Add argument to DM nullspace constructors

```{rubric} PetscViewer:
```

- Deprecate the legacy `.vtk` (PETSC_VIEWER_ASCII_VTK) viewer.
  Please use `.vtr` or `.vts` for structured grids (DMDA) and
  `.vtu` for unstructured (DMPlex)

```{rubric} SYS:
```

- Add PetscPowInt64 returning a 64bit integer result for cases where
  PetscPowInt result overflows 32bit representations
- Add PetscTimSort\[WithArray\]() for improved performance when
  sorting semi-ordered arrays of any type
- Add PetscIntSortSemiOrdered\[WithArray\](),
  PetscMPIIntSortSemiOrdered\[WithArray\](),
  PetscRealSort\[WithArrayInt\]() which employ
  PetscTimSort\[WithArray\]() as backends respectively to more
  efficiently sort semi-ordered arrays of various Petsc datatypes
- Add PetscMallocTraceSet/Get() to allow tracing of all PetscMalloc
  calls
- Add PetscMallocLogRequestedSizeSet/Get() to allow reporting of the
  original requested size for mallocs, rather than the total size
  with alignment and header

```{rubric} AO:
```

```{rubric} Convest:
```

- Add argument to PetscConvEstUseTS(), so you can use
  -ts_convergence_temporal 0 to check spatial convergence of a TS
  model

```{rubric} Fortran:
```