<!--

@license Apache-2.0

Copyright (c) 2018 The Stdlib Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

-->

# Benchmark

> Benchmark commands.

<!-- Section to include introductory text. Make sure to keep an empty line after the intro `section` element and another before the `/section` close. -->

<section class="intro">

This directory contains [`make`][make] rules for running language benchmarks.

</section>

<!-- /.intro -->

<!-- Usage documentation. -->

<section class="usage">

## Usage

```text
Usage: make <command> [<ENV_VAR>=<value> <ENV_VAR>=<value> ...]
```

### Commands

#### benchmark

Runs benchmarks.

<!-- run-disable -->

```bash
$ make benchmark
```

The command supports the following environment variables:

-   **BENCHMARKS_FILTER**: file path pattern; e.g., `.*/blas/base/dasum/.*`.

This command is an **alias** for `benchmark-javascript`, which is documented below.

#### benchmark-lang

Runs cross-language benchmarks.

<!-- run-disable -->

```bash
$ make benchmark-lang
```

The command supports the environment variables supported by each language-specific (`benchmark-<lang>`) command documented below.

This command is useful when wanting to glob for benchmark files, irrespective of language, for a particular package in order to compare cross-language performance. 

* * *

### C

> **Note**: C benchmark commands delegate to local Makefiles which are responsible for actually compiling and running the respective benchmarks.

#### benchmark-c

Runs C benchmarks.

<!-- run-disable -->

```bash
$ make benchmark-c
```

The command supports the following environment variables:

-   **BENCHMARKS_FILTER**: file path pattern; e.g., `.*/math/base/special/abs/.*`.
-   **BLAS**: BLAS library name; e.g., `openblas`.
-   **BLAS_DIR**: BLAS directory.
-   **C_COMPILER**: C compiler; e.g., `gcc`.

This command is useful when wanting to glob for C benchmark files (e.g., run all C benchmarks for a particular package).

#### benchmark-c-files

Runs a specified list of C benchmark files.

<!-- run-disable -->

```bash
$ make benchmark-c-files FILES='/foo/benchmark.c /bar/benchmark.c'
```

The command supports the following environment variables:

-   **FILES**: list of C benchmark files.
-   **BLAS**: BLAS library name; e.g., `openblas`.
-   **BLAS_DIR**: BLAS directory.
-   **C_COMPILER**: C compiler; e.g., `gcc`.

This command is useful when wanting to run a list of C benchmark files generated by some other command (e.g., a filtered list of changed C benchmark files obtained via `git diff`).

* * *

### C++

> **Note**: C++ benchmark commands delegate to local Makefiles which are responsible for actually compiling and running the respective benchmarks.

#### benchmark-cpp

Runs C++ benchmarks.

<!-- run-disable -->

```bash
$ make benchmark-cpp
```

The command supports the following environment variables:

-   **BENCHMARKS_FILTER**: file path pattern; e.g., `.*/math/base/special/beta/.*`.
-   **CXX_COMPILER**: C++ compiler; e.g., `g++`.

This command is useful when wanting to glob for C++ benchmark files (e.g., run all C++ benchmarks for a particular package).

#### benchmark-cpp-files

Runs a specified list of C++ benchmark files.

<!-- run-disable -->

```bash
$ make benchmark-cpp-files FILES='/foo/benchmark.cpp /bar/benchmark.cpp'
```

The command supports the following environment variables:

-   **FILES**: list of C++ benchmark files.
-   **CXX_COMPILER**: C++ compiler; e.g., `g++`.

This command is useful when wanting to run a list of C++ benchmark files generated by some other command (e.g., a filtered list of changed C++ benchmark files obtained via `git diff`).

* * *

### Fortran

> **Note**: Fortran benchmark commands delegate to local Makefiles which are responsible for actually compiling and running the respective benchmarks.

#### benchmark-fortran

Runs Fortran benchmarks.

<!-- run-disable -->

```bash
$ make benchmark-fortran
```

The command supports the following environment variables:

-   **BENCHMARKS_FILTER**: file path pattern; e.g., `.*/blas/base/daxpy/.*`.
-   **FORTRAN_COMPILER**: Fortran compiler; e.g., `gfortran`.

This command is useful when wanting to glob for Fortran benchmark files (e.g., run all Fortran benchmarks for a particular package).

#### benchmark-fortran-files

Runs a specified list of Fortran benchmark files.

<!-- run-disable -->

```bash
$ make benchmark-fortran-files FILES='/foo/benchmark.f /bar/benchmark.f'
```

The command supports the following environment variables:

-   **FILES**: list of Fortran benchmark files.
-   **FORTRAN_COMPILER**: Fortran compiler; e.g., `gfortran`.

This command is useful when wanting to run a list of Fortran benchmark files generated by some other command (e.g., a filtered list of changed Fortran benchmark files obtained via `git diff`).

* * *

### JavaScript

#### benchmark-javascript

Runs JavaScript benchmarks.

<!-- run-disable -->

```bash
$ make benchmark-javascript
```

The command supports the following environment variables:

-   **BENCHMARKS_FILTER**: file path pattern; e.g., `.*/utils/group-by/.*`.

This command is useful when wanting to glob for JavaScript benchmark files (e.g., run all JavaScript benchmarks for a particular package).

#### benchmark-javascript-files

Runs a specified list of JavaScript benchmark files.

<!-- run-disable -->

```bash
$ make benchmark-javascript-files FILES='/foo/benchmark.js /bar/benchmark.js'
```

The command supports the following environment variables:

-   **FILES**: list of JavaScript benchmark files.

This command is useful when wanting to run a list of JavaScript benchmark files generated by some other command (e.g., a filtered list of changed JavaScript benchmark files obtained via `git diff`).

* * *

### Julia

#### benchmark-julia

Runs Julia benchmarks.

<!-- run-disable -->

```bash
$ make benchmark-julia
```

The command supports the following environment variables:

-   **BENCHMARKS_FILTER**: file path pattern; e.g., `.*/math/base/special/erf/.*`.

This command is useful when wanting to glob for Julia benchmark files (e.g., run all Julia benchmarks for a particular package).

#### benchmark-julia-files

Runs a specified list of Julia benchmark files.

<!-- run-disable -->

```bash
$ make benchmark-julia-files FILES='/foo/benchmark.jl /bar/benchmark.jl'
```

The command supports the following environment variables:

-   **FILES**: list of Julia benchmark files.

This command is useful when wanting to run a list of Julia benchmark files generated by some other command (e.g., a filtered list of changed Julia benchmark files obtained via `git diff`).

* * *

### Python

#### benchmark-python

Runs Python benchmarks.

<!-- run-disable -->

```bash
$ make benchmark-python
```

The command supports the following environment variables:

-   **BENCHMARKS_FILTER**: file path pattern; e.g., `.*/math/base/special/digamma/.*`.

This command is useful when wanting to glob for Python benchmark files (e.g., run all Python benchmarks for a particular package).

#### benchmark-python-files

Runs a specified list of Python benchmark files.

<!-- run-disable -->

```bash
$ make benchmark-python-files FILES='/foo/benchmark.py /bar/benchmark.py'
```

The command supports the following environment variables:

-   **FILES**: list of Python benchmark files.

This command is useful when wanting to run a list of Python benchmark files generated by some other command (e.g., a filtered list of changed Python benchmark files obtained via `git diff`).

* * *

### R

#### benchmark-r

Runs R benchmarks.

<!-- run-disable -->

```bash
$ make benchmark-r
```

The command supports the following environment variables:

-   **BENCHMARKS_FILTER**: file path pattern; e.g., `.*/math/base/special/expm1/.*`.

This command is useful when wanting to glob for R benchmark files (e.g., run all R benchmarks for a particular package).

#### benchmark-r-files

Runs a specified list of R benchmark files.

<!-- run-disable -->

```bash
$ make benchmark-r-files FILES='/foo/benchmark.R /bar/benchmark.R'
```

The command supports the following environment variables:

-   **FILES**: list of R benchmark files.

This command is useful when wanting to run a list of R benchmark files generated by some other command (e.g., a filtered list of changed R benchmark files obtained via `git diff`).

</section>

<!-- /.usage -->

<!-- Section to include notes. Make sure to keep an empty line after the `section` element and another before the `/section` close. -->

<section class="notes">

</section>

<!-- /.notes -->

<!-- Section for all links. Make sure to keep an empty line after the `section` element and another before the `/section` close. -->

<section class="links">

[make]: https://www.gnu.org/software/make/

</section>

<!-- /.links -->