1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
|
# Contributing to Alcotest
## Setting up your working environment
To clone the project's sources and install its various dependencies, run:
```sh
git clone https://github.com/mirage/alcotest.git # Get the repository
cd alcotest
opam switch create ./ ocaml-base-compiler.5.0.0 # OPTIONAL: install a project-local Opam switch
opam install -t --deps-only . # Install regular and test dependencies
```
Optionally, install `js_of_ocaml-compiler` and [nodejs](https://nodejs.org/en/download/package-manager/) to be able
run the testsuite using `js_of_ocaml`.
```sh
opam install -t js_of_ocaml-compiler
```
Now you can work on the project using the standard Dune commands:
```sh
dune build # Build all libraries, executables and tests
dune test # Run the test-suite
dune build @runtest-js # Run the test-suite on nodejs (using js_of_ocaml)
dune clean # Delete all artefacts
```
## Project structure
The high-level project structure is as follows:
```sh
├── src/
│ ├── alcotest-engine/
│ │ ├── core.ml # The main test runner, parameterised over...
│ │ ├── monad.ml # ... a concurrency monad
│ │ ├── platform.ml # ... a platform implementation / OS bindings
│ │ │
│ │ ├── cli.ml # Extends `core.ml` with a command-line API
│ │ │
│ │ ├── pp.ml # Pretty-printers for Alcotest output
│ │ ├── test.ml # Combinators for test assertions (with `Alcotest.check`)
│ │ └── utils.ml # Standard-library extension
│ │
│ │ # Specific backends:
│ ├── alcotest/ # - Unix-specific API (compatible with js_of_ocaml)
│ ├── alcotest-async/ # - Extends `alcotest/` for test-suites using Async concurrency
│ ├── alcotest-lwt/ # - Extends `alcotest/` for test-suites using Lwt concurrency
│ └── alcotest-mirage/ # - MirageOS-specific API
│
├── test/ # Project tests (see '§ Expect tests' below)
│
├── dune-project # Project metadata (opam files are generated from
├── alcotest-async.opam # data in the `dune-project` file by running
├── alcotest-lwt.opam # `dune build @install`).
├── alcotest-mirage.opam
├── alcotest.opam
│
├── alcotest-help.txt # Manpage output (generated by `dune test --auto-promote`)
├── dune
│
├── CHANGES.md # All user-facing changes get an entry here
└── README.md
```
## Expect tests
The repository contains a number of 'expect' tests in
https://github.com/mirage/alcotest/tree/main/test/e2e. These are short,
self-contained usages of the Alcotest API, with the corresponding output
snapshotted in an `.expected` file:
```sh
test/e2e/alcotest/passing/
├── basic.ml # The test case
├── basic.expected # The expected output of the test case
└── basic.opts # OPTIONAL: command-line options to pass to the test
```
If you change the Alcotest output in some way, you will likely need to update
the expect test snapshots:
1. run `dune test` to get a list of diffs to the tests,
1. check that the diffs are as expected,
1. run `dune promote` to update the snapshots in the repository.
To add a new test, create a new `foo.{ml,expected}` pair in one of the test
directories and run `dune test --auto-promote` to automatically generate the
Dune rules necessary to run your test.
### Random output in tests
Some sections of the Alcotest output are not portable / reproducible under test,
for instance:
- the time taken to run a particular test;
- randomly-generated unique identifiers;
- the full path to the repository on disk;
- the executable file extension.
When running an expect test, these portions of the output are sanitized by
[`strip_randomness.ml`](./test/e2e/strip_randomness.ml) before being compared to
the `.expected` file, resulting in files with portable placeholders:
```
The full test results are available in `<build-context>/_build/_tests/<uuid>`.
Test Successful in <test-duration>s. 4 tests run.
```
If your change adds non-determinism to the Alcotest output, you may need to
alter `strip_randomness.ml` to get it past the CI.
|
