1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
|
<img src="http://mypy-lang.org/static/mypy_light.svg" alt="mypy logo" width="300px"/>
# pytest plugin for testing mypy types, stubs, and plugins
[](https://github.com/typeddjango/pytest-mypy-plugins/actions/workflows/test.yml)
[](http://mypy-lang.org/)
[](https://gitter.im/mypy-django/Lobby)
[](https://pypi.org/project/pytest-mypy-plugins/)
[](https://anaconda.org/conda-forge/pytest-mypy-plugins)
## Installation
This package is available on [PyPI](https://pypi.org/project/pytest-mypy-plugins/)
```bash
pip install pytest-mypy-plugins
```
and [conda-forge](https://anaconda.org/conda-forge/pytest-mypy-plugins)
```bash
conda install -c conda-forge pytest-mypy-plugins
```
## Usage
### Running
Plugin, after installation, is automatically picked up by `pytest` therefore it is sufficient to
just execute:
```bash
pytest
```
### Asserting types
There are two ways to assert types.
The custom one and regular [`typing.assert_type`](https://docs.python.org/3/library/typing.html#typing.assert_type).
Our custom type assertion uses `reveal_type` helper and custom output matchers:
```yml
- case: using_reveal_type
main: |
instance = 1
reveal_type(instance) # N: Revealed type is 'builtins.int'
```
This method also allows to use `# E:` for matching exact error messages and codes.
But, you can also use regular `assert_type`, examples can be [found here](https://github.com/typeddjango/pytest-mypy-plugins/blob/master/pytest_mypy_plugins/tests/test-assert-type.yml).
### Paths
The `PYTHONPATH` and `MYPYPATH` environment variables, if set, are passed to `mypy` on invocation.
This may be helpful if you are testing a local plugin and need to provide an import path to it.
Be aware that when `mypy` is run in a subprocess (the default) the test cases are run in temporary working directories
where relative paths such as `PYTHONPATH=./my_plugin` do not reference the directory which you are running `pytest` from.
If you encounter this, consider invoking `pytest` with `--mypy-same-process` or make your paths absolute,
e.g. `PYTHONPATH=$(pwd)/my_plugin pytest`.
You can also specify `PYTHONPATH`, `MYPYPATH`, or any other environment variable in `env:` section of `yml` spec:
```yml
- case: mypy_path_from_env
main: |
from pair import Pair
instance: Pair
reveal_type(instance) # N: Revealed type is 'pair.Pair'
env:
- MYPYPATH=../fixtures
```
### What is a test case?
In general each test case is just an element in an array written in a properly formatted `YAML` file.
On top of that, each case must comply to following types:
| Property | Type | Description |
| --------------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------- |
| `case` | `str` | Name of the test case, complies to `[a-zA-Z0-9]` pattern |
| `main` | `str` | Portion of the code as if written in `.py` file |
| `files` | `Optional[List[File]]=[]`\* | List of extra files to simulate imports if needed |
| `disable_cache` | `Optional[bool]=False` | Set to `true` disables `mypy` caching |
| `mypy_config` | `Optional[str]` | Inline `mypy` configuration, passed directly to `mypy` as `--config-file` option, possibly joined with `--mypy-pyproject-toml-file` or `--mypy-ini-file` contents if they are passed. By default is treated as `ini`, treated as `toml` only if `--mypy-pyproject-toml-file` is passed |
| `env` | `Optional[Dict[str, str]]={}` | Environmental variables to be provided inside of test run |
| `parametrized` | `Optional[List[Parameter]]=[]`\* | List of parameters, similar to [`@pytest.mark.parametrize`](https://docs.pytest.org/en/stable/parametrize.html) |
| `skip` | `str` | Expression evaluated with following globals set: `sys`, `os`, `pytest` and `platform` |
| `expect_fail` | `bool` | Mark test case as an expected failure, like [`@pytest.mark.xfail`](https://docs.pytest.org/en/stable/skipping.html) |
| `regex` | `str` | Allow regular expressions in comments to be matched against actual output. Defaults to "no", i.e. matches full text.|
(*) Appendix to **pseudo** types used above:
```python
class File:
path: str
content: Optional[str] = None
Parameter = Mapping[str, Any]
```
Implementation notes:
- `main` must be non-empty string that evaluates to valid **Python** code,
- `content` of each of extra files must evaluate to valid **Python** code,
- `parametrized` entries must all be the objects of the same _type_. It simply means that each
entry must have **exact** same set of keys,
- `skip` - an expression set in `skip` is passed directly into
[`eval`](https://docs.python.org/3/library/functions.html#eval). It is advised to take a peek and
learn about how `eval` works.
Repository also offers a [JSONSchema](pytest_mypy_plugins/schema.json), with which
it validates the input. It can also offer your editor auto-completions, descriptions, and validation.
All you have to do, add the following line at the top of your YAML file:
```yaml
# yaml-language-server: $schema=https://raw.githubusercontent.com/typeddjango/pytest-mypy-plugins/master/pytest_mypy_plugins/schema.json
```
### Example
#### 1. Inline type expectations
```yaml
# typesafety/test_request.yml
- case: request_object_has_user_of_type_auth_user_model
main: |
from django.http.request import HttpRequest
reveal_type(HttpRequest().user) # N: Revealed type is 'myapp.models.MyUser'
# check that other fields work ok
reveal_type(HttpRequest().method) # N: Revealed type is 'Union[builtins.str, None]'
files:
- path: myapp/__init__.py
- path: myapp/models.py
content: |
from django.db import models
class MyUser(models.Model):
pass
```
#### 2. `@parametrized`
```yaml
- case: with_params
parametrized:
- val: 1
rt: builtins.int
- val: 1.0
rt: builtins.float
main: |
reveal_type({{ val }}) # N: Revealed type is '{{ rt }}'
```
Properties that you can parametrize:
- `main`
- `mypy_config`
- `out`
#### 3. Longer type expectations
```yaml
- case: with_out
main: |
reveal_type('abc')
out: |
main:1: note: Revealed type is 'builtins.str'
```
#### 4. Regular expressions in expectations
```yaml
- case: expected_message_regex_with_out
regex: yes
main: |
a = 'abc'
reveal_type(a)
out: |
main:2: note: .*str.*
```
#### 5. Regular expressions specific lines of output.
```yaml
- case: expected_single_message_regex
main: |
a = 'hello'
reveal_type(a) # NR: .*str.*
```
## Options
```
mypy-tests:
--mypy-testing-base=MYPY_TESTING_BASE
Base directory for tests to use
--mypy-pyproject-toml-file=MYPY_PYPROJECT_TOML_FILE
Which `pyproject.toml` file to use
as a default config for tests.
Incompatible with `--mypy-ini-file`
--mypy-ini-file=MYPY_INI_FILE
Which `.ini` file to use as a default config for tests.
Incompatible with `--mypy-pyproject-toml-file`
--mypy-same-process
Run in the same process. Useful for debugging,
will create problems with import cache
--mypy-extension-hook=MYPY_EXTENSION_HOOK
Fully qualified path to the extension hook function,
in case you need custom yaml keys. Has to be top-level
--mypy-only-local-stub
mypy will ignore errors from site-packages
--mypy-closed-schema
Use closed schema to validate YAML test cases,
which won't allow any extra keys
(does not work well with `--mypy-extension-hook`)
```
## Further reading
- [Testing mypy stubs, plugins, and types](https://sobolevn.me/2019/08/testing-mypy-types)
## License
[MIT](https://github.com/typeddjango/pytest-mypy-plugins/blob/master/LICENSE)
|
