1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
|
Frequently Asked Questions
==========================
.. note:: This section provides a collection of questions with
answers that don't otherwise fit in the rest of the
documentation. If something is missing, please create an
issue_.
As this document grows it will gain a more refined
structure.
.. highlight:: yaml
General
~~~~~~~
Is gabbi only for testing Python-based APIs?
--------------------------------------------
No, you can use :doc:`gabbi-run <runner>` to test an HTTP service
built in any programming language.
How do I run just one test?
---------------------------
Each YAML file contains a sequence of tests, each test within each file has a
name. That name is translated to the name of the test by replacing spaces with
an ``_``.
When running tests that are :doc:`generated dynamically <loader>`, filtering
based on the test name prior to the test being collected will not work in some
test runners. Test runners that use a ``--load-list`` functionality can be
convinced to filter after discovery.
`pytest` does this directly with the ``-k`` keyword flag.
When using testrepository with tox as used in gabbi's own tests it is possible
to pass a filter in the tox command::
tox -epy27 -- get_the_widget
When using ``testtools.run`` and similar test runners it's a bit more
complicated. It is necessary to provide the full name of the test as a list to
``--load-list``::
python -m testtools.run --load-list \
<(echo package.tests.test_api.yamlfile_get_the_widge.test_request)
How do I run just one test, without running prior tests in a sequence?
----------------------------------------------------------------------
By default, when you select a single test to run, all tests prior to that one
in a file will be run as well: the file is treated as as sequence of dependent
tests. If you do not want this you can adjust the ``use_prior_test`` test
:ref:`metadata <metadata>` in one of three ways:
* Set it in the YAML file for the one test you are concerned with.
* Set the ``defaults`` for all tests in that file.
* set ``use_prior_test`` to false when calling :func:`~gabbi.driver.build_tests`
Be aware that doing this breaks a fundamental assumption that gabbi
makes about how tests work. Any :ref:`substitutions <state-substitution>`
will fail.
Testing Style
~~~~~~~~~~~~~
Can I have variables in my YAML file?
-------------------------------------
Gabbi provides the ``$ENVIRON`` :ref:`substitution
<state-substitution>` which can operate a bit like variables that
are set elsewhere and then used in the tests defined by the YAML.
If you find it necessary to have variables within a single YAML file
you take advantage of YAML `alias nodes`_ list this::
vars:
- &uuid_1 5613AABF-BAED-4BBA-887A-252B2D3543F8
tests:
- name: send a uuid to a post
POST: /resource
request_headers:
content-type: application/json
data:
uuid: *uuid_1
You can alias all sorts of nodes, not just single items. Be aware
that the replacement of an alias node happens while the YAML is
being loaded, before gabbi does any processing.
.. _alias nodes: http://www.yaml.org/spec/1.2/spec.html#id2786196
How many tests should be put in one YAML file?
----------------------------------------------
For the sake of readability it is best to keep each YAML file
relatively short. Since each YAML file represents a sequence of
requests, it usually makes sense to create a new file when a test is
not dependent on any before it.
It's tempting to put all the tests for any resource or URL in the
same file, but this eventually leads to files that are too long and
are thus difficult to read.
.. _issue: https://github.com/cdent/gabbi/issues
|
