1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
|
Testing
=======
General mechanisms used by git-ubuntu tests
-------------------------------------------
We use `pytest <https://pytest.org>`_. On Ubuntu, with dependencies installed,
it should be sufficient to just run :program:`py.test-3` at the top level to
run the tests locally. In CI, we additionally use `pylint
<https://pylint.org>`_.
We ship the test suite and this is available from the snap as
:program:`git-ubuntu.self-test`. This is done by CI prior to publication of the
snap.
You can use :program:`git-ubuntu.self-test` from the snap to test a local
development tree with dependencies from the snap. TBC: how.
Many tests are parameterized using `pytest's parametrize mechanism
<https://docs.pytest.org/en/latest/how-to/parametrize.html>`_.
General approach to testing
---------------------------
We try to test what is most likely to fail.
For example, testing that we call an API in the expected way when the code that
does it is straightforward is of limited value if the risk is that we've
misunderstood the API and are calling it wrong.
Most code should come with tests. Some old code doesn't have tests; we are
trying to improve this incrementally by writing tests in areas of code that we
touch. Don't feel that you have to go down the rabbit hole. Landing a simple
fix doesn't require that you raise the quality of everything you touch, but
some incremental improvement is appreciated such that it's possible to say that
the quality is higher than it was previously.
git-ubuntu operations often operate on inputs that are git repositories or
source packages, and then outputs the same. These inputs and outputs are
fundamentally data structures just as a simple list is a data structure. We
might test a function that operates on a list by providing sample lists as
inputs and making assertions about the output. Similarly, most git-ubuntu tests
provide sample repositories and source packages as inputs, and make assertions
about output repositories.
Providing an input repository or input source package is a tedious and
repetitive task, so it is automated. Input source repositories can be specified
as data structures by using the :mod:`gitubuntu.source_builder` module.
Likewise, the :mod:`gitubuntu.repo_builder` module allows git repositories to
be specified as data structures. Where an input git repository needs a source
package in a tree object, :class:`gitubuntu.repo_builder.SourceTree` can be
used to construct it.
Please avoid checking in already constructed git repositories or source trees
as test inputs. These will inevitably contains binary blobs and/or boilerplate.
These are opaque and hide the important part of data structure that actually
matter to the tests, making such tests harder to work with. Instead, the
"builder" style described above eliminates binary blobs and, with default
arguments, the boilerplate. Where edge case inputs are required that are not
already supported by the builder modules, the builder modules themselves should
be enhanced instead. And if the way the builder modules are called become
obtuse, then they should be refactored together with the tests.
Commonly used fixtures
----------------------
git-ubuntu tests make extensive use of `pytest's fixture functionality
<https://docs.pytest.org/en/latest/how-to/fixtures.html#how-to-fixtures>`_.
You should make yourself familiar with the pytest built-in tmpdir fixture. Note
that it supplies a :class:`py.path` object, not a string. In some cases, this
needs explicit conversion with :func:`str`.
git-ubuntu uses the `pygit2 <https://pygit2.org>`_ library extensively. However
git-ubuntu repositories have a particular structure for which we have many
helper methods so we wrap the :class:`pygit2.Repository` in a
:class:`~gitubuntu.git_repository.GitUbuntuRepository`. The underlying
:class:`pygit2.Repository` object is available via the
:attr:`gitubuntu.git_repository.GitUbuntuRepository.raw_repo` attribute on a
:class:`~gitubuntu.git_repository.GitUbuntuRepository` instance.
The :func:`gitubuntu.test_fixtures.repo` fixture provides a
:class:`gitubuntu.git_repository.GitUbuntuRepository` instance in a temporary
directory. Alternatively you can use the lower level
:func:`gitubuntu.test_fixtures.pygit2_repo` fixture to get an unwrapped
:class:`pygit2.Repository` instance in a temporary directory.
:mod:`gitubuntu.repo_builder` operates on :class:`pygit2.Repository` objects
directly, so tests can either use a :func:`gitubuntu.test_fixtures.pygit2_repo`
fixture directly, or they can use a :func:`gitubuntu.test_fixtures.repo`
fixture and access its
:attr:`gitubuntu.git_repository.GitUbuntuRepository.raw_repo` attribute to use
with :class:`gitubuntu.repo_builder.Repo`.
Example
-------
Let's say you want to test a function that looks up the package version string
from debian/changelog in a git tree object::
def test_changelog_version_lookup(repo):
# repo is our test fixture. When pytest runs this test, it will supply
# it with an empty GitUbuntuRepository instance in a temporary
# directory.
# First we describe a git repository in a data structure and ask it to
# write itself into repo.raw_repo.
repo_builder.Repo(
commits=[
# We give the commit a name here, purely for the internal
# reference that comes later
repo_builder.Commit(name='root', tree=repo_builder.SourceTree())
],
# The Placeholder() instance is used to internally reference the
# commit we created earlier
tags={'root': repo_builder.Placeholder('root')},
).write(repo.raw_repo)
# In the line above, raw_repo is the underlying pygit2.Repository
# instance
# Now our git repository contains a commit that contains a source
# package tree and a tag that refers to the commit.
# We can fetch the tree object using normal pygit2 operations
tree = (
repo
.raw_repo.lookup_reference('refs/tags/root')
.peel(pygit2.Tree)
)
# Now we can test the our code in the normal way.
# repo_builder.SourceTree() above defaults to a version of '1-1' and we
# didn't override it, so that's the version string that our
# function-under-test should return.
actual_result = repo.get_changelog_versions_from_treeish(str(tree.id))
assert actual_result == '1-1'
|
