1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
|
# Python SDK advanced testing guide
This guide covers advanced testing scenarios for Azure SDK for Python libraries.
## Table of contents
- [Mixin classes](#mixin-classes)
- [Pre-test setup](#pre-test-setup)
- [xunit-style setup](#xunit-style-setup)
- [Fixture setup](#fixture-setup)
- [Use HTTPS test proxy endpoint](#use-https-test-proxy-endpoint)
## Mixin classes
Many of our test suites use a base/mixin class to consolidate shared test logic. Mixin classes can define instance attributes to handle environment variables, make complex assertions, and more. By inheriting from these mixins, test classes can then share this logic throughout multiple files.
For example, in the Tables test suite there is a `_shared` directory containing two of these mixin classes: a
[sync version][mixin_sync] and an [async version][mixin_async].
```python
class TableTestCase(object):
def account_url(self, account, endpoint_type):
"""Return an url of storage account.
:param str storage_account: Storage account name
:param str storage_type: The Storage type part of the URL. Should be "table", or "cosmos", etc.
"""
try:
if endpoint_type == "table":
return account.primary_endpoints.table.rstrip("/")
if endpoint_type == "cosmos":
cosmos_suffix = os.getenv("TABLES_COSMOS_ENDPOINT_SUFFIX", DEFAULT_COSMOS_ENDPOINT_SUFFIX)
return f"https://{account.name}.table.{cosmos_suffix}"
except AttributeError: # Didn't find "account.primary_endpoints"
if endpoint_type == "table":
storage_suffix = os.getenv("TABLES_STORAGE_ENDPOINT_SUFFIX", DEFAULT_STORAGE_ENDPOINT_SUFFIX)
return f"https://{account}.table.{storage_suffix}"
if endpoint_type == "cosmos":
cosmos_suffix = os.getenv("TABLES_COSMOS_ENDPOINT_SUFFIX", DEFAULT_COSMOS_ENDPOINT_SUFFIX)
return f"https://{account}.table.{cosmos_suffix}"
...
def _assert_delete_retention_policy_equal(self, policy1, policy2):
"""Assert that two deletion retention policies are equal."""
if policy1 is None or policy2 is None:
assert policy1 == policy2
return
assert policy1.enabled == policy2.enabled
assert policy1.days == policy2.days
...
```
In action this class can be used in functional tests:
```python
class TestTable(AzureRecordedTestCase, TableTestCase):
@tables_decorator
@recorded_by_proxy
def test_create_properties(self, tables_storage_account_name, tables_primary_storage_account_key):
# # Arrange
account_url = self.account_url(tables_storage_account_name, "table")
ts = TableServiceClient(credential=tables_primary_storage_account_key, endpoint=account_url)
table_name = self._get_table_reference()
# Act
created = ts.create_table(table_name)
...
```
Or can be used in a unit test:
```python
class TestTablesUnit(TableTestCase):
...
def test_valid_url(self):
account = "fake_tables_account"
credential = "fake_tables_account_key_0123456789"
url = self.account_url(account, "tables")
client = TableClient(account_url=url, credential=credential)
assert client is not None
assert client.account_url == f"https://{account}.tables.core.windows.net/"
```
## Pre-test setup
Tests will often use shared resources that make sense to set up before tests execute. There are two recommended
approaches for this kind of setup, with each having benefits and drawbacks.
### xunit-style setup
Pytest has [documentation][xunit_setup] describing this setup style. For example:
```python
from devtools_testutils.azure_recorded_testcase import get_credential
class TestService(AzureRecordedTestCase):
def setup_method(self, method):
"""This method is called before each test in the class executes."""
credential = self.get_credential(ServiceClient) # utility from parent class
self.client = ServiceClient("...", credential)
@classmethod
def setup_class(cls):
"""This method is called only once, before any tests execute."""
credential = get_credential() # only module-level and classmethod utilities are available
cls.client = ServiceClient("...", credential)
```
The primary benefit of using `setup_method` is retaining access to the utilities provided your test class. You could
use `self.get_credential`, for example, to pick up our core utility for selecting a client credential based on your
environment. A drawback is that `setup_method` runs before each test method in the class, so your setup needs to be
idempotent to avoid issues caused by repeated invocations.
Alternatively, the class-level `setup_class` method runs once before all tests, but doesn't give you access to all
instance attributes on the class. You can still set attributes on the test class to reference from tests, and
module-level utilities can be used in place of instance attributes, as shown in the example above.
### Fixture setup
Pytest has [documentation][fixtures] explaining how to implement and use fixtures. For example, in a library's
`conftest.py`:
```python
from devtools_testutils.azure_recorded_testcase import get_credential
@pytest.fixture(scope="session")
def setup_teardown_fixture():
# Note that we can't reference AzureRecordedTestCase.get_credential but can use the module-level function
client = ServiceClient("...", get_credential())
client.set_up_resource()
yield # <-- Tests run here, and execution resumes after they finish
client.tear_down_resources()
```
We can then request the fixture from a test class:
```python
@pytest.mark.usefixtures("setup_teardown_fixture")
class TestService(AzureRecordedTestCase):
...
```
By requesting a fixture from the test class, the fixture will execute before any tests in the class do. Fixtures are the
preferred solution from pytest's perspective and offer a great deal of modular functionality.
As shown in the example above, the [`yield`][fixture_yield] command will defer to test execution -- after tests finish
running, the fixture code after `yield` will execute. This enables the use of a fixture for both setup and teardown.
However, fixtures in this context have similar drawbacks to the `setup_class` method described in
[xunit-style setup](#xunit-style-setup). Since their scope is outside of the test class, test class instance utilities
can't be accessed and class state can't be modified.
By convention, fixtures should be defined in a library's `tests/conftest.py` file. This will provide access to the
fixture across test files, and the fixture can be requested without having to manually import it.
## Use HTTPS test proxy endpoint
By default, the test proxy is reached at `http://localhost:5000`. Service requests are ultimately made as usual with a
secure connection, but some libraries may require that the immediate proxy endpoint uses an SSL connection.
In that scenario, you can set the `PROXY_URL` environment variable to target the test proxy at an HTTPS URL:
```text
PROXY_URL='https://localhost:5001'
```
The test proxy's certificate is [automatically configured][cert_setup] during proxy startup, though async tests may
exhibit [inconsistent behavior][async_cert_troubleshoot].
<!-- Links -->
[async_cert_troubleshoot]: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/test_proxy_troubleshooting.md#servicerequesterror-cannot-connect-to-host
[cert_setup]: https://github.com/Azure/azure-sdk-for-python/blob/9958caf6269247f940c697a3f982bbbf0a47a19b/eng/tools/azure-sdk-tools/devtools_testutils/proxy_startup.py#L210
[fixture_yield]: https://docs.pytest.org/latest/how-to/fixtures.html#yield-fixtures-recommended
[fixtures]: https://docs.pytest.org/en/latest/how-to/fixtures.html
[mixin_async]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/tables/azure-data-tables/tests/_shared/asynctestcase.py
[mixin_sync]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/tables/azure-data-tables/tests/_shared/testcase.py
[xunit_setup]: https://docs.pytest.org/en/latest/how-to/xunit_setup.html
|
