1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
|
.. _test_docs:
==================================
Murano automated tests description
==================================
This page describes automated tests for a Murano project:
* where tests are located
* how they are run
* how to execute tests on a local machine
* how to find the root of problems with FAILed tests
Murano continuous integration service
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Murano project has separate CI server, which runs tests for all commits and
verifies that new code does not break anything.
Murano CI uses OpenStack QA cloud for testing infrastructure.
Murano CI url: https://murano-ci.mirantis.com/jenkins/ Anyone can login
to that server, using launchpad credentials.
There you can find each job for each repository: one for **murano** and
another one for **murano-dashboard**.
* ``gate-murano-dashboard-ubuntu\*`` verifies each commit to
the murano-dashboard repository
* ``gate-murano-ubuntu\*`` verifies each commit to the murano repository
Other jobs allow one to build and test Murano documentation and to perform other
useful work to support the Murano CI infrastructure.
All jobs are run following a fresh installation of the operating system and all components
are installed on each run.
UI tests
~~~~~~~~
The Murano project has a web user interface and all possible user scenarios
should be tested.
All UI tests are located at
``https://git.openstack.org/cgit/openstack/murano-dashboard/tree/muranodashboard/tests/functional``.
Automated tests for the Murano web UI are written in Python using the special
Selenium library. This library is used to automate web browser interactions
with Python. See official `Selenium documentation <https://selenium-python.readthedocs.org/>`_
for details.
Prerequisites:
--------------
* Install the Python module called nose using either the
:command:`easy_install nose` or :command:`pip install nose` command.
This will install the nose libraries, as well as the ``nosetests`` script,
which you can use to automatically discover and run tests.
* Install external Python libraries, which are required for the Murano web UI
tests: ``testtools`` and ``selenium``.
* Verify that you have one of the following web browsers installed:
* Mozilla Firefox 46.0
.. note::
If you do not have Firefox package out of the box,
install and remove it. Otherwise, you will need to install
dependent libraries manually. To downgrade Firefox:
.. code-block:: console
apt-get remove firefox
wget https://ftp.mozilla.org/pub/firefox/releases/46.0/linux-x86_64/en-US/firefox-46.0.tar.bz2
tar -xjf firefox-46.0.tar.bz2
rm -rf /opt/firefox
mv firefox /opt/firefox46
ln -s /opt/firefox46/firefox /usr/bin/firefox
* Google Chrome
* To run the tests on a remote server, configure the remote X server.
Use VNC Software to see the test results in real-time.
#. Specify the display environment variable:
.. code-block:: console
$DISPLAY=: <value>
#. Configure remote X server and VNC Software by typing:
.. code-block:: console
apt-get install xvfb xfonts-100dpi xfonts-75dpi xfonts-cyrillic xorg dbus-x11
"Xvfb -fp "/usr/share/fonts/X11/misc/" :$DISPLAY -screen 0 "1280x1024x16" &"
apt-get install --yes x11vnc
x11vnc -bg -forever -nopw -display :$DISPLAY -ncache 10
sudo iptables -I INPUT 1 -p tcp --dport 5900 -j ACCEPT
Download and run tests
----------------------
To download and run the tests:
#. Verify that all additional components has been installed.
#. Clone the ``murano-dashboard`` git repository:
.. code-block:: console
git clone git://git.openstack.org/openstack/murano-dashboard
#. Change the default settings:
#. Specify the Murano Repository URL variable for Horizon local settings
in ``murano_dashboard/muranodashboard/local/local_settings.d/_50_murano.py``:
.. code-block:: console
MURANO_REPO_URL = 'http://localhost:8099'
#. Copy ``muranodashboard/tests/functional/config/config.conf.sample`` to
``config.conf``.
#. Set appropriate URLs and credentials for your OpenStack lab.
Only Administrator user credentials are appropriate.
.. code-block:: console
[murano]
horizon_url = http://localhost/dashboard
murano_url = http://localhost:8082
user = ***
password = ***
tenant = ***
keystone_url = http://localhost:5000/v3
All tests are kept in ``sanity_check.py`` and divided into 10 test suites:
* TestSuiteSmoke - verification of Murano panels; checks that they can be open
without errors.
* TestSuiteEnvironment - verification of all operations with environment are
finished successfully.
* TestSuiteImage - verification of operations with images.
* TestSuiteFields - verification of custom fields validators.
* TestSuitePackages - verification of operations with Murano packages.
* TestSuiteApplications - verification of Application Catalog page and of
application creation process.
* TestSuiteAppsPagination - verification of apps pagination in case of many
applications installed.
* TestSuiteRepository - verification of importing packages and bundles.
* TestSuitePackageCategory - verification of main operations with categories.
* TestSuiteCategoriesPagination - verification of categories pagination
in case of many categories created.
* TestSuiteMultipleEnvironments - verification of ability to apply action
to multiple environments.
To run the tests follow these instructions:
* To run all tests:
.. code-block:: console
nosetests sanity_check.py
* To run a single suite:
.. code-block:: console
nosetests sanity_check.py:<test suite name>
* To run a single test:
.. code-block:: console
nosetests sanity_check.py:<test suite name>.<test name>
In case of successful execution, you should see something like this:
.. code-block:: console
.........................
Ran 34 tests in 1.440s
OK
In case of failure, the folder with screenshots of the last operation of
tests that finished with errors would be created.
It is located in ``muranodashboard/tests/functional`` folder.
There are also a number of command line options that can be used to control
the test execution and generated outputs. For more details about ``nosetests``,
type:
.. code-block:: console
nosetests -h
Tempest tests
~~~~~~~~~~~~~
All Murano services have tempest-based automated tests, which verify
API interfaces and deployment scenarios.
Tempest tests for Murano are located at ``https://git.openstack.org/cgit/openstack/murano/tree/murano/tests/functional``.
The following Python files contain basic test suites for different Murano components.
API tests
---------
Murano API tests are run on the devstack gate located at
``https://git.openstack.org/cgit/openstack/murano/tree/murano/tests/functional/api``.
* ``test_murano_envs.py`` contains test suite with actions on murano
environments (create, delete, get, and others).
* ``test_murano_sessions.py`` contains test suite with actions on murano
sessions (create, delete, get, and others).
* ``test_murano_services.py`` contains test suite with actions on murano
services (create, delete, get, and others).
* ``test_murano_repository.py`` contains test suite with actions on murano
package repository.
Engine tests
------------
Murano Engine Tests are run on murano-ci at ``https://git.openstack.org/cgit/openstack/murano/tree/murano/tests/functional/engine``:
* ``base.py`` contains base test class and tests with actions on deploy
Murano services such as Telnet and Apache.
Command-line interface tests
----------------------------
Murano CLI tests are currently in the middle of creation. The current scope
is read-only operations on a cloud that are hard to test through unit tests.
All tests have description and execution steps in their docstrings.
|
