1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
|
Feature: Undefined Step
. TERMINOLOGY:
. * An undefined step is a step without matching step implementation.
.
. SPECIFICATION:
. * An undefined step should be reported after the run.
. * An undefined step should cause its scenario to fail.
. * If an undefined step is detected the remaining scenario steps are skipped.
. * All undefined steps in a scenario should be reported (issue #42).
. * Undefined steps should be detected even after a step fails in a scenario.
. * Each undefined step should be reported only once.
. * If a scenario is disabled (by tag expression, etc.),
. the undefined step discovery should not occur.
. This allows to prepare scenarios that are not intended to run (yet).
. * Option --dry-run should discover undefined steps, too.
.
. RELATED TO:
. * issue #42 Multiple undefined steps in same scenario are detected.
@setup
Scenario: Feature Setup
Given a new working directory
And a file named "features/steps/passing_steps.py" with:
"""
from behave import step
@step('a step passes')
def step_passes(context):
pass
@step('a step fails')
def step_fails(context):
assert False, "XFAIL"
"""
And a file named "features/undefined_last_step.feature" with:
"""
Feature:
Scenario:
Given a step passes
When an undefined step is used
"""
Scenario: An undefined step should be reported
When I run "behave -f plain -T features/undefined_last_step.feature"
Then it should fail
And the command output should contain:
"""
Feature:
Scenario:
Given a step passes ... passed
When an undefined step is used ... undefined
"""
And the command output should contain:
"""
You can implement step definitions for undefined steps with these snippets:
@when(u'an undefined step is used')
def step_impl(context):
raise NotImplementedError(u'STEP: When an undefined step is used')
"""
And an undefined-step snippet should exist for "When an undefined step is used"
Scenario: An undefined step should cause its scenario to fail
When I run "behave -f plain features/undefined_last_step.feature"
Then it should fail with:
"""
0 features passed, 1 failed, 0 skipped
0 scenarios passed, 1 failed, 0 skipped
1 step passed, 0 failed, 0 skipped, 1 undefined
"""
Scenario: Additional scenario steps after an undefined step are skipped
Given a file named "features/undefined_step_and_more.feature" with:
"""
Feature:
Scenario:
Given a step passes
When an undefined step is used
Then a step passes
And a step fails
"""
When I run "behave -f plain -T features/undefined_step_and_more.feature"
Then it should fail with:
"""
0 features passed, 1 failed, 0 skipped
0 scenarios passed, 1 failed, 0 skipped
1 step passed, 0 failed, 2 skipped, 1 undefined
"""
And the command output should contain:
"""
Feature:
Scenario:
Given a step passes ... passed
When an undefined step is used ... undefined
"""
Scenario: Two undefined steps in same scenario should be detected
Given a file named "features/two_undefined_steps1.feature" with:
"""
Feature:
Scenario:
Given a step passes
When an undefined step is used
And a step fails
Then another undefined step is used
"""
When I run "behave -f plain -T features/two_undefined_steps1.feature"
Then it should fail with:
"""
0 features passed, 1 failed, 0 skipped
0 scenarios passed, 1 failed, 0 skipped
1 step passed, 0 failed, 1 skipped, 2 undefined
"""
And the command output should contain:
"""
Feature:
Scenario:
Given a step passes ... passed
When an undefined step is used ... undefined
"""
And undefined-step snippets should exist for:
| Step |
| When an undefined step is used |
| Then another undefined step is used |
But the command output should not contain:
"""
And a step fails ... skipped
Then another undefined step is used ... undefined
"""
Scenario: Two undefined steps in different scenarios
Given a file named "features/two_undefined_steps2.feature" with:
"""
Feature:
Scenario:
Given a step passes
When an undefined step is used
Scenario:
Given another undefined step is used
When a step passes
"""
When I run "behave -f plain -T features/two_undefined_steps2.feature"
Then it should fail with:
"""
0 features passed, 1 failed, 0 skipped
0 scenarios passed, 2 failed, 0 skipped
1 step passed, 0 failed, 1 skipped, 2 undefined
"""
And the command output should contain:
"""
Feature:
Scenario:
Given a step passes ... passed
When an undefined step is used ... undefined
Scenario:
Given another undefined step is used ... undefined
"""
And undefined-step snippets should exist for:
| Step |
| When an undefined step is used |
| Given another undefined step is used |
Scenario: Undefined step in Scenario Outline
Given a file named "features/undefined_step_in_scenario_outline.feature" with:
"""
Feature:
Scenario Outline:
Given a step <outcome1>
When an undefined step is used
Then a step <outcome2>
Examples:
| outcome1 | outcome2 |
| passes | passes |
| passes | fails |
| fails | passes |
| fails | fails |
"""
When I run "behave -f plain -T features/undefined_step_in_scenario_outline.feature"
Then it should fail with:
"""
0 features passed, 1 failed, 0 skipped
0 scenarios passed, 4 failed, 0 skipped
2 steps passed, 2 failed, 4 skipped, 4 undefined
"""
And an undefined-step snippet should exist for "When an undefined step is used"
And the command output should contain:
"""
Feature:
Scenario Outline: -- @1.1
Given a step passes ... passed
When an undefined step is used ... undefined
Scenario Outline: -- @1.2
Given a step passes ... passed
When an undefined step is used ... undefined
Scenario Outline: -- @1.3
Given a step fails ... failed
Assertion Failed: XFAIL
Scenario Outline: -- @1.4
Given a step fails ... failed
Assertion Failed: XFAIL
"""
Scenario: Two undefined step in Scenario Outline
Given a file named "features/two_undefined_step_in_scenario_outline.feature" with:
"""
Feature:
Scenario Outline:
Given a step <outcome1>
When an undefined step is used
Then a step <outcome2>
And another undefined step is used
Examples:
| outcome1 | outcome2 |
| passes | passes |
| passes | fails |
| fails | passes |
| fails | fails |
"""
When I run "behave -f plain features/two_undefined_step_in_scenario_outline.feature"
Then it should fail with:
"""
0 features passed, 1 failed, 0 skipped
0 scenarios passed, 4 failed, 0 skipped
2 steps passed, 2 failed, 4 skipped, 8 undefined
"""
And undefined-step snippets should exist for:
| Step |
| When an undefined step is used |
| Then another undefined step is used |
Scenario: Undefined steps are detected if scenario is selected via tag
Given a file named "features/undefined_steps_with_tagged_scenario.feature" with:
"""
Feature:
@selected
Scenario: S1
Given a step passes
And an undefined step Alice
When a step fails
Then an undefined step Bob
@selected
Scenario: S2
Given a step passes
When an undefined step Charly
@not_selected
Scenario: S3
Given an undefined step Delta
Then a step fails
"""
When I run "behave -f plain --tags=@selected features/undefined_steps_with_tagged_scenario.feature"
Then it should fail with:
"""
0 features passed, 1 failed, 0 skipped
0 scenarios passed, 2 failed, 1 skipped
2 steps passed, 0 failed, 3 skipped, 3 undefined
"""
And undefined-step snippets should exist for:
| Step |
| Given an undefined step Alice |
| Then an undefined step Bob |
| When an undefined step Charly |
But undefined-step snippets should not exist for:
| Step |
| Given a step passes |
| Given an undefined step Delta |
| When a step fails |
| Then a step fails |
Scenario: Undefined steps are detected if --dry-run option is used
When I run "behave -f plain --dry-run features/undefined_steps_with_tagged_scenario.feature"
Then it should fail with:
"""
0 features passed, 0 failed, 0 skipped, 1 untested
0 scenarios passed, 0 failed, 0 skipped, 3 untested
0 steps passed, 0 failed, 0 skipped, 4 undefined, 4 untested
"""
And undefined-step snippets should exist for:
| Step |
| Given an undefined step Alice |
| Then an undefined step Bob |
| When an undefined step Charly |
| Given an undefined step Delta |
But undefined-step snippets should not exist for:
| Step |
| Given a step passes |
| When a step fails |
| Then a step fails |
|
