File: 08_Testing.pod

package info (click to toggle)
libcatalyst-manual-perl 5.9004-1
  • links: PTS, VCS
  • area: main
  • in suites: wheezy
  • size: 820 kB
  • sloc: perl: 1,974; makefile: 2
file content (448 lines) | stat: -rw-r--r-- 16,327 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
=head1 NAME

Catalyst::Manual::Tutorial::08_Testing - Catalyst Tutorial - Chapter 8: Testing


=head1 OVERVIEW

This is B<Chapter 8 of 10> for the Catalyst tutorial.

L<Tutorial Overview|Catalyst::Manual::Tutorial>

=over 4

=item 1

L<Introduction|Catalyst::Manual::Tutorial::01_Intro>

=item 2

L<Catalyst Basics|Catalyst::Manual::Tutorial::02_CatalystBasics>

=item 3

L<More Catalyst Basics|Catalyst::Manual::Tutorial::03_MoreCatalystBasics>

=item 4

L<Basic CRUD|Catalyst::Manual::Tutorial::04_BasicCRUD>

=item 5

L<Authentication|Catalyst::Manual::Tutorial::05_Authentication>

=item 6

L<Authorization|Catalyst::Manual::Tutorial::06_Authorization>

=item 7

L<Debugging|Catalyst::Manual::Tutorial::07_Debugging>

=item 8

B<08_Testing>

=item 9

L<Advanced CRUD|Catalyst::Manual::Tutorial::09_AdvancedCRUD>

=item 10

L<Appendices|Catalyst::Manual::Tutorial::10_Appendices>

=back


=head1 DESCRIPTION

You may have noticed that the Catalyst Helper scripts automatically
create basic C<.t> test scripts under the C<t> directory.  This chapter
of the tutorial briefly looks at how these tests can be used not only to
ensure that your application is working correctly at the present time,
but also provide automated regression testing as you upgrade various
pieces of your application over time.

Source code for the tutorial in included in the F</home/catalyst/Final>
directory of the Tutorial Virtual machine (one subdirectory per
chapter).  There are also instructions for downloading the code in
L<Catalyst::Manual::Tutorial::01_Intro>.

For an excellent introduction to learning the many benefits of testing
your Perl applications and modules, you might want to read 'Perl
Testing: A Developer's Notebook' by Ian Langworth and chromatic.


=head1 RUNNING THE "CANNED" CATALYST TESTS

There are a variety of ways to run Catalyst and Perl tests (for example,
C<perl Makefile.PL> and C<make test>), but one of the easiest is with
the C<prove> command.  For example, to run all of the tests in the C<t>
directory, enter:

    $ prove -wl t

There will be a lot of output because we have the C<-Debug> flag enabled
in C<lib/MyApp.pm> (see the C<CATALYST_DEBUG=0> tip below for a quick
and easy way to reduce the clutter).  Look for lines like this for
errors:

    #   Failed test 'Request should succeed'
    #   at t/controller_Books.t line 8.
    # Looks like you failed 1 test of 3.

The redirection used by the Authentication plugins will cause several
failures in the default tests.  You can fix this by making the following
changes:

1) Change the line in C<t/01app.t> that reads:

    ok( request('/')->is_success, 'Request should succeed' );

to:

    ok( request('/login')->is_success, 'Request should succeed' );

2) Change the line in C<t/controller_Logout.t> that reads:

    ok( request('/logout')->is_success, 'Request should succeed' );

to:

    ok( request('/logout')->is_redirect, 'Request should succeed' );

3) Change the line in C<t/controller_Books.t> that reads:

    ok( request('/books')->is_success, 'Request should succeed' );

to:

    ok( request('/books')->is_redirect, 'Request should succeed' );

4) Add the following statement to the top of C<t/view_HTML.t>:

    use MyApp;

As you can see in the C<prove> command line above, the C<-l> option (or
C<--lib> if you prefer) is used to set the location of the Catalyst
C<lib> directory.  With this command, you will get all of the usual
development server debug output, something most people prefer to disable
while running tests cases.  Although you can edit the C<lib/MyApp.pm> to
comment out the C<-Debug> plugin, it's generally easier to simply set
the C<CATALYST_DEBUG=0> environment variable.  For example:

    $ CATALYST_DEBUG=0 prove -wl t

During the C<t/02pod> and C<t/03podcoverage> tests, you might notice the
C<all skipped: set TEST_POD to enable this test> warning message.  To
execute the Pod-related tests, add C<TEST_POD=1> to the C<prove>
command:

    $ CATALYST_DEBUG=0 TEST_POD=1 prove -wl t

If you omitted the Pod comments from any of the methods that were
inserted, you might have to go back and fix them to get these tests to
pass. :-)

Another useful option is the C<verbose> (C<-v>) option to C<prove>.  It
prints the name of each test case as it is being run:

    $ CATALYST_DEBUG=0 prove -vwl t


=head1 RUNNING A SINGLE TEST

You can also run a single script by appending its name to the C<prove>
command. For example:

    $ CATALYST_DEBUG=0 prove -wl t/01app.t

Also note that you can also run tests directly from Perl without
C<prove>.  For example:

    $ CATALYST_DEBUG=0 perl -w -Ilib t/01app.t


=head1 ADDING YOUR OWN TEST SCRIPT

Although the Catalyst helper scripts provide a basic level of checks
"for free," testing can become significantly more helpful when you write
your own tests to exercise the various parts of your application.  The
L<Test::WWW::Mechanize::Catalyst> module is very popular for writing
these sorts of test cases.  This module extends L<Test::WWW::Mechanize>
(and therefore L<WWW::Mechanize>) to allow you to automate the action of
a user "clicking around" inside your application.  It gives you all the
benefits of testing on a live system without the messiness of having to
use an actual web server, and a real person to do the clicking.

To create a sample test case, open the C<t/live_app01.t> file in your
editor and enter the following:

    #!/usr/bin/env perl
    
    use strict;
    use warnings;
    use Test::More;
    
    # Need to specify the name of your app as arg on next line
    # Can also do:
    #   use Test::WWW::Mechanize::Catalyst "MyApp";
    
    BEGIN { use_ok("Test::WWW::Mechanize::Catalyst" => "MyApp") }
    
    # Create two 'user agents' to simulate two different users ('test01' & 'test02')
    my $ua1 = Test::WWW::Mechanize::Catalyst->new;
    my $ua2 = Test::WWW::Mechanize::Catalyst->new;
    
    # Use a simplified for loop to do tests that are common to both users
    # Use get_ok() to make sure we can hit the base URL
    # Second arg = optional description of test (will be displayed for failed tests)
    # Note that in test scripts you send everything to 'http://localhost'
    $_->get_ok("http://localhost/", "Check redirect of base URL") for $ua1, $ua2;
    # Use title_is() to check the contents of the <title>...</title> tags
    $_->title_is("Login", "Check for login title") for $ua1, $ua2;
    # Use content_contains() to match on text in the html body
    $_->content_contains("You need to log in to use this application",
        "Check we are NOT logged in") for $ua1, $ua2;
    
    # Log in as each user
    # Specify username and password on the URL
    $ua1->get_ok("http://localhost/login?username=test01&password=mypass", "Login 'test01'");
    # Could make user2 like user1 above, but use the form to show another way
    $ua2->submit_form(
        fields => {
            username => 'test02',
            password => 'mypass',
        });
    
    # Go back to the login page and it should show that we are already logged in
    $_->get_ok("http://localhost/login", "Return to '/login'") for $ua1, $ua2;
    $_->title_is("Login", "Check for login page") for $ua1, $ua2;
    $_->content_contains("Please Note: You are already logged in as ",
        "Check we ARE logged in" ) for $ua1, $ua2;
    
    # 'Click' the 'Logout' link (see also 'text_regex' and 'url_regex' options)
    $_->follow_link_ok({n => 4}, "Logout via first link on page") for $ua1, $ua2;
    $_->title_is("Login", "Check for login title") for $ua1, $ua2;
    $_->content_contains("You need to log in to use this application",
        "Check we are NOT logged in") for $ua1, $ua2;
    
    # Log back in
    $ua1->get_ok("http://localhost/login?username=test01&password=mypass",
        "Login 'test01'");
    $ua2->get_ok("http://localhost/login?username=test02&password=mypass",
        "Login 'test02'");
    # Should be at the Book List page... do some checks to confirm
    $_->title_is("Book List", "Check for book list title") for $ua1, $ua2;
    
    $ua1->get_ok("http://localhost/books/list", "'test01' book list");
    $ua1->get_ok("http://localhost/login", "Login Page");
    $ua1->get_ok("http://localhost/books/list", "'test01' book list");
    
    $_->content_contains("Book List", "Check for book list title") for $ua1, $ua2;
    # Make sure the appropriate logout buttons are displayed
    $_->content_contains("/logout\">User Logout</a>",
        "Both users should have a 'User Logout'") for $ua1, $ua2;
    $ua1->content_contains("/books/form_create\">Admin Create</a>",
        "'test01' should have a create link");
    $ua2->content_lacks("/books/form_create\">Admin Create</a>",
        "'test02' should NOT have a create link");
    
    $ua1->get_ok("http://localhost/books/list", "View book list as 'test01'");
    
    # User 'test01' should be able to create a book with the "formless create" URL
    $ua1->get_ok("http://localhost/books/url_create/TestTitle/2/4",
        "'test01' formless create");
    $ua1->title_is("Book Created", "Book created title");
    $ua1->content_contains("Added book 'TestTitle'", "Check title added OK");
    $ua1->content_contains("by 'Stevens'", "Check author added OK");
    $ua1->content_contains("with a rating of 2.", "Check rating added");
    # Try a regular expression to combine the previous 3 checks & account for whitespace
    $ua1->content_like(qr/Added book 'TestTitle'\s+by 'Stevens'\s+with a rating of 2./,
        "Regex check");
    
    # Make sure the new book shows in the list
    $ua1->get_ok("http://localhost/books/list", "'test01' book list");
    $ua1->title_is("Book List", "Check logged in and at book list");
    $ua1->content_contains("Book List", "Book List page test");
    $ua1->content_contains("TestTitle", "Look for 'TestTitle'");
    
    # Make sure the new book can be deleted
    # Get all the Delete links on the list page
    my @delLinks = $ua1->find_all_links(text => 'Delete');
    # Use the final link to delete the last book
    $ua1->get_ok($delLinks[$#delLinks]->url, 'Delete last book');
    # Check that delete worked
    $ua1->content_contains("Book List", "Book List page test");
    $ua1->content_like(qr/Deleted book \d+/, "Deleted book #");
    
    # User 'test02' should not be able to add a book
    $ua2->get_ok("http://localhost/books/url_create/TestTitle2/2/5", "'test02' add");
    $ua2->content_contains("Unauthorized!", "Check 'test02' cannot add");
    
    done_testing;

The C<live_app.t> test cases uses copious comments to explain each step
of the process.  In addition to the techniques shown here, there are a
variety of other methods available in L<Test::WWW::Mechanize::Catalyst>
(for example, regex-based matching). Consult
L<Test::WWW::Mechanize::Catalyst>, L<Test::WWW::Mechanize>,
L<WWW::Mechanize>, and L<Test::More> for more detail.

B<TIP>: For I<unit tests> vs. the "full application tests" approach used
by L<Test::WWW::Mechanize::Catalyst>, see L<Catalyst::Test>.

B<Note:> The test script does not test the C<form_create> and
C<form_create_do> actions.  That is left as an exercise for the reader
(you should be able to complete that logic using the existing code as a
template).

To run the new test script, use a command such as:

    $ CATALYST_DEBUG=0 prove -vwl t/live_app01.t

or

    $ DBIC_TRACE=0 CATALYST_DEBUG=0 prove -vwl t/live_app01.t

Experiment with the C<DBIC_TRACE>, C<CATALYST_DEBUG> and C<-v> settings.
If you find that there are errors, use the techniques discussed in the
"Catalyst Debugging" section (Chapter 7) to isolate and fix any
problems.

If you want to run the test case under the Perl interactive debugger,
try a command such as:

    $ DBIC_TRACE=0 CATALYST_DEBUG=0 perl -d -Ilib t/live_app01.t

Note that although this tutorial uses a single custom test case for
simplicity, you may wish to break your tests into different files for
better organization.

B<TIP:> If you have a test case that fails, you will receive an error
similar to the following:

    #   Failed test 'Check we are NOT logged in'
    #   in t/live_app01.t at line 31.
    #     searched: "\x{0a}<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Tran"...
    #   can't find: "You need to log in to use this application."

Unfortunately, this only shows us the first 50 characters of the HTML
returned by the request -- not enough to determine where the problem
lies.  A simple technique that can be used in such situations is to
temporarily insert a line similar to the following right after the
failed test:

    diag $ua1->content;

This will cause the full HTML returned by the request to be displayed.

Another approach to see the full HTML content at the failure point in a
series of tests would be to insert a "C<$DB::single=1;> right above the
location of the failure and run the test under the Perl debugger (with
C<-d>) as shown above.  Then you can use the debugger to explore the
state of the application right before or after the failure.


=head1 SUPPORTING BOTH PRODUCTION AND TEST DATABASES

You may wish to leverage the techniques discussed in this tutorial to
maintain both a "production database" for your live application and a
"testing database" for your test cases.  One advantage to
L<Test::WWW::Mechanize::Catalyst> is that it runs your full application;
however, this can complicate things when you want to support multiple
databases.

=head2 DATABASE CONFIG SWITCHING IN YOUR MODEL CLASS

One solution is to allow the database specification to be overridden
with an environment variable.  For example, open
C<lib/MyApp/Model/DB.pm> in your editor and change the
C<__PACKAGE__-E<gt>config(...> declaration to resemble:

    my $dsn = $ENV{MYAPP_DSN} ||= 'dbi:SQLite:myapp.db';
    __PACKAGE__->config(
        schema_class => 'MyApp::Schema',
    
        connect_info => {
            dsn => $dsn,
            user => '',
            password => '',
            on_connect_do => q{PRAGMA foreign_keys = ON},
        }
    );

Then, when you run your test case, you can use commands such as:

    $ cp myapp.db myappTEST.db
    $ CATALYST_DEBUG=0 MYAPP_DSN="dbi:SQLite:myappTEST.db" prove -vwl t/live_app01.t

This will modify the DSN only while the test case is running.  If you
launch your normal application without the C<MYAPP_DSN> environment
variable defined, it will default to the same C<dbi:SQLite:myapp.db> as
before.


=head2 DATABASE CONFIG SWITCHING USING MULTIPLE CONFIG FILES

L<Catalyst::Plugin::ConfigLoader> has functionality to load loading
multiple config files based on environment variablesi, allowing you to
override your default (production) database connection settings during
development (or vice versa).

Setting C<$ENV{ MYAPP_CONFIG_LOCAL_SUFFIX }> to 'testing' in your test
script results in loading of an additional config file named
C<myapp_testing.conf> after C<myapp.conf> which will override any
parameters in C<myapp.conf>.

You should set the environment variable in the BEGIN block of your test
script to make sure it's set before your Catalyst application is
started.

The following is an example for a config and test script for a
DBIx::Class model named MyDB and a controller named Foo:

myapp_testing.conf:

    <Model::MyDB>
        <connect_info>
            dsn dbi:SQLite:myapp.db
        </connect_info>
    </Model::MyDB>


t/controller_Foo.t:

    use strict;
    use warnings;
    use Test::More;
    
    BEGIN {
        $ENV{ MYAPP_CONFIG_LOCAL_SUFFIX } = 'testing';
    }
    
    eval "use Test::WWW::Mechanize::Catalyst 'MyApp'";
    plan $@
        ? ( skip_all => 'Test::WWW::Mechanize::Catalyst required' )
        : ( tests => 2 );
    
    ok( my $mech = Test::WWW::Mechanize::Catalyst->new, 'Created mech object' );
    
    $mech->get_ok( 'http://localhost/foo' );


You can jump to the next chapter of the tutorial here:
L<Advanced CRUD|Catalyst::Manual::Tutorial::09_AdvancedCRUD>


=head1 AUTHOR

Kennedy Clark, C<hkclark@gmail.com>

Feel free to contact the author for any errors or suggestions, but the
best way to report issues is via the CPAN RT Bug system at
L<https://rt.cpan.org/Public/Dist/Display.html?Name=Catalyst-Manual>.

Copyright 2006-2011, Kennedy Clark, under the
Creative Commons Attribution Share-Alike License Version 3.0
(L<http://creativecommons.org/licenses/by-sa/3.0/us/>).