1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
|
== Description
This is a custom test suite for Ruby. It includes tests for both the core
classes and the standard library. A series of benchmarks are also included.
== Prerequisites
Rake. Version 0.7.2 or later recommended. Earlier versions will delete
the core directory by mistake if you use the 'clean' task, thinking it's
a coredump file.
== Conventions
=== Directory Layout
bench - toplevel directory for all benchmarks
test - toplevel directory for all tests
bench/core - toplevel directory for benchmarks of core Ruby methods.
bench/stdlib - toplevel directory for benchmarks of libraries in the stdlib.
test/core - toplevel directory for tests of core Ruby classes.
test/stdlib - toplevel directory for tests of stdlib.
test/lib - contains helper modules that can be used in your test cases.
Under test/core there is a folder for each of the core classes. Under
those folders are one or two subdirectories - 'class' and/or 'instance'.
Under the 'class' folders are the tests for the class methods of the class
in question. Under the 'instance' folders are the tests for the instance
methods of the class in question, if applicable.
=== Test suites
The test program shall be test/unit, by Nathaniel Talbott. I do not use
RSpec at this time because I find it too verbose, but may at a later date.
All test files shall start with "tc_", and end with the name of the method,
or an analogue based on the internal method name, e.g. "aref" to refer to
Array#[].
All test class names shall start with "TC_", followed by the class name,
followed by the class or instance method (capitalized), followed by the word
"Class", or "Instance", as appropriate.
- Since writing this, I have changed it to 'ClassMethod' and
'InstanceMethod' instead. So, for the moment, you will see both approaches.
For example, TC_Dir_Getwd_ClassMethod < Test::Unit::TestCase
You can optionally use the Test::Helper module to alleviate some of the
mundane tasks, such as getting the user's name, home directory, setting
the base directory, etc.
Running the tests should be handled via tasks in the Rakefile. Some tests
are skipped on certain platforms. Other tests are skipped unless you run
the tests as root. Still others are skipped for alternate implementations,
such as JRuby.
== Testing guidelines for writers
One test file per method for the core classes.
- Exception: bang and non-bang methods may optionally be grouped together,
as well as aliases.
Comment your tests as appropriate.
Test basic functionality using most likely real world uses.
Test for expected errors.
Test edge cases (nil, 0, true, false, empty string).
Validate $SAFE behavior if appropriate.
Validate taint behavior if appropriate.
Go out of your way to break things. :)
Any bugs found in MRI as a result of the test suite should be marked in
the SCORECARD file.
== Coding guidelines for writers
Three space indentation.
Tabs to spaces, always.
Meaningful test names.
Avoid tests that depend on other tests.
Always reset your instance variables to nil in the teardown method.
Use the Test::Helper methods where appropriate.
== Benchmark suites
The benchmark library shall be "benchmark", the library that comes bundled
as part of the Ruby standard library.
All benchmark programs shall start with "bench_", and end with the name of
the class. There shall be one benchmark program per class, although
benchmark suites are also allowed per method, if desired. I have created
a few method benchmarks in order to compare changes to the C source with
original source code.
== Notes on the Benchmark suite
The purpose of the benchmark suite is to determine overall speed, perform
speed comparisons between minor releases, high iteration testing, look for
any pathological slowdowns, and to find methods that can be optimized.
== Running the tests
Use the Rake tasks to run the various tests. You can run the individual
test suites by using the name of the class or package, e.g. 'rake
test_array'.
To perform all core tests run 'rake test_core'. Likewise, to perform all
of the stdlib tests run 'rake test_stdlib'.
To perform all tests run 'rake test'.
See the output of 'rake -T' to see all available tests.
== On JRuby
As of 25-May-2007 I've decided to go ahead and tailor some of the tests
for JRuby. This is easy enough to do by checking the value of the JRUBY
constant (defined in the Test::Helper module). You will have to stay
somewhat apprised of JRuby development to know what tests can and/or
should be skipped.
== On Sapphire
Sapphire is a custom fork of Ruby. You can find more information at
http://www.sapphire-lang.org. I have added an SAPPHIRE constant (set to
true or false) in the Test::Helper module.
== Miscellaneous
The Object and Kernel methods are separated in the same manner as the
Pickaxe (2nd Edition) breaks them out, for clarity.
== TODO
Many of the individual tests have 'TODO' markers in them to indicate that
more and/or better tests need to be added.
(Obviously, any method that hasn't been tested is a 'TODO' item)
Generally speaking, most of the String tests could use additional tests
that include tests that use extended ASCII and/or $KCODE testing.
- I'll add more to this section as I think of it.
== Acknowledgements
Some tests shamelessly plagiarized from Rubicon, Rubinius, BFTS, or the
JRuby test suite.
== License
Ruby's
== Warranty
This package is provided "as is" and without any express or
implied warranties, including, without limitation, the implied
warranties of merchantability and fitness for a particular purpose.
== Author
Daniel J. Berger
djberg96 at gmail dot com
imperator on IRC (irc.freenode.net)
|
