Unit Testing

GSF makes unit testing your specific features easy. By extending the GsfTestBase unit test class (which in turn extends NbTestCase which in turn extends JUnit's TestCase), you inherit a number of additional unit testing methods.

Unfortunately, you have to write a few configuration steps for your language, once - this is described at the end of this document in the Configuration section.

Unit Test Workflow

All GSF unit tests are based on "golden files".

• You have a source file in your language.
• The source file is parsed and then your language feature is run on the given parse result
• The result from your feature is pretty printed
• The pretty-printed output is compared to a previously saved golden file result. If there is a difference, the test fails.
• Crucially, the FIRST time you run the test, there is no golden file, so one is created instead! This makes it trivial to create new tests. Just add more test files, manually check that your feature is producing the correct result for these test files, and then run them as unit tests. You now have golden files which ensure that for these testcases you get the same feature output every time.

Example: Semantic Highlighting

To illustrate how this works, I'll use an example. To create a semantic highlighting unit test, create a new test method like this in your SemanticHighlighterTest:

    
    public void testSemantic1() throws Exception {
        checkSemantic("testfiles/test1.js");
    }
            

    
    Draggable.prototype = {
      scroll: function() {
        var wind, window2, y = 1, win3 = 50;
        var windo
        if(this.scroll == window) {
        }
      }
    }
            

    
    |>CLASS:Draggable<|.prototype = {
      |>METHOD:scroll<|: function() {
        var |>UNUSED:wind<|, |>UNUSED:window2<|, |>UNUSED:y<| = 1, |>UNUSED:win3<| = 50;
        var |>UNUSED:windo<|
        if(this.scroll == |>GLOBAL:window<|) {
        }
      }
    }
            

Repeat this process for all the different scenarios you want to test your semantic highlighter for.

Occasionally, you get a unit test failure. Perhaps you've accidentally broken your code - or perhaps you've actually fixed a bug, which changes the output of your semantic highlighter in some way, so your golden file diff fails and the test files. There is a textual diff printed to the output when you run the test in the IDE, but personally I find it hard to read. Your tests, and your golden files, should all be checked in with your code, and therefore version controlled. So what I do in this scenario is just remove all the golden files (rm *.semantic for example), and run the tests again. I then use the builtin IDE diff tool to diff all the golden files and see what the changes are. Sometimes, the changes are an improvement, and I want the goldenfile updated (which it now has been!) to reflect the new behavior. And in some cases, the change is a bad one, and I have a regression on my hands, so I revert the file change, and start debugging to track down the problem. Here's an example of diffing golden files, where in this case there is definitely a problem - the highlighter is no longer identifying regular expressions in JavaScript (which has to be done at the parse level rather than at the lexical level in JavaScript):

Caret-based Tests

Many GSF features are associated with a caret position. For example, code completion applies at a particular offset, and ditto for mark occurrences.

The convention in GSF is to tell unit tests about a caret position by passing in a substring of the testfile, with an extra ^ (arrow up) indicating the caret position. For example, in the above semantic highlighting code fragment, if I wanted to identify a caret position between the "f" and the "u" of the function keyword on the second line, I could pass in this string:

    
    "scroll: f^unction"
            

Here's an example of a unit test which uses this:

    
    checkOccurrences("testfiles/semantic4.js", "function myfunc(param1, pa^ram2) {", true);
            

    
    Spry.Effect.Registry.prototype.getRegisteredEffect = function(|>MARK_OCCURRENCES:elem^ent<|, effect)
            var eleIdx = this.getIndexOfElement(|>MARK_OCCURRENCES:element<|);
                    var addedElement = new Spry.Effect.AnimatedElement(|>MARK_OCCURRENCES:element<|);
            

Code Completion Tests

Here's an example of a code completion test. Again, we point to some test file, as well as a caret position - in this case, it's at the beginning of the symbol "escape", so there's no prefix:

    
    public void testExpression1() throws Exception {
        checkCompletion("testfiles/completion/lib/expressions.js", "^escape", false);
    }
            

    
Results for escape with queryType=COMPLETION and nameKind=PREFIX
METHOD     exec(str) : void                           RegExp
METHOD     test(str) : Boolean                        RegExp
METHOD     toSource() : void                          RegExp
METHOD     toString() : String                        RegExp
METHOD     watch(prop, handler) : void                RegExp
PROPERTY   constructor                                RegExp
PROPERTY   global : Boolean                           RegExp
PROPERTY   ignoreCase : Boolean                       RegExp
PROPERTY   lastIndex : Number                         RegExp
PROPERTY   multiline                                  RegExp
PROPERTY   prototype                                  RegExp
PROPERTY   source : String                            RegExp
------------------------------------                  
METHOD     eval(string) : void                        Object
METHOD     hasOwnProperty(prop) : Boolean             Object
METHOD     isPrototypeOf(object) : Boolea             Object
METHOD     propertyIsEnumerable(prop) : B             Object
METHOD     toLocaleString() : String                  Object
            

As you can see, writing code completion tests is pretty trivial.

Other Features

There are many other feature tests like this, and all are very similar to the above: a single line call with test data which gets pretty printed and diffed. For example, the insertChar test (and removeChar and insertBreak etc) let you test what happens when you type keys in the editor. You provide the document contents and caret position before, the character to be typed, and the contents and caret position after the insert, like this:

    insertChar("x = ^", '\'', "x = '^'");                
            

    insertChar("x = '^'", '\'', "x = ''^");
    insertChar("// Hello^", '\'', "// Hello'^");
    etc.
            

AST Offset Tests

Unlike the feature tests, the AST offset test support involves a bit more work. That's because GSF does not have any clue about your AST. GSF doesn't even know about it; you typically stash it in your own subclass of the ParserResult class. However, GSF makes it easier to make sure that your AST offsets are correct, which is typically vital for correct IDE operation. To test your own ASTs, you need to override a couple of protected methods in the GsfTestBase:

    
    protected String describeNode(CompilationInfo info, Object node, boolean includePath);
    protected void initializeNodes(CompilationInfo info, List validNodes,
            Map<Object,OffsetRange> positions, List invalidNodes);
            

The first method just lets you create a description of the node (which will show up in the AST golden file). Typically there will be some AST specific way to identify node types - for example, STRING, EXPRESSION, FUNCTION, etc.

In the second method, you need to go and pull ALL nodes out of the AST for your parser result, and put all of them into a map, where each node maps to a given OffsetRange (the start/end offsets) of your nodes that you are trying to test. You can ignore the other offsets for now. Once you do that, you can write tests like this:

    
    public void testOffsets1() throws Exception {
        checkOffsets("testfiles/test1.js");
    }
            

    
<SCRIPT><EXPR_RESULT><SETPROP><NAME>Draggable</NAME>.<STRING>prototype</STRING> = <OBJECTLIT>{
  <OBJLITNAME>scroll</OBJLITNAME>: <FUNCTION>function() <BLOCK>{ 
    <VAR>var <NAME>wind</NAME>, <NAME>window2</NAME>, <NAME>y = <NUMBER>1</NUMBER></NAME>, <NAME>win3 = <NUMBER>50</NUMBER></NAME></VAR>;
    <VAR>var <NAME>windo</NAME></VAR>
    <BLOCK>if(<IFNE><EQ><GETPROP><THIS>this</THIS>.<STRING>scroll</STRING></GETPROP> == <NAME>window</NAME></EQ></IFNE>) <BLOCK>{
    }<TARGET></TARGET></BLOCK></BLOCK>  <RETURN></RETURN></BLOCK>}</FUNCTION>
}</OBJECTLIT></SETPROP></EXPR_RESULT></SCRIPT>   
            

I have made a lot of diffs to the Rhino parser for the JavaScript support to add in AST offset support (Rhino doesn't record source offsets, we've added that for NetBeans). Having tests like these makes it a lot easier to feel confident when I make any changes to the parser. As shown earlier, if there are diffs, I delete the offset files, run the tests, and then diff the golden files through the IDE.

    
            TODO - document this. For now, anyone curious, go look at
            javascript.editing/test/unit/src/org/netbeans/modules/javascript/editing/JsTestBase.java