File: tests.md

package info (click to toggle)
chromium 73.0.3683.75-1~deb9u1
  • links: PTS, VCS
  • area: main
  • in suites: stretch
  • size: 1,792,156 kB
  • sloc: cpp: 13,473,466; ansic: 1,577,080; python: 898,539; javascript: 655,737; xml: 341,883; asm: 306,070; java: 289,969; perl: 80,911; objc: 67,198; sh: 43,184; cs: 27,853; makefile: 12,092; php: 11,064; yacc: 10,373; tcl: 8,875; ruby: 3,941; lex: 1,800; pascal: 1,473; lisp: 812; awk: 41; jsp: 39; sed: 19; sql: 3
file content (147 lines) | stat: -rw-r--r-- 4,217 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
# Accessibility

Here's a quick overview of all of the locations in the codebase where
you'll find accessibility tests, and a brief overview of the purpose of
all of them.

## Layout Tests

This is the primary place where we test accessibility code in Blink. This
code should have no platform-specific code. Use this to test anything
where there's any interesting web platform logic, or where you need to be
able to query things synchronously from the renderer thread to test them.

Don't add tests for trivial features like ARIA attributes that we just
expose directly to the next layer up. In those cases the Blink tests are
trivial and it's more valuable to test these features at a higher level
where we can ensure they actually work.

Note that many of these tests are inherited from WebKit and the coding style
has evolved a lot since then. Look for more recent tests as a guide if writing
a new one.

Test files:
```
third_party/blink/web_tests/accessibility
```

Source code to AccessibilityController and WebAXObjectProxy:
```
content/shell/test_runner
```

To run all accessibility web tests:
```
autoninja -C out/release blink_tests
third_party/blink/tools/run_web_tests.py --build-directory=out --target=release accessibility/
```

To run just one test by itself without the script:
```
autoninja -C out/release blink_tests
out/release/content_shell --run-web-tests third_party/blink/web_tests/accessibility/name-calc-inputs.html
```

## DumpAccessibilityTree tests

This is the best way to write both cross-platform and platform-specific tests
using only an input HTML file, some magic strings to describe what attributes
you're interested in, and one or more expectation files to enable checking
whether the resulting accessibility tree is correct or not. In particular,
almost no test code is required.

[More documentation on DumpAccessibilityTree](../../content/test/data/accessibility/readme.md)

Test files:
```
content/test/data/accessibility
```

Test runner:
```
content/browser/accessibility/dump_accessibility_tree_browsertest.cc
```

To run all tests:
```
autoninja -C out/release content_browsertests
out/release/content_browsertests --gtest_filter="DumpAccessibilityTree*"
```

## Other content_browsertests

There are many other tests in content/ that relate to accessibility.
All of these tests work by launching a full multi-process browser shell,
loading a web page in a renderer, then accessing the resulting accessibility
tree from the browser process, and running some test from there.

To run all tests:
```
autoninja -C out/release content_browsertests
out/release/content_browsertests --gtest_filter="*ccessib*"
```

## Accessibility unittests

This tests the core accessibility code that's shared by both web and non-web
accessibility infrastructure.

Code location:
```
ui/accessibility
```

To run all tests:
```
autoninja -C out/release accessibility_unittests
out/release/accessibility_unittests
```

## ChromeVox tests

ChromeVox tests are part of the browser_tests suite. You must build with
```target_os = "chromeos"``` in your GN args.

To run all tests:
```
autoninja -C out/release browser_tests
out/release/browser_tests --test-launcher-jobs=20 --gtest_filter=ChromeVox*
```

### Select-To-Speak tests

```
autoninja -C out/release unit_tests browser_tests
out/release/unit_tests --gtest_filter=*SelectToSpeak*
out/release/browser_tests --gtest_filter=*SelectToSpeak*
```

## Performance tests

We also have a page on [Performance Tests](perf.md).

## Other locations of accessibility tests:

Even this isn't a complete list. The tests described above cover more
than 90% of the accessibility tests, and the remainder are scattered
throughout the codebase. Here are a few other locations to check:

```
chrome/android/javatests/src/org/chromium/chrome/browser/accessibility
chrome/browser/accessibility
chrome/browser/chromeos/accessibility/
ui/chromeos
ui/views/accessibility
```

## Helpful flags:

Across all tests there are some helpful flags that will make testing easier.

```
--test-launcher-jobs=10  # This will run stuff in parallel and make flakes more obvious
```

```
--gtest_filter="*Cats*"  # Filter which tests run. Takes a wildcard (*) optionally.
```