1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
|
# WebSocket compliance testing
We are testing on these Pythons
* CPython 2.7
* CPython 3.6
* PyPy 2.7
* PyPy3 (3.5 lang. level)
using these networking frameworks
* Twisted
* asyncio (or Trollius fallback)
for
* WebSocket client
* WebSocket server
This gives a total of 16 combinations being tested.
The generated reports are hosted here:
* [Client](http://autobahn.ws/testsuite/reports/clients/index.html)
* [Server](http://autobahn.ws/testsuite/reports/servers/index.html)
> Note (23.3.2016): the server reports show some fails for some WebSocket compression tests - these are timeouts of the tests, because my machine was too slow / the timeout is too tight. However, it takes too now for me to repeat.
## Testing
### Prepare
Download Pythons
```console
make downloads
```
Build Pythons and wstest (the command line tool of the Autobahn WebSocket testsuite)
```console
make build
```
Check versions:
```console
oberstet@office-corei7:~/scm/crossbario/autobahn-python/wstest$ make versions
./cpy2/bin/python -V
Python 2.7.13
./cpy3/bin/python -V
Python 3.6.0
./pypy2/bin/python -V
Python 2.7.13 (fa3249d55d15, Mar 19 2017, 20:21:48)
[PyPy 5.7.0 with GCC 6.2.0 20160901]
./pypy3/bin/python -V
Python 3.5.3 (b16a4363e930, Mar 20 2017, 16:13:46)
[PyPy 5.7.0-beta0 with GCC 6.2.0 20160901]
```
Setup dependencies
```console
make setup
```
### Client Testing
In a first console, start the fuzzing server:
```console
make wstest_server
```
In a second console, run the testee clients:
```console
make test_client
```
Reports will be generated under the folder `./reports/clients`, with the summary report `./reports/clients/index.html`.
### Server Testing
In a console, start all the testee servers:
```console
make start_server
```
> Note: this will start 8 servers in the background. To stop: `make stop_server`
Now run the fuzzing client, which will test all servers:
```console
make test_server
```
### Upload reports
To upload reports:
```console
make upload_reports
```
|
