1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
|
# Reproducing libFuzzer and AFL crashes
libFuzzer and AFL crashes can be reproduced easily with the
[ClusterFuzz Reproduce Tool]. However, if you are unable to use the tool (e.g.
unsupported platform, some other tool issue), you can still reproduce the crash
manually using the steps below:
*** note
**Requirements:** For Windows, you must convert the forward slashes (/) to
backslashes (\\) in the commands below and use `set` command instead of `export`
to set the environment variable (step 4). Note that these commands are intended
to be used with cmd.exe, not PowerShell.
***
[TOC]
## Crashes reported as Reproducible
The majority of the bugs reported by ClusterFuzz have **Reproducible** label.
That means there is a testcase that can be used to reliably reproduce the crash.
1. Download the testcase from ClusterFuzz. If you are CCed on an issue filed by
ClusterFuzz, a link to it is next to "Reproducer testcase" in the bug
description.
For the rest of this walkthrough, we call the path of this
file: `$TESTCASE_PATH` and the fuzz target you want to reproduce a
crash on: `$FUZZER_NAME` (provided as "Fuzz target binary" in the bug
description).
2. Generate gn build configuration:
```
gn args out/fuzz
```
This will open up an editor. Copy the gn configuration parameters from the
values provided in `GN Config` section in the ClusterFuzz testcase report.
3. Build the fuzzer:
```
autoninja -C out/fuzz $FUZZER_NAME
```
4. Set the `*SAN_OPTIONS` environment variable as provided in the
`Crash Stacktrace` section in the testcase report.
Here is an example value of `ASAN_OPTIONS` that is similar to its value on
ClusterFuzz:
```
export ASAN_OPTIONS=redzone=256:print_summary=1:handle_sigill=1:allocator_release_to_os_interval_ms=500:print_suppressions=0:strict_memcmp=1:allow_user_segv_handler=0:use_sigaltstack=1:handle_sigfpe=1:handle_sigbus=1:detect_stack_use_after_return=0:alloc_dealloc_mismatch=0:detect_leaks=0:print_scariness=1:allocator_may_return_null=1:handle_abort=1:check_malloc_usable_size=0:detect_container_overflow=0:quarantine_size_mb=256:detect_odr_violation=0:symbolize=1:handle_segv=1:fast_unwind_on_fatal=1
```
5. Run the fuzz target:
```
out/fuzz/$FUZZER_NAME -runs=100 $TESTCASE_PATH
```
If you see an un-symbolized stacktrace, please see the instructions [here].
[File a bug] if you run into any issues.
## Crashes reported as Unreproducible
ClusterFuzz generally does not report issues that it cannot reliably reproduce,
unless the following condition is met. If a certain crash is occurring often
enough, such a crash might be reported with **Unreproducible** label and an
explicit clarification that there is no convenient way to reproduce it. There
are two ways to work with such crashes.
1. Try a speculative fix based on the stacktrace. Once the fix is landed, wait a
couple days and then check Crash Statistics section on the ClusterFuzz
testcase report page. If the fix works out, you will see that the crash is
not happening anymore. If the crash does not occur again for a little while,
ClusterFuzz will automatically close the issue as Verified.
2. (libFuzzer only) Try to reproduce the whole fuzzing session. This workflow is
very similar to the one described above for the **Reproducible** crashes. The
only differences are:
* On step 1, instead of downloading a single testcase, you need to download
corpus backup. This can be done using the following command:
```
gsutil cp gs://clusterfuzz-libfuzzer-backup/corpus/libfuzzer/$FUZZER_NAME/latest.zip .
```
* Alternatively, you can navigate to the following URL in your browser and
download the `latest.zip` file:
```
https://pantheon.corp.google.com/storage/browser/clusterfuzz-libfuzzer-backup/corpus/libfuzzer/$FUZZER_NAME
```
* Create an empty directory and unpack the corpus into it.
* Follow steps 2-4 in the **Reproducible** section above.
* On step 5, use the following command:
```
out/fuzz/$FUZZER_NAME -timeout=25 -rss_limit_mb=2048 -print_final_stats=1 $CORPUS_DIRECTORY_FROM_THE_PREVIOUS_STEP
```
* Wait and hope that the fuzzer will crash.
Waiting for a crash to occur may take some time (up to 1hr), but if it happens,
you will be able to test the fix locally and/or somehow debug the issue.
[ClusterFuzz Reproduce Tool]: https://github.com/google/clusterfuzz-tools
[File a bug]: https://bugs.chromium.org/p/chromium/issues/entry?component=Tools%3EStability%3ElibFuzzer&comment=What%20problem%20are%20you%20seeing
[here]: getting_started.md#symbolize-stacktrace
|
