1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188
|
// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build unix || aix || darwin || dragonfly || freebsd || linux || netbsd || openbsd || solaris || windows
// +build unix aix darwin dragonfly freebsd linux netbsd openbsd solaris windows
// The stress utility is intended for catching sporadic failures.
// It runs a given process in parallel in a loop and collects any failures.
// Usage:
//
// $ stress ./fmt.test -test.run=TestSometing -test.cpu=10
//
// You can also specify a number of parallel processes with -p flag;
// instruct the utility to not kill hanged processes for gdb attach;
// or specify the failure output you are looking for (if you want to
// ignore some other sporadic failures).
package main
import (
"bytes"
"flag"
"fmt"
"os"
"os/exec"
"path/filepath"
"regexp"
"runtime"
"sync/atomic"
"syscall"
"time"
)
var (
flagCount = flag.Int("count", 0, "stop after `N` runs (default never stop)")
flagFailure = flag.String("failure", "", "fail only if output matches `regexp`")
flagIgnore = flag.String("ignore", "", "ignore failure if output matches `regexp`")
flagKill = flag.Bool("kill", true, "kill timed out processes if true, otherwise just print pid (to attach with gdb)")
flagOutput = flag.String("o", defaultPrefix(), "output failure logs to `path` plus a unique suffix")
flagP = flag.Int("p", runtime.NumCPU(), "run `N` processes in parallel")
flagTimeout = flag.Duration("timeout", 10*time.Minute, "timeout each process after `duration`")
)
func init() {
flag.Usage = func() {
os.Stderr.WriteString(`The stress utility is intended for catching sporadic failures.
It runs a given process in parallel in a loop and collects any failures.
Usage:
$ stress ./fmt.test -test.run=TestSometing -test.cpu=10
`)
flag.PrintDefaults()
}
}
func defaultPrefix() string {
date := time.Now().Format("go-stress-20060102T150405-")
return filepath.Join(os.TempDir(), date)
}
func main() {
flag.Parse()
if *flagP <= 0 || *flagTimeout <= 0 || len(flag.Args()) == 0 {
flag.Usage()
os.Exit(1)
}
var failureRe, ignoreRe *regexp.Regexp
if *flagFailure != "" {
var err error
if failureRe, err = regexp.Compile(*flagFailure); err != nil {
fmt.Println("bad failure regexp:", err)
os.Exit(1)
}
}
if *flagIgnore != "" {
var err error
if ignoreRe, err = regexp.Compile(*flagIgnore); err != nil {
fmt.Println("bad ignore regexp:", err)
os.Exit(1)
}
}
res := make(chan []byte)
var started atomic.Int64
for i := 0; i < *flagP; i++ {
go func() {
for {
// Note: Must started.Add(1) even if not using -count,
// because it enables the '%d active' print below.
if started.Add(1) > int64(*flagCount) && *flagCount > 0 {
break
}
cmd := exec.Command(flag.Args()[0], flag.Args()[1:]...)
var buf bytes.Buffer
cmd.Stdout = &buf
cmd.Stderr = &buf
err := cmd.Start() // make cmd.Process valid for timeout goroutine
done := make(chan bool)
if err == nil && *flagTimeout > 0 {
go func() {
select {
case <-done:
return
case <-time.After(*flagTimeout):
}
if !*flagKill {
fmt.Printf("process %v timed out\n", cmd.Process.Pid)
return
}
cmd.Process.Signal(syscall.SIGABRT)
select {
case <-done:
return
case <-time.After(10 * time.Second):
}
cmd.Process.Kill()
}()
}
if err == nil {
err = cmd.Wait()
}
out := buf.Bytes()
close(done)
if err != nil && (failureRe == nil || failureRe.Match(out)) && (ignoreRe == nil || !ignoreRe.Match(out)) {
out = append(out, fmt.Sprintf("\n\nERROR: %v\n", err)...)
} else {
out = []byte{}
}
res <- out
}
}()
}
runs, fails := 0, 0
start := time.Now()
ticker := time.NewTicker(5 * time.Second).C
status := func(context string) {
elapsed := time.Since(start).Truncate(time.Second)
var pct string
if fails > 0 {
pct = fmt.Sprintf(" (%0.2f%%)", 100.0*float64(fails)/float64(runs))
}
var active string
n := started.Load() - int64(runs)
if *flagCount > 0 {
// started counts past *flagCount at end; do not count those
// TODO: n = min(n, int64(*flagCount-runs))
if x := int64(*flagCount - runs); n > x {
n = x
}
}
if n > 0 {
active = fmt.Sprintf(", %d active", n)
}
fmt.Printf("%v: %v runs %s, %v failures%s%s\n", elapsed, runs, context, fails, pct, active)
}
for {
select {
case out := <-res:
runs++
if len(out) > 0 {
fails++
dir, path := filepath.Split(*flagOutput)
f, err := os.CreateTemp(dir, path)
if err != nil {
fmt.Printf("failed to create temp file: %v\n", err)
os.Exit(1)
}
f.Write(out)
f.Close()
if len(out) > 2<<10 {
out := out[:2<<10]
fmt.Printf("\n%s\n%s\n…\n", f.Name(), out)
} else {
fmt.Printf("\n%s\n%s\n", f.Name(), out)
}
}
if *flagCount > 0 && runs >= *flagCount {
status("total")
if fails > 0 {
os.Exit(1)
}
os.Exit(0)
}
case <-ticker:
status("so far")
}
}
}
|