1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215
|
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package bench_test
import (
"flag"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"google.golang.org/protobuf/encoding/protojson"
"google.golang.org/protobuf/encoding/prototext"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/reflect/protoreflect"
"google.golang.org/protobuf/reflect/protoregistry"
benchpb "google.golang.org/protobuf/internal/testprotos/benchmarks"
_ "google.golang.org/protobuf/internal/testprotos/benchmarks/datasets/google_message1/proto2"
_ "google.golang.org/protobuf/internal/testprotos/benchmarks/datasets/google_message1/proto3"
_ "google.golang.org/protobuf/internal/testprotos/benchmarks/datasets/google_message2"
_ "google.golang.org/protobuf/internal/testprotos/benchmarks/datasets/google_message3"
_ "google.golang.org/protobuf/internal/testprotos/benchmarks/datasets/google_message4"
)
func BenchmarkWire(b *testing.B) {
bench(b, "Unmarshal", func(ds dataset, pb *testing.PB) {
for pb.Next() {
for _, p := range ds.wire {
m := ds.messageType.New().Interface()
if err := proto.Unmarshal(p, m); err != nil {
b.Fatal(err)
}
}
}
})
bench(b, "Marshal", func(ds dataset, pb *testing.PB) {
for pb.Next() {
for _, m := range ds.messages {
if _, err := proto.Marshal(m); err != nil {
b.Fatal(err)
}
}
}
})
bench(b, "Size", func(ds dataset, pb *testing.PB) {
for pb.Next() {
for _, m := range ds.messages {
proto.Size(m)
}
}
})
}
func BenchmarkText(b *testing.B) {
bench(b, "Unmarshal", func(ds dataset, pb *testing.PB) {
for pb.Next() {
for _, p := range ds.text {
m := ds.messageType.New().Interface()
if err := prototext.Unmarshal(p, m); err != nil {
b.Fatal(err)
}
}
}
})
bench(b, "Marshal", func(ds dataset, pb *testing.PB) {
for pb.Next() {
for _, m := range ds.messages {
if _, err := prototext.Marshal(m); err != nil {
b.Fatal(err)
}
}
}
})
}
func BenchmarkJSON(b *testing.B) {
bench(b, "Unmarshal", func(ds dataset, pb *testing.PB) {
for pb.Next() {
for _, p := range ds.json {
m := ds.messageType.New().Interface()
if err := protojson.Unmarshal(p, m); err != nil {
b.Fatal(err)
}
}
}
})
bench(b, "Marshal", func(ds dataset, pb *testing.PB) {
for pb.Next() {
for _, m := range ds.messages {
if _, err := protojson.Marshal(m); err != nil {
b.Fatal(err)
}
}
}
})
}
func Benchmark(b *testing.B) {
bench(b, "Clone", func(ds dataset, pb *testing.PB) {
for pb.Next() {
for _, src := range ds.messages {
proto.Clone(src)
}
}
})
}
func bench(b *testing.B, name string, f func(dataset, *testing.PB)) {
b.Helper()
b.Run(name, func(b *testing.B) {
for _, ds := range datasets {
b.Run(ds.name, func(b *testing.B) {
b.RunParallel(func(pb *testing.PB) {
f(ds, pb)
})
})
}
})
}
type dataset struct {
name string
messageType protoreflect.MessageType
messages []proto.Message
wire [][]byte
text [][]byte
json [][]byte
}
var datasets []dataset
func TestMain(m *testing.M) {
// Load benchmark data early, to avoid including this step in -cpuprofile/-memprofile.
//
// For the larger benchmark datasets (not downloaded by default), preparing
// this data is quite expensive. In addition, keeping the unmarshaled messages
// in memory makes GC scans a substantial fraction of runtime CPU cost.
//
// It would be nice to avoid loading the data we aren't going to use. Unfortunately,
// there isn't any simple way to tell what benchmarks are going to run; we can examine
// the -test.bench flag, but parsing it is quite complicated.
flag.Parse()
if v := flag.Lookup("test.bench").Value.(flag.Getter).Get(); v == "" {
// Don't bother loading data if we aren't going to run any benchmarks.
// Avoids slowing down go test ./...
return
}
if v := flag.Lookup("test.timeout").Value.(flag.Getter).Get().(time.Duration); v != 0 && v <= 10*time.Minute {
// The default test timeout of 10m is too short if running all the benchmarks.
// It's quite frustrating to discover this 10m through a benchmark run, so
// catch the condition.
//
// The -timeout and -test.timeout flags are handled by the go command, which
// forwards them along to the test binary, so we can't just set the default
// to something reasonable; the go command will override it with its default.
// We also can't ignore the timeout, because the go command kills a test which
// runs more than a minute past its deadline.
fmt.Fprintf(os.Stderr, "Test timeout of %v is probably too short; set -test.timeout=0.\n", v)
os.Exit(1)
}
out, err := exec.Command("git", "rev-parse", "--show-toplevel").CombinedOutput()
if err != nil {
panic(err)
}
repoRoot := strings.TrimSpace(string(out))
dataDir := filepath.Join(repoRoot, ".cache", "benchdata")
filepath.Walk(dataDir, func(path string, _ os.FileInfo, _ error) error {
if filepath.Ext(path) != ".pb" {
return nil
}
raw, err := os.ReadFile(path)
if err != nil {
panic(err)
}
dspb := &benchpb.BenchmarkDataset{}
if err := proto.Unmarshal(raw, dspb); err != nil {
panic(err)
}
mt, err := protoregistry.GlobalTypes.FindMessageByName(protoreflect.FullName(dspb.MessageName))
if err != nil {
panic(err)
}
ds := dataset{
name: dspb.Name,
messageType: mt,
wire: dspb.Payload,
}
for _, payload := range dspb.Payload {
m := mt.New().Interface()
if err := proto.Unmarshal(payload, m); err != nil {
panic(err)
}
ds.messages = append(ds.messages, m)
b, err := prototext.Marshal(m)
if err != nil {
panic(err)
}
ds.text = append(ds.text, b)
b, err = protojson.Marshal(m)
if err != nil {
panic(err)
}
ds.json = append(ds.json, b)
}
datasets = append(datasets, ds)
return nil
})
os.Exit(m.Run())
}
|