1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
|
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>LLVM Branch Weight Metadata — LLVM 13 documentation</title>
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/llvm-theme.css" type="text/css" />
<script id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="LLVM bugpoint tool: design and usage" href="Bugpoint.html" />
<link rel="prev" title="LLVM Block Frequency Terminology" href="BlockFrequencyTerminology.html" />
<style type="text/css">
table.right { float: right; margin-left: 20px; }
table.right td { border: 1px solid #ccc; }
</style>
</head><body>
<div class="logo">
<a href="index.html">
<img src="_static/logo.png"
alt="LLVM Logo" width="250" height="88"/></a>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="Bugpoint.html" title="LLVM bugpoint tool: design and usage"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="BlockFrequencyTerminology.html" title="LLVM Block Frequency Terminology"
accesskey="P">previous</a> |</li>
<li><a href="https://llvm.org/">LLVM Home</a> | </li>
<li><a href="index.html">Documentation</a>»</li>
<li class="nav-item nav-item-1"><a href="Reference.html" accesskey="U">Reference</a> »</li>
<li class="nav-item nav-item-this"><a href="">LLVM Branch Weight Metadata</a></li>
</ul>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3>Documentation</h3>
<ul class="want-points">
<li><a href="https://llvm.org/docs/GettingStartedTutorials.html">Getting Started/Tutorials</a></li>
<li><a href="https://llvm.org/docs/UserGuides.html">User Guides</a></li>
<li><a href="https://llvm.org/docs/Reference.html">Reference</a></li>
</ul>
<h3>Getting Involved</h3>
<ul class="want-points">
<li><a href="https://llvm.org/docs/Contributing.html">Contributing to LLVM</a></li>
<li><a href="https://llvm.org/docs/HowToSubmitABug.html">Submitting Bug Reports</a></li>
<li><a href="https://llvm.org/docs/GettingInvolved.html#mailing-lists">Mailing Lists</a></li>
<li><a href="https://llvm.org/docs/GettingInvolved.html#irc">IRC</a></li>
<li><a href="https://llvm.org/docs/GettingInvolved.html#meetups-and-social-events">Meetups and Social Events</a></li>
</ul>
<h3>Additional Links</h3>
<ul class="want-points">
<li><a href="https://llvm.org/docs/FAQ.html">FAQ</a></li>
<li><a href="https://llvm.org/docs/Lexicon.html">Glossary</a></li>
<li><a href="https://llvm.org/pubs">Publications</a></li>
<li><a href="https://github.com/llvm/llvm-project//">Github Repository</a></li>
</ul>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/BranchWeightMetadata.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="llvm-branch-weight-metadata">
<h1>LLVM Branch Weight Metadata<a class="headerlink" href="#llvm-branch-weight-metadata" title="Permalink to this headline">¶</a></h1>
<div class="contents local topic" id="contents">
<ul class="simple">
<li><p><a class="reference internal" href="#introduction" id="id3">Introduction</a></p></li>
<li><p><a class="reference internal" href="#supported-instructions" id="id4">Supported Instructions</a></p>
<ul>
<li><p><a class="reference internal" href="#branchinst" id="id5"><code class="docutils literal notranslate"><span class="pre">BranchInst</span></code></a></p></li>
<li><p><a class="reference internal" href="#switchinst" id="id6"><code class="docutils literal notranslate"><span class="pre">SwitchInst</span></code></a></p></li>
<li><p><a class="reference internal" href="#indirectbrinst" id="id7"><code class="docutils literal notranslate"><span class="pre">IndirectBrInst</span></code></a></p></li>
<li><p><a class="reference internal" href="#callinst" id="id8"><code class="docutils literal notranslate"><span class="pre">CallInst</span></code></a></p></li>
<li><p><a class="reference internal" href="#invokeinst" id="id9"><code class="docutils literal notranslate"><span class="pre">InvokeInst</span></code></a></p></li>
<li><p><a class="reference internal" href="#other" id="id10">Other</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#built-in-expect-instructions" id="id11">Built-in <code class="docutils literal notranslate"><span class="pre">expect</span></code> Instructions</a></p>
<ul>
<li><p><a class="reference internal" href="#if-statement" id="id12"><code class="docutils literal notranslate"><span class="pre">if</span></code> statement</a></p></li>
<li><p><a class="reference internal" href="#switch-statement" id="id13"><code class="docutils literal notranslate"><span class="pre">switch</span></code> statement</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#built-in-expect-with-probability-instruction" id="id14">Built-in <code class="docutils literal notranslate"><span class="pre">expect.with.probability</span></code> Instruction</a></p>
<ul>
<li><p><a class="reference internal" href="#id1" id="id15"><code class="docutils literal notranslate"><span class="pre">if</span></code> statement</a></p></li>
<li><p><a class="reference internal" href="#id2" id="id16"><code class="docutils literal notranslate"><span class="pre">switch</span></code> statement</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#cfg-modifications" id="id17">CFG Modifications</a></p></li>
<li><p><a class="reference internal" href="#function-entry-counts" id="id18">Function Entry Counts</a></p></li>
</ul>
</div>
<div class="section" id="introduction">
<h2><a class="toc-backref" href="#id3">Introduction</a><a class="headerlink" href="#introduction" title="Permalink to this headline">¶</a></h2>
<p>Branch Weight Metadata represents branch weights as its likeliness to be taken
(see <a class="reference internal" href="BlockFrequencyTerminology.html"><span class="doc">LLVM Block Frequency Terminology</span></a>). Metadata is assigned to an
<code class="docutils literal notranslate"><span class="pre">Instruction</span></code> that is a terminator as a <code class="docutils literal notranslate"><span class="pre">MDNode</span></code> of the <code class="docutils literal notranslate"><span class="pre">MD_prof</span></code> kind.
The first operator is always a <code class="docutils literal notranslate"><span class="pre">MDString</span></code> node with the string
“branch_weights”. Number of operators depends on the terminator type.</p>
<p>Branch weights might be fetch from the profiling file, or generated based on
<a class="reference internal" href="#builtin-expect">__builtin_expect</a> and <a class="reference internal" href="#builtin-expect-with-probability">__builtin_expect_with_probability</a> instruction.</p>
<p>All weights are represented as an unsigned 32-bit values, where higher value
indicates greater chance to be taken.</p>
</div>
<div class="section" id="supported-instructions">
<h2><a class="toc-backref" href="#id4">Supported Instructions</a><a class="headerlink" href="#supported-instructions" title="Permalink to this headline">¶</a></h2>
<div class="section" id="branchinst">
<h3><a class="toc-backref" href="#id5"><code class="docutils literal notranslate"><span class="pre">BranchInst</span></code></a><a class="headerlink" href="#branchinst" title="Permalink to this headline">¶</a></h3>
<p>Metadata is only assigned to the conditional branches. There are two extra
operands for the true and the false branch.</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>!0 = metadata !{
metadata !"branch_weights",
i32 <TRUE_BRANCH_WEIGHT>,
i32 <FALSE_BRANCH_WEIGHT>
}
</pre></div>
</div>
</div>
<div class="section" id="switchinst">
<h3><a class="toc-backref" href="#id6"><code class="docutils literal notranslate"><span class="pre">SwitchInst</span></code></a><a class="headerlink" href="#switchinst" title="Permalink to this headline">¶</a></h3>
<p>Branch weights are assigned to every case (including the <code class="docutils literal notranslate"><span class="pre">default</span></code> case which
is always case #0).</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>!0 = metadata !{
metadata !"branch_weights",
i32 <DEFAULT_BRANCH_WEIGHT>
[ , i32 <CASE_BRANCH_WEIGHT> ... ]
}
</pre></div>
</div>
</div>
<div class="section" id="indirectbrinst">
<h3><a class="toc-backref" href="#id7"><code class="docutils literal notranslate"><span class="pre">IndirectBrInst</span></code></a><a class="headerlink" href="#indirectbrinst" title="Permalink to this headline">¶</a></h3>
<p>Branch weights are assigned to every destination.</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>!0 = metadata !{
metadata !"branch_weights",
i32 <LABEL_BRANCH_WEIGHT>
[ , i32 <LABEL_BRANCH_WEIGHT> ... ]
}
</pre></div>
</div>
</div>
<div class="section" id="callinst">
<h3><a class="toc-backref" href="#id8"><code class="docutils literal notranslate"><span class="pre">CallInst</span></code></a><a class="headerlink" href="#callinst" title="Permalink to this headline">¶</a></h3>
<p>Calls may have branch weight metadata, containing the execution count of
the call. It is currently used in SamplePGO mode only, to augment the
block and entry counts which may not be accurate with sampling.</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>!0 = metadata !{
metadata !"branch_weights",
i32 <CALL_BRANCH_WEIGHT>
}
</pre></div>
</div>
</div>
<div class="section" id="invokeinst">
<h3><a class="toc-backref" href="#id9"><code class="docutils literal notranslate"><span class="pre">InvokeInst</span></code></a><a class="headerlink" href="#invokeinst" title="Permalink to this headline">¶</a></h3>
<p>Invoke instruction may have branch weight metadata with one or two weights.
The second weight is optional and corresponds to the unwind branch.
If only one weight is set then it contains the execution count of the call
and used in SamplePGO mode only as described for the call instruction. If both
weights are specified then the second weight contains count of unwind branch
taken and the first weights contains the execution count of the call minus
the count of unwind branch taken. Both weights specified are used to calculate
BranchProbability as for BranchInst and for SamplePGO the sum of both weights
is used.</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>!0 = metadata !{
metadata !"branch_weights",
i32 <INVOKE_NORMAL_WEIGHT>
[ , i32 <INVOKE_UNWIND_WEIGHT> ]
}
</pre></div>
</div>
</div>
<div class="section" id="other">
<h3><a class="toc-backref" href="#id10">Other</a><a class="headerlink" href="#other" title="Permalink to this headline">¶</a></h3>
<p>Other terminator instructions are not allowed to contain Branch Weight Metadata.</p>
</div>
</div>
<div class="section" id="built-in-expect-instructions">
<span id="builtin-expect"></span><h2><a class="toc-backref" href="#id11">Built-in <code class="docutils literal notranslate"><span class="pre">expect</span></code> Instructions</a><a class="headerlink" href="#built-in-expect-instructions" title="Permalink to this headline">¶</a></h2>
<p><code class="docutils literal notranslate"><span class="pre">__builtin_expect(long</span> <span class="pre">exp,</span> <span class="pre">long</span> <span class="pre">c)</span></code> instruction provides branch prediction
information. The return value is the value of <code class="docutils literal notranslate"><span class="pre">exp</span></code>.</p>
<p>It is especially useful in conditional statements. Currently Clang supports two
conditional statements:</p>
<div class="section" id="if-statement">
<h3><a class="toc-backref" href="#id12"><code class="docutils literal notranslate"><span class="pre">if</span></code> statement</a><a class="headerlink" href="#if-statement" title="Permalink to this headline">¶</a></h3>
<p>The <code class="docutils literal notranslate"><span class="pre">exp</span></code> parameter is the condition. The <code class="docutils literal notranslate"><span class="pre">c</span></code> parameter is the expected
comparison value. If it is equal to 1 (true), the condition is likely to be
true, in other case condition is likely to be false. For example:</p>
<div class="highlight-c++ notranslate"><div class="highlight"><pre><span></span><span class="k">if</span> <span class="p">(</span><span class="n">__builtin_expect</span><span class="p">(</span><span class="n">x</span> <span class="o">></span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">))</span> <span class="p">{</span>
<span class="c1">// This block is likely to be taken.</span>
<span class="p">}</span>
</pre></div>
</div>
</div>
<div class="section" id="switch-statement">
<h3><a class="toc-backref" href="#id13"><code class="docutils literal notranslate"><span class="pre">switch</span></code> statement</a><a class="headerlink" href="#switch-statement" title="Permalink to this headline">¶</a></h3>
<p>The <code class="docutils literal notranslate"><span class="pre">exp</span></code> parameter is the value. The <code class="docutils literal notranslate"><span class="pre">c</span></code> parameter is the expected
value. If the expected value doesn’t show on the cases list, the <code class="docutils literal notranslate"><span class="pre">default</span></code>
case is assumed to be likely taken.</p>
<div class="highlight-c++ notranslate"><div class="highlight"><pre><span></span><span class="k">switch</span> <span class="p">(</span><span class="n">__builtin_expect</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span> <span class="p">{</span>
<span class="k">default</span><span class="o">:</span> <span class="k">break</span><span class="p">;</span>
<span class="k">case</span> <span class="mi">0</span><span class="o">:</span> <span class="c1">// ...</span>
<span class="k">case</span> <span class="mi">3</span><span class="o">:</span> <span class="c1">// ...</span>
<span class="k">case</span> <span class="mi">5</span><span class="o">:</span> <span class="c1">// This case is likely to be taken.</span>
<span class="p">}</span>
</pre></div>
</div>
</div>
</div>
<div class="section" id="built-in-expect-with-probability-instruction">
<span id="builtin-expect-with-probability"></span><h2><a class="toc-backref" href="#id14">Built-in <code class="docutils literal notranslate"><span class="pre">expect.with.probability</span></code> Instruction</a><a class="headerlink" href="#built-in-expect-with-probability-instruction" title="Permalink to this headline">¶</a></h2>
<p><code class="docutils literal notranslate"><span class="pre">__builtin_expect_with_probability(long</span> <span class="pre">exp,</span> <span class="pre">long</span> <span class="pre">c,</span> <span class="pre">double</span> <span class="pre">probability)</span></code> has
the same semantics as <code class="docutils literal notranslate"><span class="pre">__builtin_expect</span></code>, but the caller provides the
probability that <code class="docutils literal notranslate"><span class="pre">exp</span> <span class="pre">==</span> <span class="pre">c</span></code>. The last argument <code class="docutils literal notranslate"><span class="pre">probability</span></code> must be
constant floating-point expression and be in the range [0.0, 1.0] inclusive.
The usage is also similar as <code class="docutils literal notranslate"><span class="pre">__builtin_expect</span></code>, for example:</p>
<div class="section" id="id1">
<h3><a class="toc-backref" href="#id15"><code class="docutils literal notranslate"><span class="pre">if</span></code> statement</a><a class="headerlink" href="#id1" title="Permalink to this headline">¶</a></h3>
<p>If the expect comparison value <code class="docutils literal notranslate"><span class="pre">c</span></code> is equal to 1(true), and probability
value <code class="docutils literal notranslate"><span class="pre">probability</span></code> is set to 0.8, that means the probability of condition
to be true is 80% while that of false is 20%.</p>
<div class="highlight-c++ notranslate"><div class="highlight"><pre><span></span><span class="k">if</span> <span class="p">(</span><span class="n">__builtin_expect_with_probability</span><span class="p">(</span><span class="n">x</span> <span class="o">></span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mf">0.8</span><span class="p">))</span> <span class="p">{</span>
<span class="c1">// This block is likely to be taken with probability 80%.</span>
<span class="p">}</span>
</pre></div>
</div>
</div>
<div class="section" id="id2">
<h3><a class="toc-backref" href="#id16"><code class="docutils literal notranslate"><span class="pre">switch</span></code> statement</a><a class="headerlink" href="#id2" title="Permalink to this headline">¶</a></h3>
<p>This is basically the same as <code class="docutils literal notranslate"><span class="pre">switch</span></code> statement in <code class="docutils literal notranslate"><span class="pre">__builtin_expect</span></code>.
The probability that <code class="docutils literal notranslate"><span class="pre">exp</span></code> is equal to the expect value is given in
the third argument <code class="docutils literal notranslate"><span class="pre">probability</span></code>, while the probability of other value is
the average of remaining probability(<code class="docutils literal notranslate"><span class="pre">1.0</span> <span class="pre">-</span> <span class="pre">probability</span></code>). For example:</p>
<div class="highlight-c++ notranslate"><div class="highlight"><pre><span></span><span class="k">switch</span> <span class="p">(</span><span class="n">__builtin_expect_with_probability</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mf">0.7</span><span class="p">))</span> <span class="p">{</span>
<span class="k">default</span><span class="o">:</span> <span class="k">break</span><span class="p">;</span> <span class="c1">// Take this case with probability 10%</span>
<span class="k">case</span> <span class="mi">0</span><span class="o">:</span> <span class="k">break</span><span class="p">;</span> <span class="c1">// Take this case with probability 10%</span>
<span class="k">case</span> <span class="mi">3</span><span class="o">:</span> <span class="k">break</span><span class="p">;</span> <span class="c1">// Take this case with probability 10%</span>
<span class="k">case</span> <span class="mi">5</span><span class="o">:</span> <span class="k">break</span><span class="p">;</span> <span class="c1">// This case is likely to be taken with probability 70%</span>
<span class="p">}</span>
</pre></div>
</div>
</div>
</div>
<div class="section" id="cfg-modifications">
<h2><a class="toc-backref" href="#id17">CFG Modifications</a><a class="headerlink" href="#cfg-modifications" title="Permalink to this headline">¶</a></h2>
<p>Branch Weight Metatada is not proof against CFG changes. If terminator operands’
are changed some action should be taken. In other case some misoptimizations may
occur due to incorrect branch prediction information.</p>
</div>
<div class="section" id="function-entry-counts">
<h2><a class="toc-backref" href="#id18">Function Entry Counts</a><a class="headerlink" href="#function-entry-counts" title="Permalink to this headline">¶</a></h2>
<p>To allow comparing different functions during inter-procedural analysis and
optimization, <code class="docutils literal notranslate"><span class="pre">MD_prof</span></code> nodes can also be assigned to a function definition.
The first operand is a string indicating the name of the associated counter.</p>
<p>Currently, one counter is supported: “function_entry_count”. The second operand
is a 64-bit counter that indicates the number of times that this function was
invoked (in the case of instrumentation-based profiles). In the case of
sampling-based profiles, this operand is an approximation of how many times
the function was invoked.</p>
<p>For example, in the code below, the instrumentation for function foo()
indicates that it was called 2,590 times at runtime.</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="k">define</span> <span class="k">i32</span> <span class="vg">@foo</span><span class="p">()</span> <span class="nv">!prof</span> <span class="nv nv-Anonymous">!1</span> <span class="p">{</span>
<span class="k">ret</span> <span class="k">i32</span> <span class="m">0</span>
<span class="p">}</span>
<span class="nv nv-Anonymous">!1</span> <span class="p">=</span> <span class="p">!{</span><span class="nv">!"function_entry_count"</span><span class="p">,</span> <span class="k">i64</span> <span class="m">2590</span><span class="p">}</span>
</pre></div>
</div>
<p>If “function_entry_count” has more than 2 operands, the later operands are
the GUID of the functions that needs to be imported by ThinLTO. This is only
set by sampling based profile. It is needed because the sampling based profile
was collected on a binary that had already imported and inlined these functions,
and we need to ensure the IR matches in the ThinLTO backends for profile
annotation. The reason why we cannot annotate this on the callsite is that it
can only goes down 1 level in the call chain. For the cases where
foo_in_a_cc()->bar_in_b_cc()->baz_in_c_cc(), we will need to go down 2 levels
in the call chain to import both bar_in_b_cc and baz_in_c_cc.</p>
</div>
</div>
<div class="clearer"></div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="Bugpoint.html" title="LLVM bugpoint tool: design and usage"
>next</a> |</li>
<li class="right" >
<a href="BlockFrequencyTerminology.html" title="LLVM Block Frequency Terminology"
>previous</a> |</li>
<li><a href="https://llvm.org/">LLVM Home</a> | </li>
<li><a href="index.html">Documentation</a>»</li>
<li class="nav-item nav-item-1"><a href="Reference.html" >Reference</a> »</li>
<li class="nav-item nav-item-this"><a href="">LLVM Branch Weight Metadata</a></li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2003-2021, LLVM Project.
Last updated on 2021-09-18.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.5.4.
</div>
</body>
</html>
|
