1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
|
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Garbage Collection Safepoints in LLVM — LLVM 13 documentation</title>
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/llvm-theme.css" type="text/css" />
<script id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script src="_static/jquery.js"></script>
<script src="_static/underscore.js"></script>
<script src="_static/doctools.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="System Library" href="SystemLibrary.html" />
<link rel="prev" title="Speculative Load Hardening" href="SpeculativeLoadHardening.html" />
<style type="text/css">
table.right { float: right; margin-left: 20px; }
table.right td { border: 1px solid #ccc; }
</style>
</head><body>
<div class="logo">
<a href="index.html">
<img src="_static/logo.png"
alt="LLVM Logo" width="250" height="88"/></a>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="SystemLibrary.html" title="System Library"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="SpeculativeLoadHardening.html" title="Speculative Load Hardening"
accesskey="P">previous</a> |</li>
<li><a href="https://llvm.org/">LLVM Home</a> | </li>
<li><a href="index.html">Documentation</a>»</li>
<li class="nav-item nav-item-1"><a href="Reference.html" accesskey="U">Reference</a> »</li>
<li class="nav-item nav-item-this"><a href="">Garbage Collection Safepoints in LLVM</a></li>
</ul>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3>Documentation</h3>
<ul class="want-points">
<li><a href="https://llvm.org/docs/GettingStartedTutorials.html">Getting Started/Tutorials</a></li>
<li><a href="https://llvm.org/docs/UserGuides.html">User Guides</a></li>
<li><a href="https://llvm.org/docs/Reference.html">Reference</a></li>
</ul>
<h3>Getting Involved</h3>
<ul class="want-points">
<li><a href="https://llvm.org/docs/Contributing.html">Contributing to LLVM</a></li>
<li><a href="https://llvm.org/docs/HowToSubmitABug.html">Submitting Bug Reports</a></li>
<li><a href="https://llvm.org/docs/GettingInvolved.html#mailing-lists">Mailing Lists</a></li>
<li><a href="https://llvm.org/docs/GettingInvolved.html#irc">IRC</a></li>
<li><a href="https://llvm.org/docs/GettingInvolved.html#meetups-and-social-events">Meetups and Social Events</a></li>
</ul>
<h3>Additional Links</h3>
<ul class="want-points">
<li><a href="https://llvm.org/docs/FAQ.html">FAQ</a></li>
<li><a href="https://llvm.org/docs/Lexicon.html">Glossary</a></li>
<li><a href="https://llvm.org/pubs">Publications</a></li>
<li><a href="https://github.com/llvm/llvm-project//">Github Repository</a></li>
</ul>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="_sources/Statepoints.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" />
<input type="submit" value="Go" />
</form>
</div>
</div>
<script>$('#searchbox').show(0);</script>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="garbage-collection-safepoints-in-llvm">
<h1>Garbage Collection Safepoints in LLVM<a class="headerlink" href="#garbage-collection-safepoints-in-llvm" title="Permalink to this headline">¶</a></h1>
<div class="contents local topic" id="contents">
<ul class="simple">
<li><p><a class="reference internal" href="#status" id="id4">Status</a></p></li>
<li><p><a class="reference internal" href="#overview-core-concepts" id="id5">Overview & Core Concepts</a></p>
<ul>
<li><p><a class="reference internal" href="#abstract-machine-model" id="id6">Abstract Machine Model</a></p></li>
<li><p><a class="reference internal" href="#explicit-representation" id="id7">Explicit Representation</a></p></li>
<li><p><a class="reference internal" href="#simplifications-for-non-relocating-gcs" id="id8">Simplifications for Non-Relocating GCs</a></p></li>
<li><p><a class="reference internal" href="#recording-on-stack-regions" id="id9">Recording On Stack Regions</a></p></li>
<li><p><a class="reference internal" href="#base-derived-pointers" id="id10">Base & Derived Pointers</a></p></li>
<li><p><a class="reference internal" href="#gc-transitions" id="id11">GC Transitions</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#stack-map-format" id="id12">Stack Map Format</a></p></li>
<li><p><a class="reference internal" href="#safepoint-semantics-verification" id="id13">Safepoint Semantics & Verification</a></p></li>
<li><p><a class="reference internal" href="#utility-passes-for-safepoint-insertion" id="id14">Utility Passes for Safepoint Insertion</a></p>
<ul>
<li><p><a class="reference internal" href="#rewritestatepointsforgc" id="id15">RewriteStatepointsForGC</a></p></li>
<li><p><a class="reference internal" href="#rewritestatepointsforgc-intrinsic-lowering" id="id16">RewriteStatepointsForGC intrinsic lowering</a></p></li>
<li><p><a class="reference internal" href="#placesafepoints" id="id17">PlaceSafepoints</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#supported-architectures" id="id18">Supported Architectures</a></p></li>
<li><p><a class="reference internal" href="#limitations-and-half-baked-ideas" id="id19">Limitations and Half Baked Ideas</a></p>
<ul>
<li><p><a class="reference internal" href="#mixing-references-and-raw-pointers" id="id20">Mixing References and Raw Pointers</a></p></li>
<li><p><a class="reference internal" href="#objects-on-the-stack" id="id21">Objects on the Stack</a></p></li>
<li><p><a class="reference internal" href="#lowering-quality-and-representation-overhead" id="id22">Lowering Quality and Representation Overhead</a></p></li>
<li><p><a class="reference internal" href="#relocations-along-exceptional-edges" id="id23">Relocations Along Exceptional Edges</a></p></li>
<li><p><a class="reference internal" href="#support-for-alternate-stackmap-formats" id="id24">Support for alternate stackmap formats</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#bugs-and-enhancements" id="id25">Bugs and Enhancements</a></p></li>
</ul>
</div>
<div class="section" id="status">
<h2><a class="toc-backref" href="#id4">Status</a><a class="headerlink" href="#status" title="Permalink to this headline">¶</a></h2>
<p>This document describes a set of extensions to LLVM to support garbage
collection. By now, these mechanisms are well proven with commercial java
implementation with a fully relocating collector having shipped using them.
There are a couple places where bugs might still linger; these are called out
below.</p>
<p>They are still listed as “experimental” to indicate that no forward or backward
compatibility guarantees are offered across versions. If your use case is such
that you need some form of forward compatibility guarantee, please raise the
issue on the llvm-dev mailing list.</p>
<p>LLVM still supports an alternate mechanism for conservative garbage collection
support using the <code class="docutils literal notranslate"><span class="pre">gcroot</span></code> intrinsic. The <code class="docutils literal notranslate"><span class="pre">gcroot</span></code> mechanism is mostly of
historical interest at this point with one exception - its implementation of
shadow stacks has been used successfully by a number of language frontends and
is still supported.</p>
</div>
<div class="section" id="overview-core-concepts">
<h2><a class="toc-backref" href="#id5">Overview & Core Concepts</a><a class="headerlink" href="#overview-core-concepts" title="Permalink to this headline">¶</a></h2>
<p>To collect dead objects, garbage collectors must be able to identify
any references to objects contained within executing code, and,
depending on the collector, potentially update them. The collector
does not need this information at all points in code - that would make
the problem much harder - but only at well-defined points in the
execution known as ‘safepoints’ For most collectors, it is sufficient
to track at least one copy of each unique pointer value. However, for
a collector which wishes to relocate objects directly reachable from
running code, a higher standard is required.</p>
<p>One additional challenge is that the compiler may compute intermediate
results (“derived pointers”) which point outside of the allocation or
even into the middle of another allocation. The eventual use of this
intermediate value must yield an address within the bounds of the
allocation, but such “exterior derived pointers” may be visible to the
collector. Given this, a garbage collector can not safely rely on the
runtime value of an address to indicate the object it is associated
with. If the garbage collector wishes to move any object, the
compiler must provide a mapping, for each pointer, to an indication of
its allocation.</p>
<p>To simplify the interaction between a collector and the compiled code,
most garbage collectors are organized in terms of three abstractions:
load barriers, store barriers, and safepoints.</p>
<ol class="arabic simple">
<li><p>A load barrier is a bit of code executed immediately after the
machine load instruction, but before any use of the value loaded.
Depending on the collector, such a barrier may be needed for all
loads, merely loads of a particular type (in the original source
language), or none at all.</p></li>
<li><p>Analogously, a store barrier is a code fragment that runs
immediately before the machine store instruction, but after the
computation of the value stored. The most common use of a store
barrier is to update a ‘card table’ in a generational garbage
collector.</p></li>
<li><p>A safepoint is a location at which pointers visible to the compiled
code (i.e. currently in registers or on the stack) are allowed to
change. After the safepoint completes, the actual pointer value
may differ, but the ‘object’ (as seen by the source language)
pointed to will not.</p></li>
</ol>
<blockquote>
<div><p>Note that the term ‘safepoint’ is somewhat overloaded. It refers to
both the location at which the machine state is parsable and the
coordination protocol involved in bring application threads to a
point at which the collector can safely use that information. The
term “statepoint” as used in this document refers exclusively to the
former.</p>
</div></blockquote>
<p>This document focuses on the last item - compiler support for
safepoints in generated code. We will assume that an outside
mechanism has decided where to place safepoints. From our
perspective, all safepoints will be function calls. To support
relocation of objects directly reachable from values in compiled code,
the collector must be able to:</p>
<ol class="arabic simple">
<li><p>identify every copy of a pointer (including copies introduced by
the compiler itself) at the safepoint,</p></li>
<li><p>identify which object each pointer relates to, and</p></li>
<li><p>potentially update each of those copies.</p></li>
</ol>
<p>This document describes the mechanism by which an LLVM based compiler
can provide this information to a language runtime/collector, and
ensure that all pointers can be read and updated if desired.</p>
<div class="section" id="abstract-machine-model">
<h3><a class="toc-backref" href="#id6">Abstract Machine Model</a><a class="headerlink" href="#abstract-machine-model" title="Permalink to this headline">¶</a></h3>
<p>At a high level, LLVM has been extended to support compiling to an abstract
machine which extends the actual target with a non-integral pointer type
suitable for representing a garbage collected reference to an object. In
particular, such non-integral pointer type have no defined mapping to an
integer representation. This semantic quirk allows the runtime to pick a
integer mapping for each point in the program allowing relocations of objects
without visible effects.</p>
<p>This high level abstract machine model is used for most of the optimizer. As
a result, transform passes do not need to be extended to look through explicit
relocation sequence. Before starting code generation, we switch
representations to an explicit form. The exact location chosen for lowering
is an implementation detail.</p>
<p>Note that most of the value of the abstract machine model comes for collectors
which need to model potentially relocatable objects. For a compiler which
supports only a non-relocating collector, you may wish to consider starting
with the fully explicit form.</p>
<p>Warning: There is one currently known semantic hole in the definition of
non-integral pointers which has not been addressed upstream. To work around
this, you need to disable speculation of loads unless the memory type
(non-integral pointer vs anything else) is known to unchanged. That is, it is
not safe to speculate a load if doing causes a non-integral pointer value to
be loaded as any other type or vice versa. In practice, this restriction is
well isolated to isSafeToSpeculate in ValueTracking.cpp.</p>
</div>
<div class="section" id="explicit-representation">
<h3><a class="toc-backref" href="#id7">Explicit Representation</a><a class="headerlink" href="#explicit-representation" title="Permalink to this headline">¶</a></h3>
<p>A frontend could directly generate this low level explicit form, but
doing so may inhibit optimization. Instead, it is recommended that
compilers with relocating collectors target the abstract machine model just
described.</p>
<p>The heart of the explicit approach is to construct (or rewrite) the IR in a
manner where the possible updates performed by the garbage collector are
explicitly visible in the IR. Doing so requires that we:</p>
<ol class="arabic simple">
<li><p>create a new SSA value for each potentially relocated pointer, and
ensure that no uses of the original (non relocated) value is
reachable after the safepoint,</p></li>
<li><p>specify the relocation in a way which is opaque to the compiler to
ensure that the optimizer can not introduce new uses of an
unrelocated value after a statepoint. This prevents the optimizer
from performing unsound optimizations.</p></li>
<li><p>recording a mapping of live pointers (and the allocation they’re
associated with) for each statepoint.</p></li>
</ol>
<p>At the most abstract level, inserting a safepoint can be thought of as
replacing a call instruction with a call to a multiple return value
function which both calls the original target of the call, returns
its result, and returns updated values for any live pointers to
garbage collected objects.</p>
<blockquote>
<div><p>Note that the task of identifying all live pointers to garbage
collected values, transforming the IR to expose a pointer giving the
base object for every such live pointer, and inserting all the
intrinsics correctly is explicitly out of scope for this document.
The recommended approach is to use the <a class="reference internal" href="#statepoint-utilities"><span class="std std-ref">utility passes</span></a> described below.</p>
</div></blockquote>
<p>This abstract function call is concretely represented by a sequence of
intrinsic calls known collectively as a “statepoint relocation sequence”.</p>
<p>Let’s consider a simple call in LLVM IR:</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="k">define</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@test1</span><span class="p">(</span><span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">)</span>
<span class="k">gc</span> <span class="s">"statepoint-example"</span> <span class="p">{</span>
<span class="k">call</span> <span class="k">void</span> <span class="p">()*</span> <span class="vg">@foo</span><span class="p">()</span>
<span class="k">ret</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Depending on our language we may need to allow a safepoint during the execution
of <code class="docutils literal notranslate"><span class="pre">foo</span></code>. If so, we need to let the collector update local values in the
current frame. If we don’t, we’ll be accessing a potential invalid reference
once we eventually return from the call.</p>
<p>In this example, we need to relocate the SSA value <code class="docutils literal notranslate"><span class="pre">%obj</span></code>. Since we can’t
actually change the value in the SSA value <code class="docutils literal notranslate"><span class="pre">%obj</span></code>, we need to introduce a new
SSA value <code class="docutils literal notranslate"><span class="pre">%obj.relocated</span></code> which represents the potentially changed value of
<code class="docutils literal notranslate"><span class="pre">%obj</span></code> after the safepoint and update any following uses appropriately. The
resulting relocation sequence is:</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="k">define</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@test1</span><span class="p">(</span><span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">)</span>
<span class="k">gc</span> <span class="s">"statepoint-example"</span> <span class="p">{</span>
<span class="nv nv-Anonymous">%0</span> <span class="p">=</span> <span class="k">call</span> <span class="k">token</span> <span class="p">(</span><span class="k">i64</span><span class="p">,</span> <span class="k">i32</span><span class="p">,</span> <span class="k">void</span> <span class="p">()*,</span> <span class="k">i32</span><span class="p">,</span> <span class="k">i32</span><span class="p">,</span> <span class="p">...)*</span> <span class="vg">@llvm.experimental.gc.statepoint.p0f_isVoidf</span><span class="p">(</span><span class="k">i64</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">void</span> <span class="p">()*</span> <span class="vg">@foo</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">)</span>
<span class="nv">%obj.relocated</span> <span class="p">=</span> <span class="k">call</span> <span class="k">coldcc</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@llvm.experimental.gc.relocate.p1i8</span><span class="p">(</span><span class="k">token</span> <span class="nv nv-Anonymous">%0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">7</span><span class="p">,</span> <span class="k">i32</span> <span class="m">7</span><span class="p">)</span>
<span class="k">ret</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj.relocated</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Ideally, this sequence would have been represented as a M argument, N
return value function (where M is the number of values being
relocated + the original call arguments and N is the original return
value + each relocated value), but LLVM does not easily support such a
representation.</p>
<p>Instead, the statepoint intrinsic marks the actual site of the
safepoint or statepoint. The statepoint returns a token value (which
exists only at compile time). To get back the original return value
of the call, we use the <code class="docutils literal notranslate"><span class="pre">gc.result</span></code> intrinsic. To get the relocation
of each pointer in turn, we use the <code class="docutils literal notranslate"><span class="pre">gc.relocate</span></code> intrinsic with the
appropriate index. Note that both the <code class="docutils literal notranslate"><span class="pre">gc.relocate</span></code> and <code class="docutils literal notranslate"><span class="pre">gc.result</span></code> are
tied to the statepoint. The combination forms a “statepoint relocation
sequence” and represents the entirety of a parseable call or ‘statepoint’.</p>
<p>When lowered, this example would generate the following x86 assembly:</p>
<div class="highlight-gas notranslate"><div class="highlight"><pre><span></span> <span class="na">.globl</span> <span class="no">test1</span>
<span class="na">.align</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">0x90</span>
<span class="nf">pushq</span> <span class="nv">%rax</span>
<span class="nf">callq</span> <span class="no">foo</span>
<span class="nl">.Ltmp1:</span>
<span class="nf">movq</span> <span class="p">(</span><span class="nv">%rsp</span><span class="p">),</span> <span class="nv">%rax</span> <span class="c1"># This load is redundant (oops!)</span>
<span class="nf">popq</span> <span class="nv">%rdx</span>
<span class="nf">retq</span>
</pre></div>
</div>
<p>Each of the potentially relocated values has been spilled to the
stack, and a record of that location has been recorded to the
<a class="reference internal" href="StackMaps.html#stackmap-section"><span class="std std-ref">Stack Map section</span></a>. If the garbage collector
needs to update any of these pointers during the call, it knows
exactly what to change.</p>
<p>The relevant parts of the StackMap section for our example are:</p>
<div class="highlight-gas notranslate"><div class="highlight"><pre><span></span><span class="c1"># This describes the call site</span>
<span class="c1"># Stack Maps: callsite 2882400000</span>
<span class="na">.quad</span> <span class="mi">2882400000</span>
<span class="na">.long</span> <span class="no">.Ltmp1-test1</span>
<span class="na">.short</span> <span class="mi">0</span>
<span class="c1"># .. 8 entries skipped ..</span>
<span class="c1"># This entry describes the spill slot which is directly addressable</span>
<span class="c1"># off RSP with offset 0. Given the value was spilled with a pushq,</span>
<span class="c1"># that makes sense.</span>
<span class="c1"># Stack Maps: Loc 8: Direct RSP [encoding: .byte 2, .byte 8, .short 7, .int 0]</span>
<span class="na">.byte</span> <span class="mi">2</span>
<span class="na">.byte</span> <span class="mi">8</span>
<span class="na">.short</span> <span class="mi">7</span>
<span class="na">.long</span> <span class="mi">0</span>
</pre></div>
</div>
<p>This example was taken from the tests for the <a class="reference internal" href="#rewritestatepointsforgc"><span class="std std-ref">RewriteStatepointsForGC</span></a>
utility pass. As such, its full StackMap can be easily examined with the
following command.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>opt -rewrite-statepoints-for-gc test/Transforms/RewriteStatepointsForGC/basics.ll -S <span class="p">|</span> llc -debug-only<span class="o">=</span>stackmaps
</pre></div>
</div>
</div>
<div class="section" id="simplifications-for-non-relocating-gcs">
<h3><a class="toc-backref" href="#id8">Simplifications for Non-Relocating GCs</a><a class="headerlink" href="#simplifications-for-non-relocating-gcs" title="Permalink to this headline">¶</a></h3>
<p>Some of the complexity in the previous example is unnecessary for a
non-relocating collector. While a non-relocating collector still needs the
information about which location contain live references, it doesn’t need to
represent explicit relocations. As such, the previously described explicit
lowering can be simplified to remove all of the <code class="docutils literal notranslate"><span class="pre">gc.relocate</span></code> intrinsic
calls and leave uses in terms of the original reference value.</p>
<p>Here’s the explicit lowering for the previous example for a non-relocating
collector:</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="k">define</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@test1</span><span class="p">(</span><span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">)</span>
<span class="k">gc</span> <span class="s">"statepoint-example"</span> <span class="p">{</span>
<span class="k">call</span> <span class="k">token</span> <span class="p">(</span><span class="k">i64</span><span class="p">,</span> <span class="k">i32</span><span class="p">,</span> <span class="k">void</span> <span class="p">()*,</span> <span class="k">i32</span><span class="p">,</span> <span class="k">i32</span><span class="p">,</span> <span class="p">...)*</span> <span class="vg">@llvm.experimental.gc.statepoint.p0f_isVoidf</span><span class="p">(</span><span class="k">i64</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">void</span> <span class="p">()*</span> <span class="vg">@foo</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">)</span>
<span class="k">ret</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span>
<span class="p">}</span>
</pre></div>
</div>
</div>
<div class="section" id="recording-on-stack-regions">
<h3><a class="toc-backref" href="#id9">Recording On Stack Regions</a><a class="headerlink" href="#recording-on-stack-regions" title="Permalink to this headline">¶</a></h3>
<p>In addition to the explicit relocation form previously described, the
statepoint infrastructure also allows the listing of allocas within the gc
pointer list. Allocas can be listed with or without additional explicit gc
pointer values and relocations.</p>
<p>An alloca in the gc region of the statepoint operand list will cause the
address of the stack region to be listed in the stackmap for the statepoint.</p>
<p>This mechanism can be used to describe explicit spill slots if desired. It
then becomes the generator’s responsibility to ensure that values are
spill/filled to/from the alloca as needed on either side of the safepoint.
Note that there is no way to indicate a corresponding base pointer for such
an explicitly specified spill slot, so usage is restricted to values for
which the associated collector can derive the object base from the pointer
itself.</p>
<p>This mechanism can be used to describe on stack objects containing
references provided that the collector can map from the location on the
stack to a heap map describing the internal layout of the references the
collector needs to process.</p>
<p>WARNING: At the moment, this alternate form is not well exercised. It is
recommended to use this with caution and expect to have to fix a few bugs.
In particular, the RewriteStatepointsForGC utility pass does not do
anything for allocas today.</p>
</div>
<div class="section" id="base-derived-pointers">
<h3><a class="toc-backref" href="#id10">Base & Derived Pointers</a><a class="headerlink" href="#base-derived-pointers" title="Permalink to this headline">¶</a></h3>
<p>A “base pointer” is one which points to the starting address of an allocation
(object). A “derived pointer” is one which is offset from a base pointer by
some amount. When relocating objects, a garbage collector needs to be able
to relocate each derived pointer associated with an allocation to the same
offset from the new address.</p>
<p>“Interior derived pointers” remain within the bounds of the allocation
they’re associated with. As a result, the base object can be found at
runtime provided the bounds of allocations are known to the runtime system.</p>
<p>“Exterior derived pointers” are outside the bounds of the associated object;
they may even fall within <em>another</em> allocations address range. As a result,
there is no way for a garbage collector to determine which allocation they
are associated with at runtime and compiler support is needed.</p>
<p>The <code class="docutils literal notranslate"><span class="pre">gc.relocate</span></code> intrinsic supports an explicit operand for describing the
allocation associated with a derived pointer. This operand is frequently
referred to as the base operand, but does not strictly speaking have to be
a base pointer, but it does need to lie within the bounds of the associated
allocation. Some collectors may require that the operand be an actual base
pointer rather than merely an internal derived pointer. Note that during
lowering both the base and derived pointer operands are required to be live
over the associated call safepoint even if the base is otherwise unused
afterwards.</p>
<p>If we extend our previous example to include a pointless derived pointer,
we get:</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="k">define</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@test1</span><span class="p">(</span><span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">)</span>
<span class="k">gc</span> <span class="s">"statepoint-example"</span> <span class="p">{</span>
<span class="nv">%gep</span> <span class="p">=</span> <span class="k">getelementptr</span> <span class="k">i8</span><span class="p">,</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">,</span> <span class="k">i64</span> <span class="m">20000</span>
<span class="nv">%token</span> <span class="p">=</span> <span class="k">call</span> <span class="k">token</span> <span class="p">(</span><span class="k">i64</span><span class="p">,</span> <span class="k">i32</span><span class="p">,</span> <span class="k">void</span> <span class="p">()*,</span> <span class="k">i32</span><span class="p">,</span> <span class="k">i32</span><span class="p">,</span> <span class="p">...)*</span> <span class="vg">@llvm.experimental.gc.statepoint.p0f_isVoidf</span><span class="p">(</span><span class="k">i64</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">void</span> <span class="p">()*</span> <span class="vg">@foo</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">,</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%gep</span><span class="p">)</span>
<span class="nv">%obj.relocated</span> <span class="p">=</span> <span class="k">call</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@llvm.experimental.gc.relocate.p1i8</span><span class="p">(</span><span class="k">token</span> <span class="nv">%token</span><span class="p">,</span> <span class="k">i32</span> <span class="m">7</span><span class="p">,</span> <span class="k">i32</span> <span class="m">7</span><span class="p">)</span>
<span class="nv">%gep.relocated</span> <span class="p">=</span> <span class="k">call</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@llvm.experimental.gc.relocate.p1i8</span><span class="p">(</span><span class="k">token</span> <span class="nv">%token</span><span class="p">,</span> <span class="k">i32</span> <span class="m">7</span><span class="p">,</span> <span class="k">i32</span> <span class="m">8</span><span class="p">)</span>
<span class="nv">%p</span> <span class="p">=</span> <span class="k">getelementptr</span> <span class="k">i8</span><span class="p">,</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%gep</span><span class="p">,</span> <span class="k">i64</span> <span class="m">-20000</span>
<span class="k">ret</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%p</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Note that in this example %p and %obj.relocate are the same address and we
could replace one with the other, potentially removing the derived pointer
from the live set at the safepoint entirely.</p>
</div>
<div class="section" id="gc-transitions">
<span id="gc-transition-args"></span><h3><a class="toc-backref" href="#id11">GC Transitions</a><a class="headerlink" href="#gc-transitions" title="Permalink to this headline">¶</a></h3>
<p>As a practical consideration, many garbage-collected systems allow code that is
collector-aware (“managed code”) to call code that is not collector-aware
(“unmanaged code”). It is common that such calls must also be safepoints, since
it is desirable to allow the collector to run during the execution of
unmanaged code. Furthermore, it is common that coordinating the transition from
managed to unmanaged code requires extra code generation at the call site to
inform the collector of the transition. In order to support these needs, a
statepoint may be marked as a GC transition, and data that is necessary to
perform the transition (if any) may be provided as additional arguments to the
statepoint.</p>
<blockquote>
<div><p>Note that although in many cases statepoints may be inferred to be GC
transitions based on the function symbols involved (e.g. a call from a
function with GC strategy “foo” to a function with GC strategy “bar”),
indirect calls that are also GC transitions must also be supported. This
requirement is the driving force behind the decision to require that GC
transitions are explicitly marked.</p>
</div></blockquote>
<p>Let’s revisit the sample given above, this time treating the call to <code class="docutils literal notranslate"><span class="pre">@foo</span></code>
as a GC transition. Depending on our target, the transition code may need to
access some extra state in order to inform the collector of the transition.
Let’s assume a hypothetical GC–somewhat unimaginatively named “hypothetical-gc”
–that requires that a TLS variable must be written to before and after a call
to unmanaged code. The resulting relocation sequence is:</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="vg">@flag</span> <span class="p">=</span> <span class="k">thread_local</span> <span class="k">global</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">align</span> <span class="m">4</span>
<span class="k">define</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@test1</span><span class="p">(</span><span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)</span> <span class="p">*</span><span class="nv">%obj</span><span class="p">)</span>
<span class="k">gc</span> <span class="s">"hypothetical-gc"</span> <span class="p">{</span>
<span class="nv nv-Anonymous">%0</span> <span class="p">=</span> <span class="k">call</span> <span class="k">token</span> <span class="p">(</span><span class="k">i64</span><span class="p">,</span> <span class="k">i32</span><span class="p">,</span> <span class="k">void</span> <span class="p">()*,</span> <span class="k">i32</span><span class="p">,</span> <span class="k">i32</span><span class="p">,</span> <span class="p">...)*</span> <span class="vg">@llvm.experimental.gc.statepoint.p0f_isVoidf</span><span class="p">(</span><span class="k">i64</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">void</span> <span class="p">()*</span> <span class="vg">@foo</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">1</span><span class="p">,</span> <span class="k">i32</span><span class="p">*</span> <span class="vg">@Flag</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">)</span>
<span class="nv">%obj.relocated</span> <span class="p">=</span> <span class="k">call</span> <span class="k">coldcc</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@llvm.experimental.gc.relocate.p1i8</span><span class="p">(</span><span class="k">token</span> <span class="nv nv-Anonymous">%0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">7</span><span class="p">,</span> <span class="k">i32</span> <span class="m">7</span><span class="p">)</span>
<span class="k">ret</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj.relocated</span>
<span class="p">}</span>
</pre></div>
</div>
<p>During lowering, this will result in an instruction selection DAG that looks
something like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">CALLSEQ_START</span>
<span class="o">...</span>
<span class="n">GC_TRANSITION_START</span> <span class="p">(</span><span class="n">lowered</span> <span class="n">i32</span> <span class="o">*</span><span class="nd">@Flag</span><span class="p">),</span> <span class="n">SRCVALUE</span> <span class="n">i32</span><span class="o">*</span> <span class="n">Flag</span>
<span class="n">STATEPOINT</span>
<span class="n">GC_TRANSITION_END</span> <span class="p">(</span><span class="n">lowered</span> <span class="n">i32</span> <span class="o">*</span><span class="nd">@Flag</span><span class="p">),</span> <span class="n">SRCVALUE</span> <span class="n">i32</span> <span class="o">*</span><span class="n">Flag</span>
<span class="o">...</span>
<span class="n">CALLSEQ_END</span>
</pre></div>
</div>
<p>In order to generate the necessary transition code, the backend for each target
supported by “hypothetical-gc” must be modified to lower <code class="docutils literal notranslate"><span class="pre">GC_TRANSITION_START</span></code>
and <code class="docutils literal notranslate"><span class="pre">GC_TRANSITION_END</span></code> nodes appropriately when the “hypothetical-gc”
strategy is in use for a particular function. Assuming that such lowering has
been added for X86, the generated assembly would be:</p>
<div class="highlight-gas notranslate"><div class="highlight"><pre><span></span> <span class="na">.globl</span> <span class="no">test1</span>
<span class="na">.align</span> <span class="mi">16</span><span class="p">,</span> <span class="mi">0x90</span>
<span class="nf">pushq</span> <span class="nv">%rax</span>
<span class="nf">movl</span> <span class="no">$1</span><span class="p">,</span> <span class="nv">%fs</span><span class="p">:</span><span class="no">Flag@TPOFF</span>
<span class="nf">callq</span> <span class="no">foo</span>
<span class="nf">movl</span> <span class="no">$0</span><span class="p">,</span> <span class="nv">%fs</span><span class="p">:</span><span class="no">Flag@TPOFF</span>
<span class="nl">.Ltmp1:</span>
<span class="nf">movq</span> <span class="p">(</span><span class="nv">%rsp</span><span class="p">),</span> <span class="nv">%rax</span> <span class="c1"># This load is redundant (oops!)</span>
<span class="nf">popq</span> <span class="nv">%rdx</span>
<span class="nf">retq</span>
</pre></div>
</div>
<p>Note that the design as presented above is not fully implemented: in particular,
strategy-specific lowering is not present, and all GC transitions are emitted as
as single no-op before and after the call instruction. These no-ops are often
removed by the backend during dead machine instruction elimination.</p>
<p>Before the abstract machine model is lowered to the explicit statepoint model
of relocations by the <a class="reference internal" href="#rewritestatepointsforgc"><span class="std std-ref">RewriteStatepointsForGC</span></a> pass it is possible for
any derived pointer to get its base pointer and offset from the base pointer
by using the <code class="docutils literal notranslate"><span class="pre">gc.get.pointer.base</span></code> and the <code class="docutils literal notranslate"><span class="pre">gc.get.pointer.offset</span></code>
intrinsics respectively. These intrinsics are inlined by the
<a class="reference internal" href="#rewritestatepointsforgc"><span class="std std-ref">RewriteStatepointsForGC</span></a> pass and must not be used after this pass.</p>
</div>
</div>
<div class="section" id="stack-map-format">
<span id="statepoint-stackmap-format"></span><h2><a class="toc-backref" href="#id12">Stack Map Format</a><a class="headerlink" href="#stack-map-format" title="Permalink to this headline">¶</a></h2>
<p>Locations for each pointer value which may need read and/or updated by
the runtime or collector are provided in a separate section of the
generated object file as specified in the PatchPoint documentation.
This special section is encoded per the
<a class="reference internal" href="StackMaps.html#stackmap-format"><span class="std std-ref">Stack Map format</span></a>.</p>
<p>The general expectation is that a JIT compiler will parse and discard this
format; it is not particularly memory efficient. If you need an alternate
format (e.g. for an ahead of time compiler), see discussion under
:ref: <cite>open work items <OpenWork></cite> below.</p>
<p>Each statepoint generates the following Locations:</p>
<ul class="simple">
<li><p>Constant which describes the calling convention of the call target. This
constant is a valid <a class="reference internal" href="LangRef.html#callingconv"><span class="std std-ref">calling convention identifier</span></a> for
the version of LLVM used to generate the stackmap. No additional compatibility
guarantees are made for this constant over what LLVM provides elsewhere w.r.t.
these identifiers.</p></li>
<li><p>Constant which describes the flags passed to the statepoint intrinsic</p></li>
<li><p>Constant which describes number of following deopt <em>Locations</em> (not
operands). Will be 0 if no “deopt” bundle is provided.</p></li>
<li><p>Variable number of Locations, one for each deopt parameter listed in the
“deopt” operand bundle. At the moment, only deopt parameters with a bitwidth
of 64 bits or less are supported. Values of a type larger than 64 bits can be
specified and reported only if a) the value is constant at the call site, and
b) the constant can be represented with less than 64 bits (assuming zero
extension to the original bitwidth).</p></li>
<li><p>Variable number of relocation records, each of which consists of
exactly two Locations. Relocation records are described in detail
below.</p></li>
</ul>
<p>Each relocation record provides sufficient information for a collector to
relocate one or more derived pointers. Each record consists of a pair of
Locations. The second element in the record represents the pointer (or
pointers) which need updated. The first element in the record provides a
pointer to the base of the object with which the pointer(s) being relocated is
associated. This information is required for handling generalized derived
pointers since a pointer may be outside the bounds of the original allocation,
but still needs to be relocated with the allocation. Additionally:</p>
<ul class="simple">
<li><p>It is guaranteed that the base pointer must also appear explicitly as a
relocation pair if used after the statepoint.</p></li>
<li><p>There may be fewer relocation records then gc parameters in the IR
statepoint. Each <em>unique</em> pair will occur at least once; duplicates
are possible.</p></li>
<li><p>The Locations within each record may either be of pointer size or a
multiple of pointer size. In the later case, the record must be
interpreted as describing a sequence of pointers and their corresponding
base pointers. If the Location is of size N x sizeof(pointer), then
there will be N records of one pointer each contained within the Location.
Both Locations in a pair can be assumed to be of the same size.</p></li>
</ul>
<p>Note that the Locations used in each section may describe the same
physical location. e.g. A stack slot may appear as a deopt location,
a gc base pointer, and a gc derived pointer.</p>
<p>The LiveOut section of the StkMapRecord will be empty for a statepoint
record.</p>
</div>
<div class="section" id="safepoint-semantics-verification">
<h2><a class="toc-backref" href="#id13">Safepoint Semantics & Verification</a><a class="headerlink" href="#safepoint-semantics-verification" title="Permalink to this headline">¶</a></h2>
<p>The fundamental correctness property for the compiled code’s
correctness w.r.t. the garbage collector is a dynamic one. It must be
the case that there is no dynamic trace such that an operation
involving a potentially relocated pointer is observably-after a
safepoint which could relocate it. ‘observably-after’ is this usage
means that an outside observer could observe this sequence of events
in a way which precludes the operation being performed before the
safepoint.</p>
<p>To understand why this ‘observable-after’ property is required,
consider a null comparison performed on the original copy of a
relocated pointer. Assuming that control flow follows the safepoint,
there is no way to observe externally whether the null comparison is
performed before or after the safepoint. (Remember, the original
Value is unmodified by the safepoint.) The compiler is free to make
either scheduling choice.</p>
<p>The actual correctness property implemented is slightly stronger than
this. We require that there be no <em>static path</em> on which a
potentially relocated pointer is ‘observably-after’ it may have been
relocated. This is slightly stronger than is strictly necessary (and
thus may disallow some otherwise valid programs), but greatly
simplifies reasoning about correctness of the compiled code.</p>
<p>By construction, this property will be upheld by the optimizer if
correctly established in the source IR. This is a key invariant of
the design.</p>
<p>The existing IR Verifier pass has been extended to check most of the
local restrictions on the intrinsics mentioned in their respective
documentation. The current implementation in LLVM does not check the
key relocation invariant, but this is ongoing work on developing such
a verifier. Please ask on llvm-dev if you’re interested in
experimenting with the current version.</p>
</div>
<div class="section" id="utility-passes-for-safepoint-insertion">
<span id="statepoint-utilities"></span><h2><a class="toc-backref" href="#id14">Utility Passes for Safepoint Insertion</a><a class="headerlink" href="#utility-passes-for-safepoint-insertion" title="Permalink to this headline">¶</a></h2>
<div class="section" id="rewritestatepointsforgc">
<span id="id1"></span><h3><a class="toc-backref" href="#id15">RewriteStatepointsForGC</a><a class="headerlink" href="#rewritestatepointsforgc" title="Permalink to this headline">¶</a></h3>
<p>The pass RewriteStatepointsForGC transforms a function’s IR to lower from the
abstract machine model described above to the explicit statepoint model of
relocations. To do this, it replaces all calls or invokes of functions which
might contain a safepoint poll with a <code class="docutils literal notranslate"><span class="pre">gc.statepoint</span></code> and associated full
relocation sequence, including all required <code class="docutils literal notranslate"><span class="pre">gc.relocates</span></code>.</p>
<p>Note that by default, this pass only runs for the “statepoint-example” or
“core-clr” gc strategies. You will need to add your custom strategy to this
list or use one of the predefined ones.</p>
<p>As an example, given this code:</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="k">define</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@test1</span><span class="p">(</span><span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">)</span>
<span class="k">gc</span> <span class="s">"statepoint-example"</span> <span class="p">{</span>
<span class="k">call</span> <span class="k">void</span> <span class="vg">@foo</span><span class="p">()</span>
<span class="k">ret</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span>
<span class="p">}</span>
</pre></div>
</div>
<p>The pass would produce this IR:</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="k">define</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@test1</span><span class="p">(</span><span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">)</span>
<span class="k">gc</span> <span class="s">"statepoint-example"</span> <span class="p">{</span>
<span class="nv nv-Anonymous">%0</span> <span class="p">=</span> <span class="k">call</span> <span class="k">token</span> <span class="p">(</span><span class="k">i64</span><span class="p">,</span> <span class="k">i32</span><span class="p">,</span> <span class="k">void</span> <span class="p">()*,</span> <span class="k">i32</span><span class="p">,</span> <span class="k">i32</span><span class="p">,</span> <span class="p">...)*</span> <span class="vg">@llvm.experimental.gc.statepoint.p0f_isVoidf</span><span class="p">(</span><span class="k">i64</span> <span class="m">2882400000</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">void</span> <span class="p">()*</span> <span class="vg">@foo</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">5</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">-1</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">0</span><span class="p">,</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj</span><span class="p">)</span>
<span class="nv">%obj.relocated</span> <span class="p">=</span> <span class="k">call</span> <span class="k">coldcc</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="vg">@llvm.experimental.gc.relocate.p1i8</span><span class="p">(</span><span class="k">token</span> <span class="nv nv-Anonymous">%0</span><span class="p">,</span> <span class="k">i32</span> <span class="m">12</span><span class="p">,</span> <span class="k">i32</span> <span class="m">12</span><span class="p">)</span>
<span class="k">ret</span> <span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%obj.relocated</span>
<span class="p">}</span>
</pre></div>
</div>
<p>In the above examples, the addrspace(1) marker on the pointers is the mechanism
that the <code class="docutils literal notranslate"><span class="pre">statepoint-example</span></code> GC strategy uses to distinguish references from
non references. The pass assumes that all addrspace(1) pointers are non-integral
pointer types. Address space 1 is not globally reserved for this purpose.</p>
<p>This pass can be used an utility function by a language frontend that doesn’t
want to manually reason about liveness, base pointers, or relocation when
constructing IR. As currently implemented, RewriteStatepointsForGC must be
run after SSA construction (i.e. mem2ref).</p>
<p>RewriteStatepointsForGC will ensure that appropriate base pointers are listed
for every relocation created. It will do so by duplicating code as needed to
propagate the base pointer associated with each pointer being relocated to
the appropriate safepoints. The implementation assumes that the following
IR constructs produce base pointers: loads from the heap, addresses of global
variables, function arguments, function return values. Constant pointers (such
as null) are also assumed to be base pointers. In practice, this constraint
can be relaxed to producing interior derived pointers provided the target
collector can find the associated allocation from an arbitrary interior
derived pointer.</p>
<p>By default RewriteStatepointsForGC passes in <code class="docutils literal notranslate"><span class="pre">0xABCDEF00</span></code> as the statepoint
ID and <code class="docutils literal notranslate"><span class="pre">0</span></code> as the number of patchable bytes to the newly constructed
<code class="docutils literal notranslate"><span class="pre">gc.statepoint</span></code>. These values can be configured on a per-callsite
basis using the attributes <code class="docutils literal notranslate"><span class="pre">"statepoint-id"</span></code> and
<code class="docutils literal notranslate"><span class="pre">"statepoint-num-patch-bytes"</span></code>. If a call site is marked with a
<code class="docutils literal notranslate"><span class="pre">"statepoint-id"</span></code> function attribute and its value is a positive
integer (represented as a string), then that value is used as the ID
of the newly constructed <code class="docutils literal notranslate"><span class="pre">gc.statepoint</span></code>. If a call site is marked
with a <code class="docutils literal notranslate"><span class="pre">"statepoint-num-patch-bytes"</span></code> function attribute and its
value is a positive integer, then that value is used as the ‘num patch
bytes’ parameter of the newly constructed <code class="docutils literal notranslate"><span class="pre">gc.statepoint</span></code>. The
<code class="docutils literal notranslate"><span class="pre">"statepoint-id"</span></code> and <code class="docutils literal notranslate"><span class="pre">"statepoint-num-patch-bytes"</span></code> attributes
are not propagated to the <code class="docutils literal notranslate"><span class="pre">gc.statepoint</span></code> call or invoke if they
could be successfully parsed.</p>
<p>In practice, RewriteStatepointsForGC should be run much later in the pass
pipeline, after most optimization is already done. This helps to improve
the quality of the generated code when compiled with garbage collection support.</p>
</div>
<div class="section" id="rewritestatepointsforgc-intrinsic-lowering">
<span id="id2"></span><h3><a class="toc-backref" href="#id16">RewriteStatepointsForGC intrinsic lowering</a><a class="headerlink" href="#rewritestatepointsforgc-intrinsic-lowering" title="Permalink to this headline">¶</a></h3>
<p>As a part of lowering to the explicit model of relocations
RewriteStatepointsForGC performs GC specific lowering for the following
intrinsics:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">gc.get.pointer.base</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">gc.get.pointer.offset</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">llvm.memcpy.element.unordered.atomic.*</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">llvm.memmove.element.unordered.atomic.*</span></code></p></li>
</ul>
<p>There are two possible lowerings for the memcpy and memmove operations:
GC leaf lowering and GC parseable lowering. If a call is explicitly marked with
“gc-leaf-function” attribute the call is lowered to a GC leaf call to
‘<code class="docutils literal notranslate"><span class="pre">__llvm_memcpy_element_unordered_atomic_*</span></code>’ or
‘<code class="docutils literal notranslate"><span class="pre">__llvm_memmove_element_unordered_atomic_*</span></code>’ symbol. Such a call can not
take a safepoint. Otherwise, the call is made GC parseable by wrapping the
call into a statepoint. This makes it possible to take a safepoint during
copy operation. Note that a GC parseable copy operation is not required to
take a safepoint. For example, a short copy operation may be performed without
taking a safepoint.</p>
<p>GC parseable calls to ‘<code class="docutils literal notranslate"><span class="pre">llvm.memcpy.element.unordered.atomic.*</span></code>’,
‘<code class="docutils literal notranslate"><span class="pre">llvm.memmove.element.unordered.atomic.*</span></code>’ intrinsics are lowered to calls
to ‘<code class="docutils literal notranslate"><span class="pre">__llvm_memcpy_element_unordered_atomic_safepoint_*</span></code>’,
‘<code class="docutils literal notranslate"><span class="pre">__llvm_memmove_element_unordered_atomic_safepoint_*</span></code>’ symbols respectively.
This way the runtime can provide implementations of copy operations with and
without safepoints.</p>
<p>GC parseable lowering also involves adjusting the arguments for the call.
Memcpy and memmove intrinsics take derived pointers as source and destination
arguments. If a copy operation takes a safepoint it might need to relocate the
underlying source and destination objects. This requires the corresponding base
pointers to be available in the copy operation. In order to make the base
pointers available RewriteStatepointsForGC replaces derived pointers with base
pointer and offset pairs. For example:</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="k">declare</span> <span class="k">void</span> <span class="vg">@__llvm_memcpy_element_unordered_atomic_safepoint_1</span><span class="p">(</span>
<span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%dest_base</span><span class="p">,</span> <span class="k">i64</span> <span class="nv">%dest_offset</span><span class="p">,</span>
<span class="k">i8</span> <span class="k">addrspace</span><span class="p">(</span><span class="m">1</span><span class="p">)*</span> <span class="nv">%src_base</span><span class="p">,</span> <span class="k">i64</span> <span class="nv">%src_offset</span><span class="p">,</span>
<span class="k">i64</span> <span class="nv">%length</span><span class="p">)</span>
</pre></div>
</div>
</div>
<div class="section" id="placesafepoints">
<span id="id3"></span><h3><a class="toc-backref" href="#id17">PlaceSafepoints</a><a class="headerlink" href="#placesafepoints" title="Permalink to this headline">¶</a></h3>
<p>The pass PlaceSafepoints inserts safepoint polls sufficient to ensure running
code checks for a safepoint request on a timely manner. This pass is expected
to be run before RewriteStatepointsForGC and thus does not produce full
relocation sequences.</p>
<p>As an example, given input IR of the following:</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="k">define</span> <span class="k">void</span> <span class="vg">@test</span><span class="p">()</span> <span class="k">gc</span> <span class="s">"statepoint-example"</span> <span class="p">{</span>
<span class="k">call</span> <span class="k">void</span> <span class="vg">@foo</span><span class="p">()</span>
<span class="k">ret</span> <span class="k">void</span>
<span class="p">}</span>
<span class="k">declare</span> <span class="k">void</span> <span class="vg">@do_safepoint</span><span class="p">()</span>
<span class="k">define</span> <span class="k">void</span> <span class="vg">@gc.safepoint_poll</span><span class="p">()</span> <span class="p">{</span>
<span class="k">call</span> <span class="k">void</span> <span class="vg">@do_safepoint</span><span class="p">()</span>
<span class="k">ret</span> <span class="k">void</span>
<span class="p">}</span>
</pre></div>
</div>
<p>This pass would produce the following IR:</p>
<div class="highlight-llvm notranslate"><div class="highlight"><pre><span></span><span class="k">define</span> <span class="k">void</span> <span class="vg">@test</span><span class="p">()</span> <span class="k">gc</span> <span class="s">"statepoint-example"</span> <span class="p">{</span>
<span class="k">call</span> <span class="k">void</span> <span class="vg">@do_safepoint</span><span class="p">()</span>
<span class="k">call</span> <span class="k">void</span> <span class="vg">@foo</span><span class="p">()</span>
<span class="k">ret</span> <span class="k">void</span>
<span class="p">}</span>
</pre></div>
</div>
<p>In this case, we’ve added an (unconditional) entry safepoint poll. Note that
despite appearances, the entry poll is not necessarily redundant. We’d have to
know that <code class="docutils literal notranslate"><span class="pre">foo</span></code> and <code class="docutils literal notranslate"><span class="pre">test</span></code> were not mutually recursive for the poll to be
redundant. In practice, you’d probably want to your poll definition to contain
a conditional branch of some form.</p>
<p>At the moment, PlaceSafepoints can insert safepoint polls at method entry and
loop backedges locations. Extending this to work with return polls would be
straight forward if desired.</p>
<p>PlaceSafepoints includes a number of optimizations to avoid placing safepoint
polls at particular sites unless needed to ensure timely execution of a poll
under normal conditions. PlaceSafepoints does not attempt to ensure timely
execution of a poll under worst case conditions such as heavy system paging.</p>
<p>The implementation of a safepoint poll action is specified by looking up a
function of the name <code class="docutils literal notranslate"><span class="pre">gc.safepoint_poll</span></code> in the containing Module. The body
of this function is inserted at each poll site desired. While calls or invokes
inside this method are transformed to a <code class="docutils literal notranslate"><span class="pre">gc.statepoints</span></code>, recursive poll
insertion is not performed.</p>
<p>This pass is useful for any language frontend which only has to support
garbage collection semantics at safepoints. If you need other abstract
frame information at safepoints (e.g. for deoptimization or introspection),
you can insert safepoint polls in the frontend. If you have the later case,
please ask on llvm-dev for suggestions. There’s been a good amount of work
done on making such a scheme work well in practice which is not yet documented
here.</p>
</div>
</div>
<div class="section" id="supported-architectures">
<h2><a class="toc-backref" href="#id18">Supported Architectures</a><a class="headerlink" href="#supported-architectures" title="Permalink to this headline">¶</a></h2>
<p>Support for statepoint generation requires some code for each backend.
Today, only X86_64 is supported.</p>
</div>
<div class="section" id="limitations-and-half-baked-ideas">
<span id="openwork"></span><h2><a class="toc-backref" href="#id19">Limitations and Half Baked Ideas</a><a class="headerlink" href="#limitations-and-half-baked-ideas" title="Permalink to this headline">¶</a></h2>
<div class="section" id="mixing-references-and-raw-pointers">
<h3><a class="toc-backref" href="#id20">Mixing References and Raw Pointers</a><a class="headerlink" href="#mixing-references-and-raw-pointers" title="Permalink to this headline">¶</a></h3>
<p>Support for languages which allow unmanaged pointers to garbage collected
objects (i.e. pass a pointer to an object to a C routine) in the abstract
machine model. At the moment, the best idea on how to approach this
involves an intrinsic or opaque function which hides the connection between
the reference value and the raw pointer. The problem is that having a
ptrtoint or inttoptr cast (which is common for such use cases) breaks the
rules used for inferring base pointers for arbitrary references when
lowering out of the abstract model to the explicit physical model. Note
that a frontend which lowers directly to the physical model doesn’t have
any problems here.</p>
</div>
<div class="section" id="objects-on-the-stack">
<h3><a class="toc-backref" href="#id21">Objects on the Stack</a><a class="headerlink" href="#objects-on-the-stack" title="Permalink to this headline">¶</a></h3>
<p>As noted above, the explicit lowering supports objects allocated on the
stack provided the collector can find a heap map given the stack address.</p>
<p>The missing pieces are a) integration with rewriting (RS4GC) from the
abstract machine model and b) support for optionally decomposing on stack
objects so as not to require heap maps for them. The later is required
for ease of integration with some collectors.</p>
</div>
<div class="section" id="lowering-quality-and-representation-overhead">
<h3><a class="toc-backref" href="#id22">Lowering Quality and Representation Overhead</a><a class="headerlink" href="#lowering-quality-and-representation-overhead" title="Permalink to this headline">¶</a></h3>
<p>The current statepoint lowering is known to be somewhat poor. In the very
long term, we’d like to integrate statepoints with the register allocator;
in the near term this is unlikely to happen. We’ve found the quality of
lowering to be relatively unimportant as hot-statepoints are almost always
inliner bugs.</p>
<p>Concerns have been raised that the statepoint representation results in a
large amount of IR being produced for some examples and that this
contributes to higher than expected memory usage and compile times. There’s
no immediate plans to make changes due to this, but alternate models may be
explored in the future.</p>
</div>
<div class="section" id="relocations-along-exceptional-edges">
<h3><a class="toc-backref" href="#id23">Relocations Along Exceptional Edges</a><a class="headerlink" href="#relocations-along-exceptional-edges" title="Permalink to this headline">¶</a></h3>
<p>Relocations along exceptional paths are currently broken in ToT. In
particular, there is current no way to represent a rethrow on a path which
also has relocations. See <a class="reference external" href="https://groups.google.com/forum/#!topic/llvm-dev/AE417XjgxvI">this llvm-dev discussion</a> for more
detail.</p>
</div>
<div class="section" id="support-for-alternate-stackmap-formats">
<h3><a class="toc-backref" href="#id24">Support for alternate stackmap formats</a><a class="headerlink" href="#support-for-alternate-stackmap-formats" title="Permalink to this headline">¶</a></h3>
<p>For some use cases, it is
desirable to directly encode a final memory efficient stackmap format for
use by the runtime. This is particularly relevant for ahead of time
compilers which wish to directly link object files without the need for
post processing of each individual object file. While not implemented
today for statepoints, there is precedent for a GCStrategy to be able to
select a customer GCMetataPrinter for this purpose. Patches to enable
this functionality upstream are welcome.</p>
</div>
</div>
<div class="section" id="bugs-and-enhancements">
<h2><a class="toc-backref" href="#id25">Bugs and Enhancements</a><a class="headerlink" href="#bugs-and-enhancements" title="Permalink to this headline">¶</a></h2>
<p>Currently known bugs and enhancements under consideration can be
tracked by performing a <a class="reference external" href="https://bugs.llvm.org/buglist.cgi?cmdtype=runnamed&namedcmd=Statepoint%20Bugs&list_id=64342">bugzilla search</a>
for [Statepoint] in the summary field. When filing new bugs, please
use this tag so that interested parties see the newly filed bug. As
with most LLVM features, design discussions take place on <a class="reference external" href="http://lists.llvm.org/mailman/listinfo/llvm-dev">llvm-dev</a>, and patches
should be sent to <a class="reference external" href="http://lists.llvm.org/mailman/listinfo/llvm-commits">llvm-commits</a> for review.</p>
</div>
</div>
<div class="clearer"></div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="SystemLibrary.html" title="System Library"
>next</a> |</li>
<li class="right" >
<a href="SpeculativeLoadHardening.html" title="Speculative Load Hardening"
>previous</a> |</li>
<li><a href="https://llvm.org/">LLVM Home</a> | </li>
<li><a href="index.html">Documentation</a>»</li>
<li class="nav-item nav-item-1"><a href="Reference.html" >Reference</a> »</li>
<li class="nav-item nav-item-this"><a href="">Garbage Collection Safepoints in LLVM</a></li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2003-2021, LLVM Project.
Last updated on 2021-09-18.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 3.5.4.
</div>
</body>
</html>
|
