File: Progress.pod

package info (click to toggle)
libmarpa-r2-perl 12.000000-1
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid
  • size: 6,660 kB
  • sloc: perl: 42,628; ansic: 23,387; sh: 4,363; makefile: 157
file content (1513 lines) | stat: -rw-r--r-- 52,134 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
# Copyright 2022 Jeffrey Kegler
# This file is part of Marpa::R2.  Marpa::R2 is free software: you can
# redistribute it and/or modify it under the terms of the GNU Lesser
# General Public License as published by the Free Software Foundation,
# either version 3 of the License, or (at your option) any later version.
#
# Marpa::R2 is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser
# General Public License along with Marpa::R2.  If not, see
# http://www.gnu.org/licenses/.

=head1 NAME

Marpa::R2::Progress - Progress reports on your parse

=head1 About this document

This document describes
the progress reports for Marpa's SLIF interface.
These allow an application to know
exactly where it is in the parse at any
point.
For parse locations of the user's choosing,
progress reports list all the rules in play,
and indicate the location at which the rule started,
and how far into the rule parsing has progressed.

Progress reports are extremely useful in
debugging grammars and
the detailed example in
this document is a debugging instance.
Readers specifically interested in debugging
a grammar
should read L<the document
on tracing
problems|Marpa::R2::Tracing> before
reading this document.

=head1 Introduction to Earley items

To read the
L<C<show_progress>|Marpa::R2::Scanless::R/"show_progress()">
output, it is important to have a
basic idea of what
Earley items are,
and of what the information in them means.
Everything that the user needs to know
is explained in this section.

=head2 Dotted rules

Marpa is based on Jay Earley's algorithm for parsing.
The idea behind Earley's algorithm is that you can
parse by building a table of rules
and where you are in those rules.
"Where" means two things: location in the rule relative to the rule's
symbols,
and location relative to the parse's input stream.

Let's look at an example of a rule in a context-free grammar.
Here's the rule for assignment from the Perl distribution's C<perly.y>

S<C<E<nbsp>E<nbsp>E<nbsp>E<nbsp>termbinop -E<gt> term ASSIGNOP term>>

C<ASSIGNOP> is C<perly.y>'s internal name for
the assignment operator.
In plain Perl terms, this is the "C<=>" character.

In parsing this rule, we can be at any of four possible locations.
One location is at the beginning, before all of the symbols.
The other three locations are immediately after each of the rule's
three symbols.

Within a rule, position relative to the symbols of the rule
is traditionally indicated with a dot.  In fact, the symbol-relative
rule position is very often called the B<dot location>.  Taken as
a pair, a rule and a dot location are called a B<dotted rule>.

Here's our rule with a dot location indicated:

S<C<E<nbsp>E<nbsp>E<nbsp>E<nbsp>termbinop -E<gt> . term ASSIGNOP term>>

The dot location in this dotted rule is at the beginning.
A dot location at the beginning of a dotted rule means
that we have not recognized any symbols in the rule yet.
All we are doing is predicting that the rule will occur.
A dotted rule with the dot before all of its symbols is called a B<prediction>
or a B<predicted rule>.

Here's another dotted rule:

S<C<E<nbsp>E<nbsp>E<nbsp>E<nbsp>termbinop -E<gt> term . ASSIGNOP term>>

In this dotted rule,
we are saying we have seen a C<term>, but have not yet recognized
an C<ASSIGNOP>.

There's another special kind of dotted rule, a completion.
A B<completion> (also called a B<completed rule>)
is a dotted rule with the dot after all of the symbols.
Here is the completion for the rule that we have been using as an example:

S<C<E<nbsp>E<nbsp>E<nbsp>E<nbsp>termbinop -E<gt> term ASSIGNOP term .>>

A completion indicates that a rule has been fully recognized.

=head2 Earley items

The dotted rules contain
all but one piece of the information
that Marpa
needs to track.
The missing piece is the second of the two "wheres": where in the input stream.
To associate input stream location and dotted rules, Marpa uses what are now called Earley items.

A convenient way to think of an B<Earley item> is as a triple, or 3-tuple,
consisting of dotted rule, origin and current location.
The B<origin> is the location in the input stream where the dotted rule starts.
The B<current location> (also called the B<dot location>)
is the location in the input stream which corresponds to the
dot position.

In Marpa terms,
G1 location is location in terms of the G1 subgrammar's Earley sets.
When the term "location" is used in this document, it means G1 location unless
otherwise indicated.

A user often finds it much more convenient to think in terms of line and column position in
the input stream, instead of G1 location.
Every G1 location corresponds to a range of positions in the input stream.
When the term "position" is used in this document, it means input stream position,
unless otherwise indicated.

Two noteworthy consequences
follow from the way in which origin and current G1 location are defined.
First,
if a dotted rule is a prediction,
then origin and current location will always be the same.
Second,
the input stream location where a rule ends is not tracked unless the
dotted rule is a completion.
If its dotted rule is not a completion,
an Earley item does not tell us
if a rule will
ever be completed,
much less at which location.

=head1 The problem

For this example of debugging,
I have taken a very simple prototype of a string expression calculator
and deliberately introduced a problem.
I've commented out one of the correct rules:

=for Marpa::R2::Display
name: SLIF debug example, part 1
partial: 1
normalize-whitespace: 1

    # <numeric assignment> ::= variable '=' <numeric expression>

=for Marpa::R2::Display::End

and replaced it with a altered one:

=for Marpa::R2::Display
name: SLIF debug example, part 1
partial: 1
normalize-whitespace: 1

    <numeric assignment> ::= variable '=' expression

=for Marpa::R2::Display::End

For those readers who like to look ahead (and I encourage you to
be one of those readers)
all of the code and outputs for this example are collected in
the L</"Appendix">.

This altered rule contains an mistake of the kind that is
easy to make in actual practice.
(In this case,
a unlucky choice of naming conventions may have contributed.)
The altered version will cause problems.
In what follows, we'll pretend we don't already know
where the problem is,
and that in desk-checking the grammar our eye does not
spot the mistake, so
that we need
to use the Marpa diagnostics and tracing facilities
to "discover" it.

=head1 The example

The example we will use is a prototype string calculator.
It's extremely simple, to make the example easy to follow.
But it can be seen as a realistic example,
if it is thought of as a very
early stage in the incremental development of something useful.

=for Marpa::R2::Display
name: SLIF debug example, part 1
partial: 1
normalize-whitespace: 1

    :default ::= action => ::array bless => ::lhs
    :start ::= statements
    statements ::= statement *
    statement ::= assignment | <numeric assignment>
    assignment ::= 'set' variable 'to' expression

    # This is a deliberate error in the grammar
    # The next line should be:
    # <numeric assignment> ::= variable '=' <numeric expression>
    # I have changed the <numeric expression>  to <expression> which
    # will cause problems.
    <numeric assignment> ::= variable '=' expression

    expression ::=
           variable | string
        || 'string' '(' <numeric expression> ')'
        || expression '+' expression
    <numeric expression> ::=
           variable | number
        || <numeric expression> '*' <numeric expression>
        || <numeric expression> '+' <numeric expression>
    variable ~ [\w]+
    number ~ [\d]+
    string ~ ['] <string contents> [']
    <string contents> ~ [^'\x{0A}\x{0B}\x{0C}\x{0D}\x{0085}\x{2028}\x{2029}]+
    :discard ~ whitespace
    whitespace ~ [\s]+

=for Marpa::R2::Display::End

At this stage of developing our string calculator,
we have assignment, variables, constants, concatenation and conversion of
numerics.
For numerics, we have assignment, variables, constants, multiplication and addition.

We decide that, since string expressions and variables are the "default", that in the
grammar we'll make the symbol names for numeric assignment and expressions explicit:
C<E<lt>numeric expressionE<gt>> and C<E<lt>numeric assignmentE<gt>>.
But since strings are the default, we decide to call our string expressions simply C<E<lt>expressionE<gt>>,
and to call our string assignments simply C<E<lt>assignmentE<gt>>.
This seems like a good idea, but it is also likely to cause confusion.
For the sake of our example we will pretend that it did.

=head1 The error message

If we try the following input,

=for Marpa::R2::Display:
name: SLIF debug example, part 2
partial: 1
normalize-whitespace: 1

    my $test_input = 'a = 8675309 + 42 * 711';

=for Marpa::R2::Display::End

we will get this error message,

=for Marpa::R2::Display
name: SLIF debug example error message
partial: 1
normalize-whitespace: 1

    Error in SLIF parse: No lexemes accepted at line 1, column 18
      Rejected lexeme #0: '*'; value="*"; length = 1
    * String before error: a = 8675309 + 42\s
    * The error was at line 1, column 18, and at character 0x002a '*', ...
    * here: * 711

=for Marpa::R2::Display::End

The error message indicates that Marpa rejected the "C<*>" operator.

=head1 The value of the parse

In debugging this issue, we'll look at the value of the parse first.
The parse value differs from the other debugging aids we'll discuss.
Every other debugging tool we will describe is always available,
no matter how badly the parse failed.
But if you have a problem parsing, you often won't get a parse value.

Our luck holds.
Here's a dump of the parse value at the point of failure.
It's a nice to way to see
what Marpa thinks the parse was so far.

=for Marpa::R2::Display
name: SLIF debug example dump of value
partial: 1
normalize-whitespace: 1

    \bless(
        [
            bless(
                [
                    bless(
                        [
                            'a', '=',
                            bless(
                                [
                                    bless( ['8675309'], 'My_Nodes::expression' ),
                                    '+',
                                    bless( ['42'], 'My_Nodes::expression' )
                                ],
                                'My_Nodes::expression'
                            )
                        ],
                        'My_Nodes::numeric_assignment'
                    )
                ],
                'My_Nodes::statement'
            )
        ],
        'My_Nodes::statements'
    );

=for Marpa::R2::Display::End

If we were perceptive, we might spot the error here.
Our parse is not quite right, and that
shows up in the outer C<My_Nodes::expression> --
it should be
C<My_Nodes::numeric_expression>.
We'll assume that we don't notice this.

In fact, in the following, we'll pretend we haven't seen the
dump of the parse value.
We can't always get a parse value, so we don't want to rely on it.

=head1 Output from trace_terminals()

You can rely on getting the output from
L<C<trace_terminals>|Marpa::R2::Scanless::R/trace_terminals>,
and it is a good next place to check.
Typically, you will be interested in the last tokens
to be accepted.
Sometimes that information alone is enough to make it clear
where the problem is.

The full
L<C<trace_terminals>|Marpa::R2::Scanless::R/trace_terminals>
output for this example is in the Appendix.
We see that the recognizer accepts the input as far
as the multiplication sign ("C<*>"),
which it rejects.
In Marpa, a lexeme is "acceptable" if it fits the grammar
and the input so far.
A lexeme is rejected if it is not acceptable.

The last two lines of the C<trace_terminals> output are:

=for Marpa::R2::Display
name: SLIF debug example trace output
partial: 1
normalize-whitespace: 1

    Discarded lexeme L1c17: whitespace
    Rejected lexeme L1c18: '*'; value="*"

=for Marpa::R2::Display::End

A note in passing: Marpa shows the input string position of the tokens it accepts,
discard and rejects.
C<E<lt>whitespaceE<gt>> is supposed to be discarded
and that was what happened at line 1, column 17.
But the C<'*'> that was next in the input was rejected,
and that was not supposed to happen.

=head1 Output from show_progress()

Marpa's most powerful tool for debugging
grammars is its progress report, which
shows the
Earley items being worked on.
In the Appendix, progress reports for the entire parse are shown.
Our example in this document is a very small one,
so that producing progress reports for the entire parse is
a reasonable thing to do in this case.
If a parse is at all large, you will usually need to be selective.

The progress report that is usually of most interest is
the one for the Earley set
that you were working on when the error occurred.
This is called the B<current location>.
In our example the current location is G1 location 5.
By default,
L<C<show_progress>|Marpa::R2::Scanless::R/"show_progress()">
prints out only the progress reports for the current location.

Here are the progress reports for the current location,
location 5,
from our example.

=for Marpa::R2::Display
name: SLIF debug example progress report
partial: 1
normalize-whitespace: 1

    F0 @0-5 L1c1-16 statements -> statement * .
    P1 @5-5 L1c15-16 statement -> . assignment
    P2 @5-5 L1c15-16 statement -> . <numeric assignment>
    F2 @0-5 L1c1-16 statement -> <numeric assignment> .
    P3 @5-5 L1c15-16 assignment -> . 'set' variable 'to' expression
    P4 @5-5 L1c15-16 <numeric assignment> -> . variable '=' expression
    F4 @0-5 L1c1-16 <numeric assignment> -> variable '=' expression .
    F5 @2-5 L1c3-16 expression -> expression .
    F7 @4-5 L1c13-16 expression -> expression .
    F8 @4-5 L1c13-16 expression -> variable .
    R11:1 @2-5 L1c3-16 expression -> expression . '+' expression
    F11 @2-5 L1c3-16 expression -> expression '+' expression .
    F19 @0-5 L1c1-16 :start -> statements .

=for Marpa::R2::Display::End

=head2 Progress report lines

=for Marpa::R2::Display
name: SLIF debug example progress report
partial: 1
normalize-whitespace: 1

    F19 @0-5 L1c1-16 :start -> statements .

=for Marpa::R2::Display::End

The last field of
each progress report line shows,
in fully expanded form,
the dotted rule we were working on.
Prefixed to the dotted rule are three fields.
In the example just above they are "C<F0 @0-5 L1c1-16>".
The "C<F0>" says that this is a completed or B<final> rule,
and that it is rule number 0.
The rule number is a convenient way to refer to a rule
and is used when displaying the whole rule would take too much space.

The "C<@0-5>" describes the G1 locations of the dotted rule in
the parse.  In its simplest form, the location field is two
G1 location numbers, separated by a hyphen.
The first G1 location number is the origin, the place where Marpa
first started recognizing the rule.
The last G1 location number is the dot location, the G1 location
of the dot in a dotted rule.
"C<@0-3>" says that this rule began at G1 location 0, and that the
dot is at G1 location 3.

Following the G1 location is the range of positions in the input string:
"C<L1c1-16>".
This indicates that the origin of dotted rule is at line 1, column 1,
and that its dot position is after line 1, column 16.

The current location is also just after line 1, column 16, and at G1 location 5,
and this is no coincidence.
Whenever we are displaying the progress report for a G1 location,
all the progress report lines will have their dot location at
that G1 location.

As an aside, notice that
the left hand side symbol is C<:start>.
That is the start pseudo-symbol.
The presence of a completed start rule in our
progress report indicates that if our input had ended at location 5,
it would be a valid sentence
in the language of our grammar.
(And it is because the input at G1 location 5 was a valid sentence of the grammar,
that we were able to look at the value
of the parse at location 5 for debugging purposes.)

Let's look at another progress report line:

=for Marpa::R2::Display
name: SLIF debug example progress report
partial: 1
normalize-whitespace: 1

    R11:2 @2-4 L1c3-13 expression -> expression '+' . expression

=for Marpa::R2::Display::End

Here the "C<R11:2>" indicates that this is rule number 11
(the "C<R>" stands for rule number) and that its dot
position is after the second symbol on the right hand side.
Symbol positions are numbered using the
ordinal of the symbol just before the position.
Symbols are numbered starting with 1, and
symbol position 2 is the position immediately after
symbol 2.

Predicted rules also appear in progress reports:

=for Marpa::R2::Display
name: SLIF debug example progress report
partial: 1
normalize-whitespace: 1

    P2 @3-3 L1c5-11 statement -> . <numeric assignment>

=for Marpa::R2::Display::End

Here the "C<P>" in the summary field means "predicted".
Notice that in the predicted rule,
the origin is the same as the dot location.
This will always be the case with predicted rules.

=head2 OK!  Now to find the bug

If we look again are progress reports
at the location 5, the location where things went wrong:
We see that we have completed rules for
C<E<lt>expressionE<gt>>,
C<E<lt>numeric assignmentE<gt>>,
C<E<lt>statementE<gt>>,
C<E<lt>statementsE<gt>>,
as expected.
We also see two Earley items that show
that we are in the process of building another C<E<lt>expressionE<gt>>,
and that it is expecting a 'C<+>' symbol.

What we want to know is,
why is the recognizer B<not> expecting an 'C<*>' symbol?
Looking back at the grammar, we see that only one rule uses
the 'C<*>' symbol.
Here it is as part of a prioritized rule in the DSL:

=for Marpa::R2::Display
name: SLIF debug example, part 1
partial: 1
normalize-whitespace: 1

    <numeric expression> ::=
           variable | number
        || <numeric expression> '*' <numeric expression>
        || <numeric expression> '+' <numeric expression>

=for Marpa::R2::Display::End

Here it is from the C<show_rules()> listing:

=for Marpa::R2::Display
name: SLIF debug example show_rules() output
partial: 1
normalize-whitespace: 1

    G1 R17 <numeric expression> ::= <numeric expression> '*' <numeric expression>

=for Marpa::R2::Display::End

It's rule 17 in subgrammar G1, and for convenience we will call it R17.
The next step is to look at the Earley items for this rule.
But there is a problem.
We don't find any.

Next, we ask ourselves, what is the earliest place R17
should be appearing?
The answer is that
there should be a prediction of R17 at location 0.
So we look at the predictions at location 0.

=for Marpa::R2::Display
name: SLIF debug example progress report
partial: 1
normalize-whitespace: 1

    P0 @0-0 L0c0 statements -> . statement *
    P1 @0-0 L0c0 statement -> . assignment
    P2 @0-0 L0c0 statement -> . <numeric assignment>
    P3 @0-0 L0c0 assignment -> . 'set' variable 'to' expression
    P4 @0-0 L0c0 <numeric assignment> -> . variable '=' expression
    P19 @0-0 L0c0 :start -> . statements

=for Marpa::R2::Display::End

No R17 predicted at G1 location 0.
Next we look through the the entire progress report, at all G1 locations,
to see if R17 is predicted anywhere.
No R17.  Not anywhere.

The LHS of R17 is C<E<lt>numeric expressionE<gt>>.
We look in the progress report for dotted rules where
C<E<lt>numeric expressionE<gt>> is expected -- that is,
dotted rules where C<E<lt>numeric expressionE<gt>> is the post-dot symbol.
There are none.

Next we look for places in the progress reports where
C<E<lt>numeric expressionE<gt>> occurs at all,
whether post-dot or not.
In the progress reports,
C<E<lt>numeric expressionE<gt>> occurs in only two dotted rule instances.
Here they are:

=for Marpa::R2::Display
name: SLIF debug example progress report
partial: 1
normalize-whitespace: 1

    P10 @2-2 L1c3 expression -> . 'string' '(' <numeric expression> ')'

=for Marpa::R2::Display::End

=for Marpa::R2::Display
name: SLIF debug example progress report
partial: 1
normalize-whitespace: 1

    P10 @4-4 L1c13 expression -> . 'string' '(' <numeric expression> ')'

=for Marpa::R2::Display::End

In both cases these are predictions of a string operator, the operator we plan to
use for converting numerics to strings.
They are just predictions, predictions which go no further because there is no
'C<string>' operator in our input.
That's fine, but why no other, more relevant, occurrences of C<E<lt>numeric expressionE<gt>>?

We look back at the grammar.
Aside for the rule for the 'C<string>' operator,
C<E<lt>numeric expressionE<gt>> occurs on a RHS in two places.
One is in the prioritized rule which defines
C<E<lt>numeric expressionE<gt>>.

=for Marpa::R2::Display
name: SLIF debug example, part 1
partial: 1
normalize-whitespace: 1

    <numeric expression> ::=
           variable | number
        || <numeric expression> '*' <numeric expression>
        || <numeric expression> '+' <numeric expression>

=for Marpa::R2::Display::End

This rule
will never put
C<E<lt>numeric expressionE<gt>> into the Earley items
unless there is
a C<E<lt>numeric expressionE<gt>> already there.
But that is not its job.
This rule is just fine and does not need fixing.

That leaves one rule to look at.

=for Marpa::R2::Display
name: SLIF debug example, part 1
partial: 1
normalize-whitespace: 1

    <numeric assignment> ::= variable '=' expression

=for Marpa::R2::Display::End

This rule is one that
should lead to the prediction of a new
C<E<lt>numeric expressionE<gt>> in our example.
And now we see our problem.
This rule is never leading to the prediction of a new
C<E<lt>numeric expressionE<gt>>,
because there is no
C<E<lt>numeric expressionE<gt>> on its RHS,
or for that matter anywhere else in it.
On the RHS, where we wrote C<E<lt>expressionE<gt>>,
we should have written
C<E<lt>numeric expressionE<gt>>.
Change that and the problem is fixed.

=head1 Complications

We have finished our main example.
This section discusses some aspects of debugging which did not
arise in the example,
and which might be unexpected.

=head2 Empty rules

When a symbol is nulled in your parse,
L<C<show_progress>|Marpa::R2::Scanless::R/"show_progress()">
shows only the nulled symbol.
It does not show the symbol's expansion into rules,
or any of its nulled child symbols.
This reduces clutter,
and usually one does not notice the missing nulled rules and symbols.
Not showing these
seems to be the intuitive way to treat them.

=head2 Input string ranges

G1 locations run in a monotonic sequence, starting with 0.
G1 locations never run backwards, they are never visited twice,
and they leave no gaps.

Input string positions, on the other hand, can do all of these things.
An application is allowed to jump around in the input.
An input string position may be encountered more than once.
It is quite possible to write your application so that it
encounters, for example,
line 42 before line 7.
And your application does not have to visit line 42 on its way from
line 41 to line 43.
For that matter,
an application does not ever have to visit any position in its input.

How does Marpa deal with this when reporting input string ranges?
Marpa always reports the minimum range that includes all the input string positions
visited in the dotted rule.
The range is always reported in increasing numeric order,
even when the position at
the end of the range was visited before the input string position
at the beginning of the range.
And, if necessary to include all visited input string positions, the range
may include input string positions which were not visited.

Most applications move forward continuously in the input string,
and if yours is one of them, you don't have to worry about these issues.
But if you do unusual things when reading the input,
it helps to be aware of how input string ranges are reported by Marpa
when tracing and debugging.

=head2 Multiple instances of dotted rules

It does not happen in our main example for this document,
but a dotted rule can appear in the same Earley set
more than once.
In fact, this happens frequently.
When it does happen, the lines in the progress report will look like these

=for Marpa::R2::Display
name: SLIF Leo show_progress() example
partial: 1
normalize-whitespace: 1

    F11 x12 @0...38-41 L1c1-L2c40 <plain assignment> -> 'x' '=' expression .

=for Marpa::R2::Display::End

=for Marpa::R2::Display
name: SLIF Leo show_progress() example
partial: 1
normalize-whitespace: 1

    F1 x20 @0...38-41 L1c1-L2c40 expression -> assignment .

=for Marpa::R2::Display::End

=for Marpa::R2::Display
name: SLIF Leo show_progress() example
partial: 1
normalize-whitespace: 1

    F6 x12 @0...38-41 L1c1-L2c40 assignment -> <plain assignment> .

=for Marpa::R2::Display::End

These are some of the progress report lines for
an indirect right recursion, one that recurses from
a C<E<lt>plain assignmentE<gt>> symbol to
an  C<E<lt>expressionE<gt>> symbol, and then to
an C<E<lt>assignmentE<gt>> symbol,
before completing the recursion by returning to
a C<E<lt>plain assignmentE<gt>>.

In each of the three lines, notice that a new field appears second.
This second field is variously
"C<x12>" or "C<x20>".
These are counts,
indicating the number of instances of that dotted rule at
the dotted rule's G1 dot location.
Every dotted rule instance will have the same G1 location,
but the instances may have many different origins -- hundreds or even more.
In each of the three report lines above, the G1 dot location is 41.

Note that when parsing, Marpa handles the long series of Earley items
generated by right recursions very efficiently.
It uses a technique invented by Joop Leo to memoize and eliminate them.
When a progress report is requested at a G1 location, the Leo-memoization
is unfolded, and the full list of Earley items is reported.

Each instance may have its own span in the input string,
and the input string range will include them all.
When there are many
instances of a dotted rule at a single location,
the origins in the location field are shown as a
range, with the earliest separated from the most recent
by a "C<...>".
For example, above,
where the first four fields were
"C<F7 x12 @0...38-41 L1c1-L2c40>",
that tells us
that the dotted rule is rule 7, which has 12 instances.
All 12 instances have their dot location at G1 location 41,
but their origins are in the range from G1 location 0
to G1 location 38.

The last field in
"C<F7 x12 @0...38-41 L1c1-L2c40>" is an input string range.
"C<L1c1-L2c40>" says that
input string positions visited
by the 12 instances start at line 1, column 1,
and end at line 2, column 40.
The reported input string range will be the shortest range that includes
all of the input string positions visited by any of the dotted rule instances.

If there are only a few origins,
Marpa may explicitly list them all.
In the follow example, there are only 2 instances
of this rule, both with a dot location of 41.
Their origins are at G1 locations 8 and 18.
The range of input string positions is from line 1, column 17
to line 2, column 40.

=for Marpa::R2::Display
name: SLIF Leo show_progress() example
partial: 1
normalize-whitespace: 1

    F2 x2 @8,18-41 L1c17-L2c40 assignment -> <divide assignment> .

=for Marpa::R2::Display::End

=head1 Access to the "raw" progress report information

This section deals with the C<progress()> recognizer method,
which allows access to the raw progress report information.
This method is not needed for typical debugging and tracing
situations.
It is intended for applications which want to leverage
Marpa's "situational awareness" in innovative ways.

=head2 progress()

=for Marpa::R2::Display
name: SLIF progress(0) example
normalize-whitespace: 1

    my $report0 = $recce->progress(0);

=for Marpa::R2::Display::End

Z<>

=for Marpa::R2::Display
name: SLIF progress() example
normalize-whitespace: 1

    my $latest_report = $recce->progress();

=for Marpa::R2::Display::End

Given the G1 location (Earley set ID) as its argument,
the C<progress()> recognizer method returns a reference
to an array of "report items".
The G1 location may be given as a negative number.
An argument of I<-X>
will be interpreted as G1 location I<N-(X+1)>, where I<N> is
the latest Earley set.
This means that an argument of -1 indicates the latest
Earley set,
an argument of -2 indicates the Earley set just before
the latest one, etc.

Each report item is a triple: an array of three elements.
The three elements are, in order,
rule ID, dot position, and origin.
The data returned by the two displays above,
as well as the data for the other G1 locations
in our example,
are shown L<below|"progress() outputs">.

The rule ID is the same number that Marpa uses
to identify rules
in tracing and debugging output.
Given a rule ID, an application can expand it into
its LHS and RHS symbols using
L<the SLIF grammar's C<rule_expand()> method|Marpa::R2::Scanless::G/"rule_expand()">.
Given a symbol ID,
its name and other information can be found using
L<other SLIF grammar
methods|Marpa::R2::Scanless::G/"Accessors">.

Dot position is -1 for completions,
and 0 for predictions.
Where the report item is not for a completion or
a prediction, dot position is I<N>,
where I<N> is the number of RHS symbols
successfully recognized at the G1 location of
the progress report.

Origin is the G1 location (Earley set ID)
at which the rule application reported by
the report item began.
For a prediction, origin will always be
the same as the G1 location of the parse report.

=head2 Progress reports and efficiency

When progress reports are used for production parsing,
instead of just for debugging and tracing,
efficiency considerations become significant.
Progress reports themselves are implemented in optimized C,
and that logic is very fast.
However, the use of progress reports usually implies considerable
post-processing in Perl.
It is almost always possible to use Marpa's named events instead
of progress reports, and solutions using named events are usually
better targeted, simpler and faster.

If you do decide to use progress reports in an application,
you should be aware of the efficiency considerations when there
are right recursions in the grammar.
For most purposes,
Marpa optimizes right recursions,
so that they run in linear time.
However, to create a progress report every potential right recursion must
be fully unfolded, and at each G1 location the number of these grows linearly with the length
of the recursion.
If you are creating progress reports for more than a limited number of G1 locations,
this means processing that can be quadratic in the length of the recursion.
When a right recursion is lengthy, the impact on speed can be be very serious.

If lengthy right recursions are being expanded,
this will be evident
from the parse report itself,
which will contain one report item
for every completion in the right-recursive
chain of completions.
Note that the efficiency consideration just mentioned
for following right recursions
is never an issue for left recursions.
Left recursions produce at most two report
items per G1 location
and are extremely fast to process.
It is also not an issue for
Marpa's sequence rules, because sequence rules
are implemented internally as left recursions.

=head1 Appendix

Below are the code, the trace outputs
and the progress report
for the example used in this
document.

=head2 Code

=for Marpa::R2::Display:
name: SLIF debug example, part 1
normalize-whitespace: 1

    my $slif_debug_source = <<'END_OF_SOURCE';
    :default ::= action => ::array bless => ::lhs
    :start ::= statements
    statements ::= statement *
    statement ::= assignment | <numeric assignment>
    assignment ::= 'set' variable 'to' expression

    # This is a deliberate error in the grammar
    # The next line should be:
    # <numeric assignment> ::= variable '=' <numeric expression>
    # I have changed the <numeric expression>  to <expression> which
    # will cause problems.
    <numeric assignment> ::= variable '=' expression

    expression ::=
           variable | string
        || 'string' '(' <numeric expression> ')'
        || expression '+' expression
    <numeric expression> ::=
           variable | number
        || <numeric expression> '*' <numeric expression>
        || <numeric expression> '+' <numeric expression>
    variable ~ [\w]+
    number ~ [\d]+
    string ~ ['] <string contents> [']
    <string contents> ~ [^'\x{0A}\x{0B}\x{0C}\x{0D}\x{0085}\x{2028}\x{2029}]+
    :discard ~ whitespace
    whitespace ~ [\s]+
    END_OF_SOURCE

    my $grammar = Marpa::R2::Scanless::G->new(
        {
        bless_package => 'My_Nodes',
        source => \$slif_debug_source,
    });

=for Marpa::R2::Display::End

=for Marpa::R2::Display:
name: SLIF debug example, part 2
perltidy: '-dcsc -sil=0'

    my $recce = Marpa::R2::Scanless::R->new(
        { grammar => $grammar,
        trace_terminals => 1,
        trace_values => 1,
        } );

    my $test_input = 'a = 8675309 + 42 * 711' ;
    my $eval_error = $EVAL_ERROR if not eval { $recce->read( \$test_input ); 1 };

    $progress_report = $recce->show_progress( 0, -1 );

=for Marpa::R2::Display::End

=head2 Error message

=for Marpa::R2::Display
name: SLIF debug example error message
partial: 1
remove-display-indent: 1
remove-blank-last-line: 1

    Error in SLIF parse: No lexemes accepted at line 1, column 18
      Rejected lexeme #0: '*'; value="*"; length = 1
    * String before error: a = 8675309 + 42\s
    * The error was at line 1, column 18, and at character 0x002a '*', ...
    * here: * 711

=for Marpa::R2::Display::End

=head2 Parse value at error location

Note that when there is a parse error, there will
not always be a parse value.
But sometimes the parse is "successful" enough, in a technical sense,
to produce a value, and in those cases examining the value can be
helpful in determining what the parser thinks it has seen so far.

=for Marpa::R2::Display
name: SLIF debug example dump of value
partial: 1
normalize-whitespace: 1

    my $value_ref = $recce->value();
    my $expected_output = \bless(
        [
            bless(
                [
                    bless(
                        [
                            'a', '=',
                            bless(
                                [
                                    bless( ['8675309'], 'My_Nodes::expression' ),
                                    '+',
                                    bless( ['42'], 'My_Nodes::expression' )
                                ],
                                'My_Nodes::expression'
                            )
                        ],
                        'My_Nodes::numeric_assignment'
                    )
                ],
                'My_Nodes::statement'
            )
        ],
        'My_Nodes::statements'
    );

=for Marpa::R2::Display::End

=head2 Trace output

=for Marpa::R2::Display
name: SLIF debug example trace output
remove-display-indent: 1
remove-blank-last-line: 1

    Setting trace_terminals option
    Setting trace_values option
    Accepted lexeme L1c1 e1: variable; value="a"
    Discarded lexeme L1c2: whitespace
    Accepted lexeme L1c3 e2: '='; value="="
    Discarded lexeme L1c4: whitespace
    Rejected lexeme L1c5-11: number; value="8675309"
    Accepted lexeme L1c5-11 e3: variable; value="8675309"
    Discarded lexeme L1c12: whitespace
    Rejected lexeme L1c13: '+'; value="+"
    Accepted lexeme L1c13 e4: '+'; value="+"
    Discarded lexeme L1c14: whitespace
    Rejected lexeme L1c15-16: number; value="42"
    Accepted lexeme L1c15-16 e5: variable; value="42"
    Discarded lexeme L1c17: whitespace
    Rejected lexeme L1c18: '*'; value="*"

=for Marpa::R2::Display::End

=head2 show_progress() output

=for Marpa::R2::Display
name: SLIF debug example progress report
remove-display-indent: 1
remove-blank-last-line: 1

  P0 @0-0 L0c0 statements -> . statement *
  P1 @0-0 L0c0 statement -> . assignment
  P2 @0-0 L0c0 statement -> . <numeric assignment>
  P3 @0-0 L0c0 assignment -> . 'set' variable 'to' expression
  P4 @0-0 L0c0 <numeric assignment> -> . variable '=' expression
  P19 @0-0 L0c0 :start -> . statements
  R4:1 @0-1 L1c1 <numeric assignment> -> variable . '=' expression
  R4:2 @0-2 L1c1-3 <numeric assignment> -> variable '=' . expression
  P5 @2-2 L1c3 expression -> . expression
  P6 @2-2 L1c3 expression -> . expression
  P7 @2-2 L1c3 expression -> . expression
  P8 @2-2 L1c3 expression -> . variable
  P9 @2-2 L1c3 expression -> . string
  P10 @2-2 L1c3 expression -> . 'string' '(' <numeric expression> ')'
  P11 @2-2 L1c3 expression -> . expression '+' expression
  F0 @0-3 L1c1-11 statements -> statement * .
  P1 @3-3 L1c5-11 statement -> . assignment
  P2 @3-3 L1c5-11 statement -> . <numeric assignment>
  F2 @0-3 L1c1-11 statement -> <numeric assignment> .
  P3 @3-3 L1c5-11 assignment -> . 'set' variable 'to' expression
  P4 @3-3 L1c5-11 <numeric assignment> -> . variable '=' expression
  F4 @0-3 L1c1-11 <numeric assignment> -> variable '=' expression .
  F5 @2-3 L1c3-11 expression -> expression .
  F6 @2-3 L1c3-11 expression -> expression .
  F7 @2-3 L1c3-11 expression -> expression .
  F8 @2-3 L1c3-11 expression -> variable .
  R11:1 @2-3 L1c3-11 expression -> expression . '+' expression
  F19 @0-3 L1c1-11 :start -> statements .
  P7 @4-4 L1c13 expression -> . expression
  P8 @4-4 L1c13 expression -> . variable
  P9 @4-4 L1c13 expression -> . string
  P10 @4-4 L1c13 expression -> . 'string' '(' <numeric expression> ')'
  R11:2 @2-4 L1c3-13 expression -> expression '+' . expression
  F0 @0-5 L1c1-16 statements -> statement * .
  P1 @5-5 L1c15-16 statement -> . assignment
  P2 @5-5 L1c15-16 statement -> . <numeric assignment>
  F2 @0-5 L1c1-16 statement -> <numeric assignment> .
  P3 @5-5 L1c15-16 assignment -> . 'set' variable 'to' expression
  P4 @5-5 L1c15-16 <numeric assignment> -> . variable '=' expression
  F4 @0-5 L1c1-16 <numeric assignment> -> variable '=' expression .
  F5 @2-5 L1c3-16 expression -> expression .
  F7 @4-5 L1c13-16 expression -> expression .
  F8 @4-5 L1c13-16 expression -> variable .
  R11:1 @2-5 L1c3-16 expression -> expression . '+' expression
  F11 @2-5 L1c3-16 expression -> expression '+' expression .
  F19 @0-5 L1c1-16 :start -> statements .

=for Marpa::R2::Display::End

=head2 show_rules() output

This is the G1 portion of the C<show_rules()> output at verbosity level 3.
In ordinary work, you'd use verbosity level 1 (the default),
but the more verbose output is included here to illustrate the example.

=for Marpa::R2::Display
name: SLIF debug example show_rules() output
remove-display-indent: 1
remove-blank-last-line: 1
partial: 1

  G1 Rules:
  G1 R0 statements ::= statement *
    Symbol IDs: <16> ::= <17>
    Internal symbols: <statements> ::= <statement>
  G1 R1 statement ::= assignment
    Symbol IDs: <17> ::= <18>
    Internal symbols: <statement> ::= <assignment>
  G1 R2 statement ::= <numeric assignment>
    Symbol IDs: <17> ::= <19>
    Internal symbols: <statement> ::= <numeric assignment>
  G1 R3 assignment ::= 'set' variable 'to' expression
    Symbol IDs: <18> ::= <1> <20> <2> <21>
    Internal symbols: <assignment> ::= <[Lex-0]> <variable> <[Lex-1]> <expression>
  G1 R4 <numeric assignment> ::= variable '=' <numeric expression>
    Symbol IDs: <19> ::= <20> <3> <22>
    Internal symbols: <numeric assignment> ::= <variable> <[Lex-2]> <numeric expression>
  G1 R5 expression ::= expression
    Internal rule top priority rule for <expression>
    Symbol IDs: <21> ::= <10>
    Internal symbols: <expression> ::= <expression[0]>
  G1 R6 expression ::= expression
    Internal rule for symbol <expression> priority transition from 0 to 1
    Symbol IDs: <10> ::= <11>
    Internal symbols: <expression[0]> ::= <expression[1]>
  G1 R7 expression ::= expression
    Internal rule for symbol <expression> priority transition from 1 to 2
    Symbol IDs: <11> ::= <12>
    Internal symbols: <expression[1]> ::= <expression[2]>
  G1 R8 expression ::= variable
    Symbol IDs: <12> ::= <20>
    Internal symbols: <expression[2]> ::= <variable>
  G1 R9 expression ::= string
    Symbol IDs: <12> ::= <23>
    Internal symbols: <expression[2]> ::= <string>
  G1 R10 expression ::= 'string' '(' <numeric expression> ')'
    Symbol IDs: <11> ::= <4> <5> <22> <6>
    Internal symbols: <expression[1]> ::= <[Lex-3]> <[Lex-4]> <numeric expression> <[Lex-5]>
  G1 R11 expression ::= expression '+' expression
    Symbol IDs: <10> ::= <10> <7> <11>
    Internal symbols: <expression[0]> ::= <expression[0]> <[Lex-6]> <expression[1]>
  G1 R12 <numeric expression> ::= <numeric expression>
    Internal rule top priority rule for <numeric expression>
    Symbol IDs: <22> ::= <13>
    Internal symbols: <numeric expression> ::= <numeric expression[0]>
  G1 R13 <numeric expression> ::= <numeric expression>
    Internal rule for symbol <numeric expression> priority transition from 0 to 1
    Symbol IDs: <13> ::= <14>
    Internal symbols: <numeric expression[0]> ::= <numeric expression[1]>
  G1 R14 <numeric expression> ::= <numeric expression>
    Internal rule for symbol <numeric expression> priority transition from 1 to 2
    Symbol IDs: <14> ::= <15>
    Internal symbols: <numeric expression[1]> ::= <numeric expression[2]>
  G1 R15 <numeric expression> ::= variable
    Symbol IDs: <15> ::= <20>
    Internal symbols: <numeric expression[2]> ::= <variable>
  G1 R16 <numeric expression> ::= number
    Symbol IDs: <15> ::= <24>
    Internal symbols: <numeric expression[2]> ::= <number>
  G1 R17 <numeric expression> ::= <numeric expression> '*' <numeric expression>
    Symbol IDs: <14> ::= <14> <8> <15>
    Internal symbols: <numeric expression[1]> ::= <numeric expression[1]> <[Lex-7]> <numeric expression[2]>
  G1 R18 <numeric expression> ::= <numeric expression> '+' <numeric expression>
    Symbol IDs: <13> ::= <13> <9> <14>
    Internal symbols: <numeric expression[0]> ::= <numeric expression[0]> <[Lex-8]> <numeric expression[1]>
  G1 R19 :start ::= statements
    Symbol IDs: <0> ::= <16>
    Internal symbols: <[:start]> ::= <statements>
  Lex (L0) Rules:
  L0 R0 'set' ::= [s] [e] [t]
    Internal rule for single-quoted string 'set'
    Symbol IDs: <2> ::= <27> <21> <28>
    Internal symbols: <[Lex-0]> ::= <[[s]]> <[[e]]> <[[t]]>
  L0 R1 'to' ::= [t] [o]
    Internal rule for single-quoted string 'to'
    Symbol IDs: <3> ::= <28> <25>
    Internal symbols: <[Lex-1]> ::= <[[t]]> <[[o]]>
  L0 R2 '=' ::= [\=]
    Internal rule for single-quoted string '='
    Symbol IDs: <4> ::= <16>
    Internal symbols: <[Lex-2]> ::= <[[\=]]>
  L0 R3 'string' ::= [s] [t] [r] [i] [n] [g]
    Internal rule for single-quoted string 'string'
    Symbol IDs: <5> ::= <27> <28> <26> <23> <24> <22>
    Internal symbols: <[Lex-3]> ::= <[[s]]> <[[t]]> <[[r]]> <[[i]]> <[[n]]> <[[g]]>
  L0 R4 '(' ::= [\(]
    Internal rule for single-quoted string '('
    Symbol IDs: <6> ::= <12>
    Internal symbols: <[Lex-4]> ::= <[[\(]]>
  L0 R5 ')' ::= [\)]
    Internal rule for single-quoted string ')'
    Symbol IDs: <7> ::= <13>
    Internal symbols: <[Lex-5]> ::= <[[\)]]>
  L0 R6 '+' ::= [\+]
    Internal rule for single-quoted string '+'
    Symbol IDs: <8> ::= <15>
    Internal symbols: <[Lex-6]> ::= <[[\+]]>
  L0 R7 '*' ::= [\*]
    Internal rule for single-quoted string '*'
    Symbol IDs: <9> ::= <14>
    Internal symbols: <[Lex-7]> ::= <[[\*]]>
  L0 R8 '+' ::= [\+]
    Internal rule for single-quoted string '+'
    Symbol IDs: <10> ::= <15>
    Internal symbols: <[Lex-8]> ::= <[[\+]]>
  L0 R9 variable ::= [\w] +
    Symbol IDs: <29> ::= <19>
    Internal symbols: <variable> ::= <[[\w]]>
  L0 R10 number ::= [\d] +
    Symbol IDs: <30> ::= <17>
    Internal symbols: <number> ::= <[[\d]]>
  L0 R11 string ::= ['] <string contents> [']
    Symbol IDs: <31> ::= <11> <32> <11>
    Internal symbols: <string> ::= <[[']]> <string contents> <[[']]>
  L0 R12 <string contents> ::= [^'\x{0A}\x{0B}\x{0C}\x{0D}\x{0085}\x{2028}\x{2029}] +
    Symbol IDs: <32> ::= <20>
    Internal symbols: <string contents> ::= <[[^'\x{0A}\x{0B}\x{0C}\x{0D}\x{0085}\x{2028}\x{2029}]]>
  L0 R13 :discard ::= whitespace
    Discard rule for <whitespace>
    Symbol IDs: <0> ::= <33>
    Internal symbols: <[:discard]> ::= <whitespace>
  L0 R14 whitespace ::= [\s] +
    Symbol IDs: <33> ::= <18>
    Internal symbols: <whitespace> ::= <[[\s]]>
  L0 R15 :start_lex ::= :discard
    Internal lexical start rule for <[:discard]>
    Symbol IDs: <1> ::= <0>
    Internal symbols: <[:start_lex]> ::= <[:discard]>
  L0 R16 :start_lex ::= 'set'
    Internal lexical start rule for <[Lex-0]>
    Symbol IDs: <1> ::= <2>
    Internal symbols: <[:start_lex]> ::= <[Lex-0]>
  L0 R17 :start_lex ::= 'to'
    Internal lexical start rule for <[Lex-1]>
    Symbol IDs: <1> ::= <3>
    Internal symbols: <[:start_lex]> ::= <[Lex-1]>
  L0 R18 :start_lex ::= '='
    Internal lexical start rule for <[Lex-2]>
    Symbol IDs: <1> ::= <4>
    Internal symbols: <[:start_lex]> ::= <[Lex-2]>
  L0 R19 :start_lex ::= 'string'
    Internal lexical start rule for <[Lex-3]>
    Symbol IDs: <1> ::= <5>
    Internal symbols: <[:start_lex]> ::= <[Lex-3]>
  L0 R20 :start_lex ::= '('
    Internal lexical start rule for <[Lex-4]>
    Symbol IDs: <1> ::= <6>
    Internal symbols: <[:start_lex]> ::= <[Lex-4]>
  L0 R21 :start_lex ::= ')'
    Internal lexical start rule for <[Lex-5]>
    Symbol IDs: <1> ::= <7>
    Internal symbols: <[:start_lex]> ::= <[Lex-5]>
  L0 R22 :start_lex ::= '+'
    Internal lexical start rule for <[Lex-6]>
    Symbol IDs: <1> ::= <8>
    Internal symbols: <[:start_lex]> ::= <[Lex-6]>
  L0 R23 :start_lex ::= '*'
    Internal lexical start rule for <[Lex-7]>
    Symbol IDs: <1> ::= <9>
    Internal symbols: <[:start_lex]> ::= <[Lex-7]>
  L0 R24 :start_lex ::= '+'
    Internal lexical start rule for <[Lex-8]>
    Symbol IDs: <1> ::= <10>
    Internal symbols: <[:start_lex]> ::= <[Lex-8]>
  L0 R25 :start_lex ::= number
    Internal lexical start rule for <number>
    Symbol IDs: <1> ::= <30>
    Internal symbols: <[:start_lex]> ::= <number>
  L0 R26 :start_lex ::= string
    Internal lexical start rule for <string>
    Symbol IDs: <1> ::= <31>
    Internal symbols: <[:start_lex]> ::= <string>
  L0 R27 :start_lex ::= variable
    Internal lexical start rule for <variable>
    Symbol IDs: <1> ::= <29>
    Internal symbols: <[:start_lex]> ::= <variable>

=for Marpa::R2::Display::End

=head2 show_symbols() output

=for Marpa::R2::Display
name: SLIF debug example show_symbols() output
remove-display-indent: 1
remove-blank-last-line: 1
partial: 1

    G1 Symbols:
    G1 S0 :start -- Internal G1 start symbol
      Internal name: <[:start]>
    G1 S1 'set' -- Internal lexical symbol for "'set'"
      /* terminal */
      Internal name: <[Lex-0]>
      SLIF name: 'set'
    G1 S2 'to' -- Internal lexical symbol for "'to'"
      /* terminal */
      Internal name: <[Lex-1]>
      SLIF name: 'to'
    G1 S3 '=' -- Internal lexical symbol for "'='"
      /* terminal */
      Internal name: <[Lex-2]>
      SLIF name: '='
    G1 S4 'string' -- Internal lexical symbol for "'string'"
      /* terminal */
      Internal name: <[Lex-3]>
      SLIF name: 'string'
    G1 S5 '(' -- Internal lexical symbol for "'('"
      /* terminal */
      Internal name: <[Lex-4]>
      SLIF name: '('
    G1 S6 ')' -- Internal lexical symbol for "')'"
      /* terminal */
      Internal name: <[Lex-5]>
      SLIF name: ')'
    G1 S7 '+' -- Internal lexical symbol for "'+'"
      /* terminal */
      Internal name: <[Lex-6]>
      SLIF name: '+'
    G1 S8 '*' -- Internal lexical symbol for "'*'"
      /* terminal */
      Internal name: <[Lex-7]>
      SLIF name: '*'
    G1 S9 '+' -- Internal lexical symbol for "'+'"
      /* terminal */
      Internal name: <[Lex-8]>
      SLIF name: '+'
    G1 S10 expression -- <expression> at priority 0
      Internal name: <expression[0]>
      SLIF name: expression
    G1 S11 expression -- <expression> at priority 1
      Internal name: <expression[1]>
      SLIF name: expression
    G1 S12 expression -- <expression> at priority 2
      Internal name: <expression[2]>
      SLIF name: expression
    G1 S13 <numeric expression> -- <numeric expression> at priority 0
      Internal name: <numeric expression[0]>
      SLIF name: numeric expression
    G1 S14 <numeric expression> -- <numeric expression> at priority 1
      Internal name: <numeric expression[1]>
      SLIF name: numeric expression
    G1 S15 <numeric expression> -- <numeric expression> at priority 2
      Internal name: <numeric expression[2]>
      SLIF name: numeric expression
    G1 S16 statements
      Internal name: <statements>
    G1 S17 statement
      Internal name: <statement>
    G1 S18 assignment
      Internal name: <assignment>
    G1 S19 <numeric assignment>
      Internal name: <numeric assignment>
    G1 S20 variable
      /* terminal */
      Internal name: <variable>
    G1 S21 expression
      Internal name: <expression>
    G1 S22 <numeric expression>
      Internal name: <numeric expression>
    G1 S23 string
      /* terminal */
      Internal name: <string>
    G1 S24 number
      /* terminal */
      Internal name: <number>

=for Marpa::R2::Display::End

=head2 progress() outputs

These section contains samples of
the output of the C<progress()>
method -- the progress reports in their "raw" format.
The output is shown in L<Data::Dumper> format,
with C<Data::Dumper::Indent> set to 0
and C<Data::Dumper::Terse> set to 1.

The C<Data::Dumper> output from L<C<progress()>|/"progress()"> at G1 location 0:

=for Marpa::R2::Display
name: SLIF progress() output at location 0
remove-display-indent: 1
remove-blank-last-line: 1

    [[0,0,0],[1,0,0],[2,0,0],[3,0,0],[4,0,0],[19,0,0]]

=for Marpa::R2::Display::End

The C<Data::Dumper> output from L<C<progress()>|/"progress()"> at G1 location 1:

=for Marpa::R2::Display
name: SLIF progress() output at location 1
remove-display-indent: 1
remove-blank-last-line: 1

    [[4,1,0]]

=for Marpa::R2::Display::End

The C<Data::Dumper> output from L<C<progress()>|/"progress()"> at location 2:

=for Marpa::R2::Display
name: SLIF progress() output at location 2
remove-display-indent: 1
remove-blank-last-line: 1

    [[5,0,2],[6,0,2],[7,0,2],[8,0,2],[9,0,2],[10,0,2],[11,0,2],[4,2,0]]

=for Marpa::R2::Display::End

The default L<C<progress()>|/"progress()"> output is for the latest Earley set.
Here is the L<C<progress()>|/"progress()"> output for the latest Earley set.

=for Marpa::R2::Display
name: SLIF progress() output at default location
remove-display-indent: 1
remove-blank-last-line: 1

    [[0,-1,0],[2,-1,0],[4,-1,0],[5,-1,2],[7,-1,4],[8,-1,4],[11,-1,2],[19,-1,0],[1,0,5],[2,0,5],[3,0,5],[4,0,5],[11,1,2]]

=for Marpa::R2::Display::End

=head1 Copyright and License

=for Marpa::R2::Display
ignore: 1

  Copyright 2022 Jeffrey Kegler
  This file is part of Marpa::R2.  Marpa::R2 is free software: you can
  redistribute it and/or modify it under the terms of the GNU Lesser
  General Public License as published by the Free Software Foundation,
  either version 3 of the License, or (at your option) any later version.

  Marpa::R2 is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
  Lesser General Public License for more details.

  You should have received a copy of the GNU Lesser
  General Public License along with Marpa::R2.  If not, see
  http://www.gnu.org/licenses/.

=for Marpa::R2::Display::End

=cut

# Local Variables:
#   mode: cperl
#   cperl-indent-level: 4
#   fill-column: 100
# End:
# vim: expandtab shiftwidth=4: