1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
|
<!DOCTYPE html>
<html>
<!-- Created by GNU Texinfo 7.0.3, https://www.gnu.org/software/texinfo/ -->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<!-- Copyright (C) 2017-2022 - Matthew R. Wette.
Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
copy of the license is included with the distribution as COPYING.DOC. -->
<title>FFI Helper’s Guide</title>
<meta name="description" content="FFI Helper’s Guide">
<meta name="keywords" content="FFI Helper’s Guide">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="texi2any">
<meta name="viewport" content="width=device-width,initial-scale=1">
<link href="#Top" rel="start" title="Top">
<link href="dir.html#Top" rel="up" title="(dir)">
<link href="#Introduction" rel="next" title="Introduction">
<link href="dir.html#Top" rel="prev" title="(dir)">
<style type="text/css">
<!--
a.copiable-link {visibility: hidden; text-decoration: none; line-height: 0em}
div.example {margin-left: 3.2em}
pre.format-preformatted {font-family: inherit}
span:hover a.copiable-link {visibility: visible}
strong.def-name {font-family: monospace; font-weight: bold; font-size: larger}
ul.mark-bullet {list-style-type: disc}
-->
</style>
</head>
<body lang="en">
<div class="format">
<pre class="format-preformatted">FFI Helper User Guide
Matt Wette
October 2022
With NYACC Version 1.09.4
</pre></div>
<div class="top-level-extent" id="Top">
<div class="nav-panel">
<p>
Next: <a href="#Introduction" accesskey="n" rel="next">Introduction</a>, Previous: <a href="dir.html#Top" accesskey="p" rel="prev">(dir)</a>, Up: <a href="dir.html#Top" accesskey="u" rel="up">(dir)</a> </p>
</div>
<h1 class="top" id="NYACC-FFI-Helper-Guide">NYACC FFI Helper Guide</h1>
<p>This is a user guide for the NYACC FFI Helper.
</p>
<hr>
<a class="node-id" id="Introduction"></a><div class="nav-panel">
<p>
</p>
</div>
<h4 class="node">Introduction</h4>
<h3 class="heading" id="Introduction-1">Introduction</h3>
<p>The acronym FFI stands for “Foreign Function Interface”. It refers
to the Guile facility for binding functions and variables from C source
libraries into Guile programs. This distribution provides utilities
for generating a loadable Guile module from a set of C declarations
and associated libraries. The C declarations can, and conventionally
do, come from naming a set of C include files. The
nominal method for use is to write a <em class="emph">ffi-module</em> specification
in a file which includes a <code class="code">define-ffi-module</code> declaration, and
then use the command <code class="code">guild compile-ffi</code> to produce an associated
file of Guile Scheme code.
</p><div class="example">
<pre class="example-preformatted">$ guild compile-ffi ffi/cairo.ffi
wrote `ffi/cairo.scm'
</pre></div>
<p>The FH does not generate C code. The hooks to access functions in the
Cairo library are provided in 100% Guile Scheme via <code class="code">(system
foreign)</code>.
</p>
<p>To generate Guile Scheme for smaller C code units one can
write a ffi-module with the <code class="code">#:api-code</code> or import the
<code class="code">ffi-help</code> module an use the functions <code class="code">load-include-file</code>,
<code class="code">C-decl->scm</code> or <code class="code">C-fun-decl->scm</code>. The latter functions
are not well tested, though.
</p>
<p>The compiler for the FFI Helper (FH) is based on the C parser and utilities
which are included in the <a class="uref" href="https://www.nongnu.org/nyacc"><small class="sc">NYACC</small></a>
package. Within the <small class="sc">NYACC</small> distribution, there are a number of
example dot-ffi files in the directory <samp class="file">examples/ffi</samp>.
</p>
<p>Use of the FFI-helper module depends on the
<em class="emph">scheme-bytestructure</em> package available from
<a class="uref" href="https://github.com/TaylanUB/scheme-bytestructures">https://github.com/TaylanUB/scheme-bytestructures</a>. Releases
are available at
<a class="uref" href="https://github.com/TaylanUB/scheme-bytestructures/releases">https://github.com/TaylanUB/scheme-bytestructures/releases</a>.
</p>
<p>At runtime, after the FFI Helper has been used to create Scheme code,
the modules <code class="code">(system ffi-help-rt)</code> and <code class="code">(bytestructures
guile)</code> are required. No other code from the <small class="sc">NYACC</small> distribution
is needed. However, note that the process of creating the Scheme
output depends on reading system headers, so the generated code may
well contain operating system and machine dependencies. If you copy
code to a new machine, you should re-run <code class="code">guild compile-ffi</code>.
</p>
<p>You are probably hoping to see an example, so let’s try one.
</p>
<p>This is a small FH example to illustrate its use. We will
start with the <a class="uref" href="cairographics.org">Cairo</a> package because that is
the first one I started with in developing the FFI Helper. Say you
are an avid Guile user and want to be able to use Cairo in Guile.
On most systems Cairo comes with the associated
<em class="emph">pkg-config</em> support files; this demo depends on that support.
</p>
<p>Warning: The FFI Helper package is under active development and there
is some chance the following example will cease to work in the future.
</p>
<p>If you want to follow along and are working in the distribution
tree, you should source the file <samp class="file">env.sh</samp> in the <samp class="file">examples</samp>
directory.
</p>
<p>By practice, I like to put all FH generated modules under a directory
called <samp class="file">ffi/</samp>, so we will do that. We start by generating, in
the <samp class="file">ffi</samp> directory, a file named <samp class="file">cairo.ffi</samp> with the
following contents:
</p>
<div class="example">
<pre class="example-preformatted">(define-ffi-module (ffi cairo)
#:pkg-config "cairo"
#:include '("cairo.h" "cairo-pdf.h" "cairo-svg.h"))
</pre></div>
<p>To generate a Guile module you execute <code class="code">guild</code> as follows:
</p>
<div class="example">
<pre class="example-preformatted">$ guild compile-ffi ffi/cairo.ffi
wrote `ffi/cairo.scm'
</pre></div>
<p>Though the file <samp class="file">cairo/cairo.ffi</samp> is only three lines long, the file
<samp class="file">ffi/cairo.scm</samp> will be over five thousand lines long. It looks
like the following:
</p>
<div class="example">
<pre class="example-preformatted">(define-module (ffi cairo)
#:use-module (system ffi-help-rt)
#:use-module ((system foreign) #:prefix ffi:)
#:use-module (bytestructures guile))
(define link-libs
(list (dynamic-link "libcairo")))
;; int cairo_version(void);
(define ~cairo_version
(delay (fh-link-proc ffi:int "cairo_version" (list) link-libs)))
(define (cairo_version)
(let () ((force ~cairo_version))))
(export cairo_version)
…
;; typedef struct _cairo_matrix {
;; double xx;
;; double yx;
;; double xy;
;; double yy;
;; double x0;
;; double y0;
;; } cairo_matrix_t;
(define-public cairo_matrix_t-desc
(bs:struct
(list `(xx ,double) `(yx ,double) `(xy ,double)
`(yy ,double) `(x0 ,double) `(y0 ,double))))
(define-fh-compound-type cairo_matrix_t cairo_matrix_t-desc
cairo_matrix_t? make-cairo_matrix_t)
(export cairo_matrix_t cairo_matrix_t? make-cairo_matrix_t)
… <i class="i">many, many more declarations</i> …
;; access to enum symbols and #define'd constants:
(define ffi-cairo-symbol-val
(let ((sym-tab
'((CAIRO_SVG_VERSION_1_1 . 0)
(CAIRO_SVG_VERSION_1_2 . 1)
(CAIRO_PDF_VERSION_1_4 . 0)
(CAIRO_PDF_VERSION_1_5 . 1)
(CAIRO_REGION_OVERLAP_IN . 0)
(CAIRO_REGION_OVERLAP_OUT . 1)
… <i class="i">more constants</i> …
(CAIRO_MIME_TYPE_JBIG2_GLOBAL_ID
.
"application/x-cairo.jbig2-global-id"))))
(lambda (k) (or (assq-ref sym-tab k)))))
(export ffi-cairo-symbol-val)
(export cairo-lookup)
… <i class="i">more</i> …
</pre></div>
<p>Note that from the <em class="emph">pkg-config</em> spec the FH compiler picks up the
required libraries to bind in. Also, <code class="code">#define</code> based constants,
as well as those defined by enums, are provided in a lookup function
<code class="code">ffi-cairo-symbol-val</code>. So, for example
</p>
<div class="example">
<pre class="example-preformatted">guile> (use-modules (ffi cairo))
;;; ffi/cairo.scm:6112:11: warning:
possibly unbound variable `cairo_raster_source_acquire_func_t*'
;;; ffi/cairo.scm:6115:11: warning:
possibly unbound variable `cairo_raster_source_release_func_t*'
guile> (ffi-cairo-symbol-val 'CAIRO_FORMAT_ARGB32))
$1 = 0
</pre></div>
<p>We will discuss the warnings later. They are signals that extra code
needs to be added to the ffi module. But you see how the constants
(but not CPP function macros) can be accessed.
</p>
<p>Let’s try something more useful: a real program. Create the following
code in a file, say <code class="code">cairo-demo.scm</code>, then fire up a Guile session
and <code class="code">load</code> the file.
</p>
<div class="example">
<pre class="example-preformatted">(use-modules (ffi cairo))
(define srf (cairo_image_surface_create 'CAIRO_FORMAT_ARGB32 200 200))
(define cr (cairo_create srf))
(cairo_move_to cr 10.0 10.0)
(cairo_line_to cr 190.0 10.0)
(cairo_line_to cr 190.0 190.0)
(cairo_line_to cr 10.0 190.0)
(cairo_line_to cr 10.0 10.0)
(cairo_stroke cr)
(cairo_surface_write_to_png srf "cairo-demo.png")
(cairo_destroy cr)
(cairo_surface_destroy srf)
</pre></div>
<div class="example">
<pre class="example-preformatted">guile> (load "cairo-demo.scm")
…
;;; compiled /.../cairo.scm.go
;;; compiled /.../cairo-demo.scm.go
guile>
</pre></div>
<p>If we set up everything correctly we should have generared the target
file <samp class="file">cairo-demo.png</samp> which contains the image of a square. A
few items in the above code are notable. First, the call to
<code class="code">cairo_image_surface_create</code> accepted a symbolic form
<code class="code">'CAIRO_FORMAT_ARGB32</code> for the format argument. It would have
also accepted the associated constant <code class="code">0</code>. In addition,
procedures declared in <code class="code">(ffi cairo)</code> will accept Scheme strings
where the C function wants “pointer to string.”
</p>
<p>Now try this in your Guile session:
</p>
<div class="example">
<pre class="example-preformatted">guile> srf
$4 = #<cairo_surface_t* 0x7fda53e01880>
guile> cr
$5 = #<cairo_t* 0x7fda54828800>
</pre></div>
<p>Note that the FH keeps track of the C types you use. This can be
useful for debugging (at a potential cost of bloating the namespace).
The constants you see are the pointer values. But it goes further.
Let’s generate a matrix type:
</p>
<div class="example">
<pre class="example-preformatted">guile> (define m (make-cairo_matrix_t))
guile> m
$6 = #<cairo_matrix_t 0x10cc26c00>
guile> (use-modules (system ffh-help-rt))
guile> (pointer-to m)
$7 = #<cairo_matrix_t* 0x10cc26c00>
</pre></div>
<p>When it comes to C APIs that expect the user to allocate memory for a
structure and pass the pointer address to the C function, FH provides
a solution:
</p>
<div class="example">
<pre class="example-preformatted">guile> (cairo_get_matrix cr (pointer-to m))
guile> (fh-object-ref m 'xx)
$9 = 1.0
</pre></div>
<p>But the FFI helper can also be used on a per declaration basis, but
you must first import the proper modules and libraries. This
functionality is still under development.
</p>
<p>The following example shows how to convert to scheme code using
the procedure <code class="code">C-decl->scm</code>:
</p>
<div class="example">
<pre class="example-preformatted">guile> (use-modules (nyacc lang c99 ffi-help))
guile> (define struct-foo-desc (C-decl->scm "struct foo { int x; double y; };"))
guile> ,pp struct-foo-desc
$1 = (bs:struct (list `(x ,int) `(y ,double)))
</pre></div>
<p>If we import more modules we can use the syntax <code class="code">C-decl</code> to
complete definitions:
</p>
<div class="example">
<pre class="example-preformatted">guile> (use-modules (system ffi-help-rt))
guile> (use-modules (bytestructures guile))
guile> (use-modules ((system foreign) #:prefix ffi:))
guile> (define ffi-libs '()) ;; hack for now
guile> (define my-sqrt "double sqrt(double);")
guile> (my-sqrt 4.0)
</pre></div>
<p>Note that for functions like the above to work any dependent libraries
must be loaded first, via <code class="code">(dynamic-link)</code>.
</p>
<p>Note: currently the above defines a bytestructure, but not a FH type.
We could define a FH type as follows:
</p>
<div class="example">
<pre class="example-preformatted">guile> (define-fh-compound-type struct-foo
struct-foo-desc struct-foo? make-struct-foo)
</pre></div>
<h3 class="heading" id="The-Guile-Foreign-Function-Interface">The Guile Foreign Function Interface</h3>
<p>Guile has an API, called the Foreign Function Interface, which allows
one to avoid writing and compiling C wrapper code in order to access C
coded libraries. The API is based on <code class="code">libffi</code> and is covered in
the Guile Reference Manual. We review some important bits here. For
more insight you should read the relevant sections in the Guile
Reference Manual. For more info on libffi internals visit
<a class="uref" href="https://github.com/libffi/libffi">libffi</a>.
</p>
<p>The relevant procedures used by the FH are
</p><dl class="table">
<dt><code class="code">dynamic-link</code></dt>
<dd><p>links libraries into Guile session
</p></dd>
<dt><code class="code">dynamic-func</code></dt>
<dd><p>generated Scheme-level pointer to a C function
</p></dd>
<dt><code class="code">pointer->procedure</code></dt>
<dd><p>geneates a Scheme lambda given C function signature
</p></dd>
<dt><code class="code">dynamic-pointer</code></dt>
<dd><p>provides access to global C variables
</p></dd>
</dl>
<p>Several of the above require import of the module <code class="code">(system foreign)</code>.
</p>
<p>In order to generate a Guile procedure wrapper for a function, say
<code class="code">int foo(char *str)</code>, in some foreign library, say
<samp class="file">libbar.so</samp>, you can use something like the following:
</p><div class="example">
<pre class="example-preformatted">(use-modules (system foreign))
(define foo (pointer->procedure
int
(dynamic-func "foo" (dynamic-link "libbar"))
(list '*)))
</pre></div>
<p>The argument <code class="code">int</code> is a variable name for the return type,
the next argument is an expression for the function pointer and the
third argument is an expression for the function argument list.
To execute the function, which expects a C string, you use something like
</p><div class="example">
<pre class="example-preformatted">(define result-code (foo (string->pointer "hello")))
</pre></div>
<p>If you want to try a real example, this should work:
</p><div class="example">
<pre class="example-preformatted">guile> (use-modules (system foreign))
guile> (define strlen
(pointer->procedure
int (dynamic-func "strlen" (dynamic-link)) (list '*)))
guile> (strlen (string->pointer "hello, world"))
$1 = 12
</pre></div>
<p>It is important to realize that internally Guile takes care of
converting Scheme arguments to and from C types. Scheme does not have
the same type system as C and the Guile FFI is somewhat forgiving
here. When we declare a C function interface with, say, an uint32
argument type, in Scheme you can pass an exact numeric integer. The
FH attempts to be even more forgiving, allowing one to pass symbols
where C enums (i.e., integers) are expected.
</p>
<p>As mentioned, access to libraries not compiled into Guile is
accomplished via <code class="code">dynamic-link</code>. To link the
shared library <samp class="file">libfoo.so</samp> into Guile one would write something
like the following:
</p><div class="example">
<pre class="example-preformatted">(define foo-lib (dynamic-link "libfoo"))
</pre></div>
<p>Note that Guile takes care of dealing with the file extension (e.g.,
<samp class="file">.so</samp>). Where Guile looks for libraries is system dependent,
but usually it will find shared objects in the following
</p><ul class="itemize mark-bullet">
<li><code class="code">(assq-ref %guile-build-info 'libdir)</code>
</li><li><code class="code">(assq-ref %guile-build-info 'extensiondir)</code>
</li><li><samp class="file">/usr/lib</samp> on GNU/Linux and macOS
</li><li>$DYLD_LIBRARY_PATH on GNU/Linux and macOS
</li><li>directories listed in /etc/ld.so.conf on GNU/Linux
</li></ul>
<p>When used with no argument <code class="code">dynamic-link</code> returns a handle for
objects already linked with Guile. The procedure <code class="code">dynamic-link</code>
returns a library handle for acquiring function and variable handles,
or pointers, for objects (e.g., a pointer for a function) in the
library. Theoretically, once a library has been dynamically linked
into Guile, the expression <code class="code">(dynamic-link)</code> (with no argument)
should suffice to provide a handle to acquire object handles, but I
have found this is not always the case. The FH will try all
library handles defined by a ffi module to acquire object pointers.
</p>
<h3 class="heading" id="The-FFI-Helper-Design">The FFI Helper Design</h3>
<p>In this section we hope to provide some insight into the FH works.
The FH specification, via the dot-ffi file, determines the set of
declarations which will be included in the target Guile module. If
there is no declartion filter, then all the declarations from the
specified set of include files are targeted. With the use of a declaration
filter, this set can be reduced. By declaration we mean typedefs,
aggregate definitions (i.e., structs and unions), function
declarations, and external variables.
</p>
<p>In the C language typedefs define type aliases, so there is no harm in
expanding typedefs which appear outside the specification. For
example, say the file <samp class="file">foo.h</samp> includes a declaration for the
typedef <code class="code">foo_t</code> and the file <samp class="file">bar.h</samp> includes a declaration
for the typedef <code class="code">bar_t</code>. Furthermore, suppose <code class="code">foo_t</code> is a
struct that references <code class="code">bar_t</code>. Then the FH will preserve the
typedef <code class="code">foo_t</code> but expand <code class="code">bar_t</code>. That is, if the
declarations are
</p>
<div class="example">
<pre class="example-preformatted">typedef int bar_t; /* from bar.h */
typedef struct { bar_t x; double y; } foo_t; /* from foo.h */
</pre></div>
<p>then the FH will treat <code class="code">foo_t</code> as if it had been declared as
</p>
<div class="example">
<pre class="example-preformatted">typedef struct { int x; double y; } foo_t; /* from foo.h */
</pre></div>
<p>When it comes to handling C types in Scheme the FH tries to leave base
types (i.e., numeric types) alone and uses its own type system, based
on Guiles <em class="emph">structs</em> and associated <em class="emph">vtables</em>, for structs,
unions, function types and pointer types. Enum types are handled
specially as described below. The FH type system associates with each
type a number of procedures. One of these is the printer procedure
which provided the association of type with output seen in the demo above.
</p>
<p>One of the challenges in automating C-Scheme type conversion is that C
code uses a lot of pointers. So as the FH generates types for
aggregates, it will automatically generate types for associated
pointers. For example, in the case above with <code class="code">foo_t</code> the FH will
generate an aggregate type named <code class="code">foo_t</code> and a pointer type named
<code class="code">foo_t*</code>. In addition the FH generates code to link these two
together so that, given an object <code class="code">f1</code> of type <code class="code">foo_t</code>, the
expression <code class="code">(pointer-to f1)</code> will generate an object of type
<code class="code">foo_t*</code>. This makes the task of generating an object value in
Scheme, and then passing the pointer to that value as an argument to a
FFI-generated procedure, easy. The inverse operation <code class="code">value-at</code>
is also provided. Note that sometimes the C code needs to work with
pointer pointer types. The FH does not produce double-pointers and in
that case, the user must add code to the FH module defintion to
support the required additional type (e.g., <code class="code">foo_t**</code>).
</p>
<p>In addition, the FH type system provide unwrap and wrap procedures
used internal to ffi-generated modules for function calls. These
convert FH types to and from objects of type expected by Guile’s FFI
interface. For example, the unwrap procedure associated with the FH
pointer type <code class="code">foo_t*</code> will convert an <code class="code">foo_t*</code> object to a
Guile <code class="code">pointer</code>. Similarly, on return the wrap procedure are
applied to convert to FH types. When the FH generates a type, for
example <code class="code">foo_t</code> it also generates an exported procedure
<code class="code">make-foo_t</code> that users can use to build an object of that type.
The FH also generates a predicate <code class="code">foo_t?</code> to determine if an
object is of that type. The <code class="code">(system ffi-help-rt)</code> module
provides a procedure <code class="code">fh-object-ref</code> to convert an object of type
<code class="code">foo_t</code> to the underlying bytestructures representation. For
numeric and pointer types, this will generate a number and for
aggregate types, a bytestructure. Additional arguments to
<code class="code">fh-object-ref</code> for aggregates work as with the bytestructures
package and enable selection of components of the aggregate. Note
that the underlying type for a bytestructure pointer is an integer.
</p>
<p>Enums are handled specially. In C, enums are represented by integers.
The FH does not generate types for C enums or C enum
typedefs. Instead, the FH defines unwrap and wrap procedures to
convert Scheme values to and from integers, where the Scheme values
can be integers or symbols. For example, if, in C, the enum typedef
<code class="code">baz_t</code> has element <code class="code">OPTION_A</code> with value 1, a procedure
expecting an argument of type <code class="code">baz_t</code> will accept the symbol
<code class="code">'OPTION_A</code> or the integer <code class="code">1</code>.
</p>
<p>Where the FH generates types, the underlying representation is a
<em class="emph">bytestructure descriptor</em>. That is, the FH types are
essentially a layer on top of a bytestructure. The layer provides
identification seen at the Guile REPL, unwrap and wrap procedures
which are used in function handling (not normally visible to the user)
and procedures to convert types to and from pointier-types.
</p>
<p>For base types (e.g., <code class="code">int</code>, <code class="code">double</code>) the FH uses the
associated Scheme values or the associated bytestructures values.
(I think this is all bytestructure values now.)
</p>
<p>The underlying representation of bytestructure values is
<em class="emph">bytevectors</em>. See the Guile Reference Manual for more
information on this datatype.
</p>
<p>The following routines are user-level procedures provided by the
runtime module <code class="code">(system ffi-help-rt)</code>:
</p><dl class="table">
<dt><code class="code">fh-type?</code></dt>
<dd><p>a predicate to indicate whether an object is a FH type
</p></dd>
<dt><code class="code">fh-object?</code></dt>
<dd><p>a predicate to indicate whether an object is a FH object
</p></dd>
<dt><code class="code">fh-object-val</code></dt>
<dd><p>the underlying bytestructure value
</p></dd>
<dt><code class="code">fh-object-ref</code></dt>
<dd><p>a procedure that works like <code class="code">bytestructure-ref</code> on the underlying
object
</p></dd>
<dt><code class="code">fh-object-set!</code></dt>
<dd><p>a procedure that works like <code class="code">bytestructure-set!</code> on the underlying
object
</p></dd>
<dt><code class="code">pointer-to</code></dt>
<dd><p>a procedure, given a FH object, or a bytestructure, that returns an
associated pointer object (i.e., a pointer type whose object value is
the address of the underlying argument); this may be a FH type or a
bytestructure
</p></dd>
<dt><code class="code">value-at</code></dt>
<dd><p>a procedure to dereference an object
</p></dd>
<dt><code class="code">fh-cast</code></dt>
<dd><p>a procedure to cast arguments for varaidic C functions
</p></dd>
<dt><code class="code">make-<i class="i">type</i></code></dt>
<dd><p>make base type, as listed below; also used to make bytestructure
objects for base types (e.g., <code class="code">(make-double)</code> for <code class="code">double</code>)
</p></dd>
</dl>
<p>Supported base types are
</p><table class="multitable">
<tbody><tr><td width="25%">short</td><td width="25%">unsigned-short</td><td width="25%">int</td><td width="25%">unsigned</td></tr>
<tr><td width="25%">long</td><td width="25%">unsigned-long</td><td width="25%">float</td><td width="25%">double</td></tr>
<tr><td width="25%">size_t</td><td width="25%">ssize_t</td><td width="25%">intptr_t</td><td width="25%">uintptr_t</td></tr>
<tr><td width="25%">ptrdiff_t</td></tr>
<tr><td width="25%">int8</td><td width="25%">uint8</td><td width="25%">int16</td><td width="25%">uint16</td></tr>
<tr><td width="25%">int32</td><td width="25%">uint32</td><td width="25%">int64</td><td width="25%">uint64</td></tr>
</tbody>
</table>
<p>These types are useful for cases where the corresponding types are
passed by reference as return types. For example
</p><div class="example lisp">
<pre class="lisp-preformatted">(let ((name (make-char*)))
(some_function (pointer-to name))
(display "name: ") (display (char*->string name)) (newline))
(let ((return-val (make-double)))
(another_function (pointer-to return-val))
(simple-format #t "val is ~S\n" (fh-object-ref return-val)))
</pre></div>
<p>You can pass a bytestructure struct value:
</p><div class="example">
<pre class="example-preformatted">guile> (make-ENTRY `((key 0) (data 0)))
#<ENTRY 0x18a10b0>
</pre></div>
<p>TODO: should we support <code class="code">(make-ENTRY 0 0)</code> ?
</p>
<h3 class="heading" id="Creating-FFI-Modules-with-_0028nyacc-lang-c99-ffi_002dhelp_0029">Creating FFI Modules with <code class="code">(nyacc lang c99 ffi-help)</code></h3>
<div class="example">
<pre class="example-preformatted">(define ffi-module <var class="var">module-name</var> ...)
</pre></div>
<dl class="table">
<dt><code class="code">#:pkg-config</code></dt>
<dd><p>This option take a single string argument which provides the name used
for the <em class="emph">pkg-config</em> program. Try <code class="code">man pkg-config</code>.
</p></dd>
<dt><code class="code">#:include</code></dt>
<dd><p>This form, with expression argument, indicates the list of include
files to be processed at the top level. Without use of the
<code class="code">#:inc-filter</code> form, only declarations in these files will be
output. To constrain the set of declarations output use the
<code class="code">#:decl-filter</code> form.
</p></dd>
<dt><code class="code">#:inc-filter</code></dt>
<dd><p>This form, with predicate procedure argument taking the form
<code class="code">(proc file-spec path-spec)</code>, is used to indicate which includes
beyond the top-level should have processed declarations emitted in the
output. The <code class="code">file-spec</code> argument is a string as parsed from
<code class="code">#include</code> statements in the C code, including brackets or double
quotes (e.g., <code class="code">"<stdio.h>"</code>, <code class="code">"\"foo.h\""</code>). The
<code class="code">path-spec</code> is the full path to the file.
</p></dd>
<dt><code class="code">#:use-ffi-module</code></dt>
<dd><p>This form, with literal module-type argument (e.g., <code class="code">(ffi
glib)</code>), indicates dependency on declarations from another processed
ffi module. For example, the ffi-module for <code class="code">(ffi gobject)</code>
includes the form <code class="code">#:use-ffi-module (ffi glib)</code>.
</p></dd>
<dt><code class="code">#:decl-filter</code></dt>
<dd><p>This form, with a predicate procedure argument, is used to restrict
which declarations should be processed for output. The single
argument is either a string or a pair. The string form is used for
simple identifiers and the pair is used for struct, union and enum
forms from the C code (e.g., <code class="code">(struct . "foo")</code>).
</p></dd>
<dt><code class="code">#:library</code></dt>
<dd><p>This form, with a list of strings, indicates which (shared object)
libraries need to be loaded. The formmat of each string in the list
should be as provided to the <code class="code">dynamic-link</code> form in Guile.
</p></dd>
<dt><code class="code">#:renamer</code></dt>
<dd><p>todo
</p></dd>
<dt><code class="code">#:cpp-defs</code></dt>
<dd><p>This form, with a list of strings, provides extra C preprodessor
definitions to be used in processing the header files. The defines
take the form <code class="code">"SYM=</code><var class="var">val</var><code class="code">"</code>.
</p></dd>
<dt><code class="code">#:inc-dirs</code></dt>
<dd><p>This form, with a list of strings, provides extra directories
in which to search for include files.
</p></dd>
<dt><code class="code">#:inc-help</code></dt>
<dd><p>todo
</p></dd>
<dt><code class="code">#:api-code</code></dt>
<dd><p>todo
</p></dd>
<dt><code class="code">#:def-keepers</code></dt>
<dd><p>This form, with a list of strings, provides extra (non-function) C
preprocessor macro definitions that should be included in the output.
</p></dd>
</dl>
<div class="example">
<pre class="example-preformatted"> #:library '("libcairo" "libmisc")
#:inc-dirs '("/opt/local/include/cairo" "/opt/local/include")
#:renamer (string-renamer
(lambda (n)
(if (string=? "cairo" (substring n 0 5)) n
(string-append "cairo-" n))))
#:pkg-config "cairo"
#:include '("cairo.h" "cairo-svg.h")
#:inc-help (cond
((string-contains %host-type "darwin")
'(("__builtin" "__builtin_va_list=void*")
("sys/cdefs.h" "__DARWIN_ALIAS(X)=")))
(else '()))
#:decl-filter (string-member-proc
"cairo_t" "cairo_status_t" "cairo_surface_t"
"cairo_create" "cairo_svg_surface_create"
"cairo_destroy" "cairo_surface_destroy")
#:export (make-cairo-unit-matrix)
</pre></div>
<p>Another decl-filter, useful for debugging.
</p><div class="example">
<pre class="example-preformatted"> #:decl-filter (lambda (k)
(cond
((member k '(
"cairo_t" "cairo_status_t"
"cairo_glyph_t" "cairo_path_data_t"
)) #t)
((equal? k '(union . "union-_cairo_glyph_t")) #t)
(else #f)))
</pre></div>
<h3 class="heading" id="Direct-Usage">Direct Usage</h3>
<p>Work to go here:
</p>
<dl class="first-deffn">
<dt class="deffn" id="index-load_002dinclude_002dfile"><span class="category-def">Procedure: </span><span><strong class="def-name">load-include-file</strong> <var class="def-var-arguments">filename [#pkg-config pkg]</var><a class="copiable-link" href='#index-load_002dinclude_002dfile'> ¶</a></span></dt>
<dd><p>This is the functionality that Ludo was asking for: to be at guile
prompt and be able to issue
</p><div class="example">
<pre class="example-preformatted">(use-modules (nyacc lang c99 ffi-help))
(load-include-file "cairo.h" #:pkg-config "cairo")
</pre></div>
</dd></dl>
<div class="example">
<pre class="example-preformatted">guile> ,use (nyacc lang c99 ffi-help)
guile> (load-include-file "cairo.h" #:pkg-config "cairo")
;; wait a while
guile> ...
</pre></div>
<h3 class="heading" id="Tuning-and-Debugging">Tuning and Debugging</h3>
<p>Since this is not all straightforward you will get errors.
</p>
<p>Method
</p><ol class="enumerate">
<li> compile-ffi with flag to echo declarations
</li><li> compile -O0 the resulting scm file
</li><li> guile -c ’(use-modules (ffi mymod))’
</li></ol>
<h4 class="subheading" id="MAX_005fHEAP_005fSECTS"><code class="code">MAX_HEAP_SECTS</code></h4>
<p>The message is
</p><blockquote class="quotation">
<p>Too many heap sections: Increase MAXHINCR or MAX_HEAP_SECTS
</p></blockquote>
<p>The message comes from the garbage collector. It means you’ve run out
of memory. I found that this actually came from a bug in the ff-compiler
which generated this code:
</p><div class="example">
<pre class="example-preformatted"> (bs:struct
(list ...
`(compose_buffer ,(bs:vector #f unsigned-int))
</pre></div>
<p>The original C declaration was
</p><div class="example">
<pre class="example-preformatted">struct _GtkIMContextSimple {
...
guint compose_buffer[7 + 1];
...
};
</pre></div>
<p>This bug, failure to evaluate <code class="code">7+1</code> to an integer, was fixed.
</p>
<h4 class="subheading" id="Trimming-Things-Down">Trimming Things Down</h4>
<p>After using the FFI Helper to provide code for some packages you may
notice that the quantity of code produced is large. For example, to
generate a guile interface for gtk2+, along with glib, gobject, pango
and gdk you will end up with over 100k lines of scm code. This may
seem bulky. Instead it may be preferable to generate a small number
of calls for gtk and work from there. In order to achieve this you
could use the <code class="code">#:api-code</code> or <code class="code">#:decl-filter</code> options.
</p>
<p>For example, in the expansion of the GLU/GL FFI module, called
<samp class="file">glugl.ffi</samp>, I found that a very large number of declarations
starting with <code class="code">PF</code> were being generated. I removed these using
the <code class="code">#:decl-filter</code> option:
</p><div class="example">
<pre class="example-preformatted">(define-ffi-module (ffi glugl)
#:include '("GL/gl.h" "GL/glu.h")
#:library '("libGLU" "libGL")
#:inc-filter (lambda (spec path) (string-contains path "GL/" 0))
#:decl-filter (lambda (n) (not (and (string? n) (string-prefix? "PF" n)))))
</pre></div>
<p>Using the option reduced <samp class="file">glugl.scm</samp> from 59,274 lines down to
15,354 lines.
</p>
<p>As another example, if we wanted to just generate code for the gtk hello
world demo we could write
</p><div class="example">
<pre class="example-preformatted">(define-ffi-module (hack1)
#:pkg-config "gtk+-2.0"
#:api-code "
#include <gtk2.h>
void gtk_init(int *argc, char ***argv);
void gtk_container_set_border_width(GtkContainer *container,
guint border_width);
void gtk_container_add(GtkContainer *container, GtkWidget *widget);
void gtk_widget_show(GtkWidget *widget);
void gtk_main(void);
")
</pre></div>
<p>Since the above example does not ask the FH to pull in typedef’s then
the pointer types will be expanded to native. You could invent your
own types or echo the typedefs from the package headers
</p><div class="example">
<pre class="example-preformatted">
</pre></div>
<h4 class="subheading" id="Warning_003a-Possibly-Unbound-Variable">Warning: Possibly Unbound Variable</h4>
<div class="example smallexample">
<pre class="example-preformatted">;;; ffi/gtk2+.scm:3564:5: warning:
possibly unbound variable `GtkEnumValue*'
;;; ffi/gtk2+.scm:3581:5: warning:
possibly unbound variable `GtkFlagValue*'
;;; ffi/gtk2+.scm:10717:11: warning:
possibly unbound variable `GtkAllocation*'
;;; ffi/gtk2+.scm:15107:15: warning:
possibly unbound variable `GdkNativeWindow'
;;; ffi/gtk2+.scm:15122:15: warning:
possibly unbound variable `GdkNativeWindow'
;;; ffi/gtk2+.scm:26522:11: warning:
possibly unbound variable `GSignalCMarshaller'
;;; ffi/gtk2+.scm:62440:11: warning:
possibly unbound variable `GdkNativeWindow'
;;; ffi/gtk2+.scm:62453:5: warning:
possibly unbound variable `GdkNativeWindow'
</pre></div>
<p>When I see this I check the scm file and see one of many things
</p><dl class="table">
<dt><code class="code">(fht-unwrap GtkAllocation*)</code></dt>
<dd><p>This usually means that <code class="code">GtkAllocation</code> was somehow defined
but not the pointer type.
</p></dd>
</dl>
<h4 class="subheading" id="Other">Other</h4>
<p>User is responsible for calling string->pointer and pointer->string.
</p>
<p>By definition: wrap is c->scm; unwrap is scm->c.
</p>
<p><code class="code">define-ffi-module</code> options:
</p><dl class="table">
<dt><code class="code">#:decl-filter proc</code></dt>
<dd><p>proc is a prodicate taking a key of the form <code class="code">"name"</code>,
<code class="code">(struct . "name")</code>, <code class="code">(union . "name")</code> or <code class="code">(enum . "name")</code>.
</p></dd>
<dt><code class="code">#:inc-filter proc</code></dt>
<dt><code class="code">#:include expr</code></dt>
<dd><p>expr is string or list or procecure that evaluates to string or list
</p></dd>
<dt><code class="code">#:library expr</code></dt>
<dd><p>expr is string or list or procecure that evaluates to string or list
</p></dd>
<dt><code class="code">#:pkg-config string</code></dt>
<dt><code class="code">#:renamer proc</code></dt>
<dd><p>procdure
</p></dd>
</dl>
<p>Here are the type of hacks I need to parse inside <samp class="file">/usr/include</samp>
with NYACC’s C99 parser. There is no such thing as a working C standard.
</p><div class="example">
<pre class="example-preformatted">(define cpp-defs
(cond
((string-contains %host-type "darwin")
'("__GNUC__=6")
(remove (lambda (s)
(string-contains s "_ENVIRONMENT_MAC_OS_X_VERSION"))
(get-gcc-cpp-defs)))
(else '())))
(define fh-inc-dirs
(append
`(,(assq-ref %guile-build-info 'includedir) "/usr/include")
(get-gcc-inc-dirs)))
(define fh-inc-help
(cond
((string-contains %host-type "darwin")
'(("__builtin"
"__builtin_va_list=void*"
"__attribute__(X)="
"__inline=" "__inline__="
"__asm(X)=" "__asm__(X)="
"__has_include(X)=__has_include__(X)"
"__extension__="
"__signed=signed"
)))
(else
'(("__builtin"
"__builtin_va_list=void*" "__attribute__(X)="
"__inline=" "__inline__="
"__asm(X)=" "__asm__(X)="
"__has_include(X)=__has_include__(X)"
"__extension__="
)))))
</pre></div>
<h3 class="heading" id="The-Run_002dtime-Module-_0028system-ffi_002dhelp_002drt_0029">The Run-time Module <code class="code">(system ffi-help-rt)</code></h3>
<p>Here we provide details of the run-time support module.
</p>
<h3 class="heading" id="Work-to-Go">Work to Go</h3>
<dl class="table">
<dt>02</dt>
<dd><p>if need foo_t pointer then I gen wrapper for foo_t* but add
foo_t to *wrappers* so if I later run into need for foo_t may be prob
</p></dd>
<dt>03</dt>
<dd><p>allow user to specify #:renamer (lambda (n) "make_goo" => "make-goo")
</p></dd>
<dt>04</dt>
<dd><p>Now the hard part if we want to reference other ffi-modules for types
or other c-routines. Say ffi-module foo defines foo_t
now in ffi-module bar we want to reference, but redefine, foo_t
</p><div class="example">
<pre class="example-preformatted">(define-ffi-module (cairo cairo) ...)
(define-ffi-module (cairo cairo-svg) #:use-ffi-module (cairo cairo)
</pre></div>
</dd>
<dt>05</dt>
<dd><p>Should setters for <code class="code">bs:struct</code> enum fields check for symbolic
arg?
</p></dd>
<dt>06</dt>
<dd><p>Use guardians for <code class="code">cairo_destroy</code> and
<code class="code">cairo_surface_destroy</code>?
</p></dd>
<dt>07</dt>
<dd><p>What about vectors? If <code class="code">foo(foo_t x[]</code>,
</p><ol class="enumerate">
<li> user must make vector of foo_t
</li><li> ffi-module author should generate a make-foo_t-vector procedure
</li></ol>
</dd>
</dl>
<h4 class="subheading" id="Completed">Completed</h4>
<dl class="table">
<dt>01</dt>
<dd><div class="example">
<pre class="example-preformatted">enum-wrap 0 => 'CAIRO_STATUS_SUCCESS
enum-unwrap 'CAIRO_STATUS_SUCCESS => 0
</pre></div>
</dd>
</dl>
<h3 class="heading" id="Administrative-Items">Administrative Items</h3>
<h4 class="subheading" id="Installation">Installation</h4>
<div class="example">
<pre class="example-preformatted">./configure --prefix=xxx
make install
</pre></div>
<h4 class="subheading" id="Reporting-Bugs">Reporting Bugs</h4>
<p>Please report bugs by navigating with your browser to
‘<code class="indicateurl">https://savannah.nongnu.org/projects/nyacc</code>’ and select
the “Submit New” item under the “Bugs” menu. Alternatively,
ask on the Guile user’s mailing list <a class="email" href="mailto:guile-user@gnu.org">guile-user@gnu.org</a>.
</p>
<h4 class="subheading" id="Notes">Notes</h4>
<ol class="enumerate">
<li> The following situation is a bit tricky for me.
<div class="example">
<pre class="example-preformatted">typedef struct foo foo_t;
typedef foo_t bar_t;
struct foo { int a; };
int baz(foo_t *x);
</pre></div>
<p>Right now, on the first declaration I assign <code class="code">foo_t</code> the type
<code class="code">fh-void</code>. The second declaration is handled as a type-alias.
When I get to the third declaration I define the
<code class="code">struct foo</code> compound type, then re-define the <code class="code">foo_t</code> as
a compound type, and it’s pointer type (missed this first time).
</p></li></ol>
<h4 class="subheading" id="Copyright">Copyright</h4>
<p>Copyright (C) 2017-2019 – Matthew R. Wette.
</p>
<p>Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
copy of the license is included with the distribution as COPYING.DOC.
</p>
</div>
</body>
</html>
|
