1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
|
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>GXemul: Miscellaneous</title>
<meta name="robots" content="noarchive,nofollow,noindex">
</head>
<body style="font-family : sans-serif;">
<!-- 10 lines header. -->
<h1>GXemul: Miscellaneous</h1>
<p>
<a href="./">Back to the index.</a>
<!--
Copyright (C) 2003-2021 Anders Gavare. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
-->
<p><br>
<ul>
<li><a href="#accuracy">Emulation accuracy</a>
<li><a href="#devel">Writing operating system code, or
developing firmware, using GXemul</a>
<li><a href="#compilercontruct">Using GXemul in compiler contruction courses</a>
<li><a href="#disk">How to start the emulator with a disk image</a>
<li><a href="#tape_images">How to start the emulator with tape images</a>
<li><a href="#disk_overlays">How to use disk image overlays</a>
<li><a href="#filexfer">Transfering files to/from the guest OS</a>
<li><a href="#largeimages">How to extract large gzipped disk images</a>
<li><a href="#promdump">Using a PROM dump from a real machine</a>
</ul>
<p><br>
<a name="accuracy"></a>
<h3>Emulation accuracy:</h3>
<p>GXemul is an instruction-level emulator; things that would happen in
several steps within a real CPU are not taken into account (e.g. pipe-line
stalls or out-of-order execution). Still, instruction-level accuracy seems
to be enough to be able to run complete guest operating systems inside the
emulator.
<p>The existance of instruction and data caches is "faked" to let
operating systems think that they are there, but for all practical
purposes, these caches are non-working.
<p>The emulator is in general <i>not</i> timing-accurate, neither at the
instruction level nor on any higher level. An attempt is made to let
emulated clocks run at the same speed as the host (i.e. an emulated timer
running at 100 Hz will interrupt around 100 times per real second), but
since the host speed may vary, e.g. because of other running processes,
there is no guarantee as to how many instructions will be executed in
each of these 100 Hz cycles.
<p>If the host is very slow, the emulated clocks might even lag behind
the real-world clock.
<p><br>
<a name="devel"></a>
<h3>Writing operating system code, or developing firmware, using GXemul:</h3>
<p>Is this a good idea? The answer is yes and no, depending on the level of
detail you need in your simulations. Important things to keep in mind are:
<ul>
<li>Porting code to a specific machine mode, e.g. a Silicon Graphics
machine, using GXemul, will not "magically" cause the code to
work on a real machine. You still need to test it on real hardware.
Sometimes code works in GXemul which doesn't
work on real hardware, sometimes it's the other way around.
<p>
<li>GXemul contains bugs, and many things are not yet implemented.
<p>
<li><b>Very important!</b> I have only implemented devices in GXemul
to the degree that NetBSD, OpenBSD, Linux, etc don't complain too much.
<p>
If you are developing a driver for a device which is emulated by
GXemul, and your driver does not seem to be working, it could just
as well be that the fault is in GXemul's implementation of
the device, and not a bug in your driver.
<p>
The device implementations in GXemul are mostly based on the assumption
that the emulated OS is already developed and bug-free. They are
not primarily intended to be used for development of new device
driver code in operating systems, so if you do that, then be
prepared for bugs and inconsitencies, and (as mentioned before) make
sure to also run your code or real hardware.
<p>
<li>CPU details in GXemul are sometimes wrong. If your code depends
on specific details of a particular CPU implementation, chances
area that GXemul will not be sufficient. One example is different
revisions of ISAs: some instructions which should trigger an exception
on a real MIPS processor may execute anyway in GXemul.
<p>
<li>Caches. There is no cache emulation in GXemul right now. Caches
for R2000/R3000 are faked well enough to run NetBSD, Ultrix, etc
in the <a href="http://en.wikipedia.org/wiki/DECstation">DECstation</a>
emulation mode, but other than that, cache operations are treated as
nops. So for example after you have loaded machine code into RAM in
order to execute it, you probably need to flush/clear the instruction
cache before branching to the code. A failure to flush the instruction
cache would likely result in buggy/random behavior on real hardware
whereas the code would still run in GXemul.
</ul>
<p>The bottom line is that GXemul can be useful as a <i>complement</i> to testing
your code on real hardware, but it should not be fully relied on.
<p><br>
<a name="compilercontruct"></a>
<h3>Using GXemul in compiler contruction courses:</h3>
<p>If you are learning how to write a compiler, and wish to target a
realistic target platform, then MIPS or ARM (as emulated by GXemul)
might be suitable choices.
<p>Your compiler needs to output real assembly
language code, which the assembler (e.g. gas, the GNU assembler) can
then compile into object format, and then you need to link this
into an executable image. This is much closer to how things work
in real life than running assembly language listings in a simulator
(e.g. <a href="http://spimsimulator.sourceforge.net/">SPIM</a>).
<p>
However, GXemul does not simulate
<a href="http://en.wikipedia.org/wiki/Out-of-order_execution">out-of-order
execution</a>, penalties related to instruction scheduling, or
load-delays, so it cannot be used to evaluate optimizing compilers
that take advantage of such processor features. GXemul keeps
track of the number of instructions executed, but that's it.
<p><br>
<a name="disk"></a>
<h3>How to start the emulator with a disk image:</h3>
<p>Add <tt>-d [prefixes:]diskimagefilename</tt> to the command line, where prefixes
are one or more single-character options. Run <tt>gxemul -h</tt>
to get a list of possible options.
<p>Here are some examples. If you want to run a <a href="http://www.netbsd.org/ports/pmax/">NetBSD/pmax</a>
kernel on an emulated <a href="http://en.wikipedia.org/wiki/DECstation">DECstation</a>
machine, you would use a command line such as this:
<pre>
$ gxemul -e 3max <b>-d pmax_diskimage.fs</b> netbsd-pmax-INSTALL
</pre>
<p>NOTE: For some emulation modes, such as the DECstation mode, you do
not actually have to specify the name of the kernel, if the disk
image is bootable! Bootblocks are then read from the disk and executed,
as on a real machine.
<p>It is possible to have more than one disk. For each -d argument, a disk
image is added; the first will usually (*) be SCSI target 0, the second will be
target 1, and so on, unless you specify explicitly which ID number the devices should have.
<pre>
$ gxemul -e 3max <b>-d disk0.raw -d disk1.raw -d 5:disk2.raw</b> netbsd-pmax-INSTALL
</pre>
<p>Note: In the example above, disk2.raw will get scsi id 5.
<p><small>(*) = One exception is that in the LUNA88K machine, if no id is specified,
the first disk image gets id 6, the second id 5 and so on.</small>
<p>If a filename has a 'c' prefix, or ends with ".iso", then it is assumed to be
a CDROM device (this can be overridden with a 'd' prefix, to force a read/write disk).
For example, the following command would start the emulator with two
CDROM images, and one harddisk image:
<pre>
$ gxemul -e 3max <b>-d image.iso -d disk0.img -d c:second_cdrom.img</b> netbsd-pmax-INSTALL
</pre>
<p>Usually, the device with the lowest id becomes the boot device. To override
this, add a 'b' prefix to one of the devices:
<pre>
$ gxemul -e 3max <b>-d rootdisk.img -d bc:install-cd.iso</b> name_of_kernel
</pre>
<p>If you have a physical CD-ROM drive on the host machine, say /dev/cd0c, you can
use it as a CD-ROM directly accessible from within the emulator:
<pre>
$ gxemul -e 3max <b>-d rootdisk.img -d bc:/dev/cd0c</b> name_of_kernel
</pre>
<p>It is probably possible to use harddisks as well this way, but I would not
recommend it.
<p><br>
<a name="tape_images"></a>
<h3>How to start the emulator with tape images:</h3>
<p>Using emulated tape drives is a bit more complicated than disks, because a
tape can be made up of several "files" with space in between. The solution
I have choosen is to have one file in the host's file system space for each
tape file. The prefix for using tapes is 't', and the filename given is
for the <i>first</i> file on that tape (number zero, implicitly). For
files following file nr 0, a dot and the filenumber is appended to the
filename.
<p>
As an example, starting the emulator with
<pre>
<b>-d t4:mytape.img</b>
</pre>
<p>will cause SCSI id 4 to be a tape device, using the following file number
to name translation scheme:
<p>
<center>
<table border="0">
<tr>
<td><b>File number:</b></td>
<td><b>File name in the host's filesystem:</b></td>
</tr>
<tr>
<td align="center">0</td>
<td align="left">mytape.img</td>
</tr>
<tr>
<td align="center">1</td>
<td align="left">mytape.img.1</td>
</tr>
<tr>
<td align="center">2</td>
<td align="left">mytape.img.2</td>
</tr>
<tr>
<td align="center">..</td>
<td align="left">..</td>
</tr>
</table>
</center>
<p>
If you already have a number of tape files, which should be placed on the
same emulated tape, then you might not want to rename all those files.
Use symbolic links instead (ln -s).
<p>
There is another advantage to using symbolic links for tape filenames:
every time a tape is rewound, it is reopened using the filename given
on the command line. By changing what the symbolic name points to,
you can "switch tapes" without quiting and restarting the emulator.
<p>
<font color="#ff0000">Note: Tape support is most likely very buggy,
because it has not been tested much, and has probably also suffered
from bit-rot by now.</font>
<p><br>
<a name="disk_overlays"></a>
<h3>How to use disk image overlays:</h3>
<p>This is most likely best understood by an example:
<p><ul>
<li>Install e.g. <a href="machine_cats.html#netbsdcatsinstall">NetBSD/cats</a>.
You will end up with a disk image called
<tt>nbsd_cats.img</tt>.
<p>
<li>Running the following command will boot straight from the disk
image, with no overlay images:<pre>
<b>gxemul -XEcats -d nbsd_cats.img netbsd.aout-GENERIC.gz</b>
</pre>
<li>You may now create an overlay file, a corresponding map file,
and start the emulator with the overlay image connected to
the same (explicit) ID as the base disk image:<pre>
<b>touch overlay.img overlay.img.map
gxemul -XEcats -d 0:nbsd_cats.img -d V0:overlay.img netbsd.aout-GENERIC.gz</b>
</pre>
<li>Any changes to the filesystem you perform when using the overlay
will only be written to that overlay. For example, to perform
a "roll back", you can do the following:<pre>
<b>rm -f overlay.img overlay.img.map
touch overlay.img overlay.img.map</b>
</pre>
and then simply start the emulator again, with the newly created
overlay image.
</ul>
<p>It is also possible to add multiple overlays. In that case, writes
always go the the <i>last</i> added overlay.
<p>GXemul uses Unix' way of supporting files with "holes",
so even if <tt>ls -l overlay.img</tt> says that the overlay is several
gigabytes large, <tt>du overlay.img</tt> should reveal that only the
blocks that have actually been written to have been stored in the
overlay, e.g.:<pre>
<b>$ ls -l
..
-rw-r--r-- 1 debug wheel 3072319488 Mar 24 11:59 nbsd_cats.img
-rw-r--r-- 1 debug wheel 2465354 Mar 24 11:44 netbsd.aout-GENERIC.gz
-rw-r--r-- 1 debug wheel 2930841600 Mar 24 14:02 overlay.img
-rw-r--r-- 1 debug wheel 715538 Mar 24 14:02 overlay.img.map
$ du overlay.img
864 overlay.img
</b>
</pre>
<p>The .map file is simply a raw bitmap telling which blocks of the
overlay file that are in use.
<p>As a short-hand for creating overlay files, running the emulator once
with those overlay files, and then immediately removing the overlay files,
the disk image prefix R (uppercase) can be used instead.
In other words, instead of typing something like:<pre>
<font color="#4040ff"><b>touch overlay.img overlay.img.map</b>
gxemul -e ..... -d disk.img <b>-d V0:overlay.img</b>
<b>rm overlay.img overlay.img.map</b></font>
</pre>
one would type:<pre>
<font color="#4040ff">gxemul -e ..... -d <b>R:</b>disk.img</font>
</pre>
<p><br>
<a name="filexfer"></a>
<h3>Transfering files to/from the guest OS:</h3>
<p>If the emulated machine supports networking (see <a
href="networking.html#intro">this section</a> for more info), then the easiest
way to transfer files is probably to use FTP or similar methods.
<p>There is another way of transfering files which works for any kind of
emulated machine which supports disks (either SCSI or IDE). Any file can
be supplied as a disk image. For example, consider the following:<pre>
$ <b>gxemul -XEcats -d nbsd_cats.img -d archive.tar.gz netbsd-GENERIC</b>
</pre>
<p>This will start NetBSD/cats with <tt>nbsd_cats.img</tt> as IDE master on
controller 0 (wd0), and <tt>archive.tar.gz</tt> as IDE slave on controller
0 (wd1). From inside NetBSD, it is now possible to extract the files using
the following command:<pre>
(inside emulated NetBSD/cats)
# <b>tar zxvf /dev/wd1c</b>
</pre>
<p>Don't worry if NetBSD complains about lack of disklabel; it doesn't
matter. On some machines, NetBSD uses <tt>wd1d</tt> instead of
<tt>wd1c</tt> for the entire disk.
There is also a minor problem: reading the end of the disk image. If you
experience problems untaring archives like this, then pad out the archive
first with some zeroes.
<p>Transfering files <i>out</i> from the emulated operating system to the
host can be done the same way. First, prepare an empty archive file:<pre>
$ <b>dd if=/dev/zero of=newarchive.tar bs=1024 count=1 seek=10000</b>
</pre>
<p>This example created a 10 MB empty file. Then, start the emulator
like this:<pre>
$ <b>gxemul -XEcats -d nbsd_cats.img -d archive.tar netbsd-GENERIC</b>
</pre>
<p>and transfer files by creating an archive directly onto the disk image:<pre>
(inside emulated NetBSD/cats)
# <b>tar cvf /dev/wd1c filenames</b>
</pre>
<p>where filenames are the files or directories to transfer.
<p><br>
<a name="largeimages"></a>
<h3>How to extract large gzipped disk images:</h3>
<p>Unix filesystems usually support large files with "holes". Holes are
zero-filled blocks that don't actually exist on disk. This is very
practical for emulated disk images, as it is possible to create a very
large disk image without using up much space at all.
<p>
Using gzip and gunzip on disk images can be <i>very</i> slow, as these
files can be multiple gigabytes large, but this is usually necessary for
transfering disk images over the internet. If you receive a gzipped disk
image, say disk.img.gz, and run a naive
<p>
<pre>
$ <b>gunzip disk.img.gz</b>
</pre>
<p>
on it, you will not end up with an optimized file unless
gunzip supports that. (In my experiments, it doesn't.) In plain English,
if you type <b>ls -l</b> and the filesize is 9 GB, it will actually occupy
9 GB of disk space! This is often unacceptable.
<p>
Using a simple tool which only writes blocks that are non-zero, a lot of
space can be saved. Compile the program cp_removeblocks in the
experiments/ directory, and type:
<p>
<pre>
$ <b>gunzip -c disk.img.gz | cp_removeblocks /dev/stdin disk.img</b>
</pre>
<p>
This will give you a disk.img which looks like it is 9 GB, and works like
the real file, but the holes are not written out to the disk. (You can see
this by running for example <b>du disk.img</b> to see the physical block
count.)
<p><br>
<a name="promdump"></a>
<h3>Using a PROM dump from a real machine:</h3>
<p>In GXemul, a simple PROM/BIOS implementation is usually faked, so that
guest operating systems can start up. For example, if the PROM has services
which the guest OS can call, which tell how much memory there is in a
machine, or to print simple characters to a terminal, those can be
implemented in software without having to run the original PROM image from
a physical machine.
<p>
Raw PROM images from real machines can, in a few cases, be used in
the emulator. A few things are worth keeping in mind, though:
<ol>
<li>ROM code is usually much more sensitive to correctness
of the emulator than operating system kernels or userland programs
are, so don't expect any PROM image to just magically work.
<ul>
<li>In particular, for proprietary hardware, a guest OS such
as NetBSD or Linux may only use the parts of the hardware
that have been reverse-engineered, whereas PROM firmware
code will access other devices, or more device registers,
and assume that the hardware works to a larger degree than
NetBSD or Linux needs.
</ul>
<li>If you are running a <i>modified</i> ROM/firmware image in GXemul,
in order to see that it boots up and works, it will not automatically
mean that it will behave the same way when you put it back in real
hardware again.
<li>Most of the emulation modes of GXemul have been designed to
run without the need for a PROM image, for example the load order
(or mechanism) used to boot from a CDROM or SCSI disk, so running
with actual machine firmware might not really be supported, more
than for simple experiments.
</ol>
<p>
Useful command line options:
<p>
<ul>
<li><tt>-Q</tt> disables the software/builtin PROM emulation in GXemul.
<li><tt>-T</tt> stops the emulation in case the PROM tries to access
a memory range which does not exist. (Default behavior is to
ignore writes and return 0 on reads.)
<li><tt>-K</tt> attempts to drop you into the debugger prompt, instead
of quitting, on some errors.
<li><tt>-v</tt> enables verbose output. In particular, it shows the
output of any <tt>debug()</tt> calls in the code.
</ul>
<h4>Preparation:</h4>
<p>The ROM image first needs to be extracted from your real machine. There are
several ways to do this, and it depends on your available hardware, the
specifics of the machine in question, and how much time you have.
<p>
<ul>
<li>Hook up a serial console and dump using the PROM's own dump
command. On machines for which you need to use a serial console to
access anyway, this is probably the easiest. The syntax for the
dump commands vary between machines, and you often need to do some
kind of post-processing to convert the dumped data (e.g. in some
kind of hex dump text format) to a raw binary. Alternatively, you
may be able to run a small program on the machine, which dumps
suitable memory ranges to a serial port. There are some examples in
the <tt>experiments/</tt> subdirectory which may help with these
tasks.
<li>Use hardware to read the PROM chip(s) directly.
Not easy if you don't have such a hardware reader.
<li>Copy the PROM memory range into a file, from a running
operating system. You need a running OS, and it must
have access to the PROM memory range. NetBSD, for example,
doesn't allow that from userland, as far as I have understood,
but a modified NetBSD kernel could probably allow this.
<li>Dump the ROM contents "visually", by showing the individual bits on
a display. Then use e.g. a webcam to record the images, and write a
program which decodes them into plain data. This can be used for
example on the Dreamcast, if you do not want to build a serial cable,
and do not have a PROM chip reader.
</ul>
</body>
</html>
|
