\start
Date: Fri, 14 Feb 2003 09:15:42 -0500
From: Tim Daly
To: Martin Rubey
Subject:  Re: lattice graph program

Martin Rubey wrote:
> is this what you have in mind?

> > I've been messing with XREF a little more, and using graphviz and xref
> > I made a chart which maps the risch.lisp functions, as well as the
> > RZMAC and RATMAC files.  I doubt it's useful for much but I thought I'd
> > upload it in case anyone is interested.  One that is definitely useless
> > is a joint mapping I did of all the rat*.lisp files.  It's way too
> > complex to be useful, but does give you an appreciation for just how
> > complex this program really is.  I uploaded the postscript files in
> > case anyone is curious:
> 
> > http://maxima.sourceforge.net/risch.ps
> > http://maxima.sourceforge.net/rat_files_chart.ps
> 
> > CY

I don't know but I'll check into that. Thanks.


> BTW: two questions:
> 
> 1. Is there an estimate when there will be a first axiom (alpha :-) 
> release?

The first release that includes compiled algebra code will happen this
weekend. I've finished the remaining domains and, except for a few
that failed to compile, the whole interpreter/compiler/algebra code
will be available to developers. Once some developers have
successfully compiled it I'll turn my attention to releasing the same
codebase on savannah. I've been working on a private CVS so I can make
my initial mistakes there. Now that Axiom compiles algebra from
scratch it is becoming usable. Of course, "usable" is questionable as
it is nowhere near the version that NAG shipped. Much work needs to be
done both to fix up the algebra layer, to build the rest of the code,
to clean up the docs, etc. However, the algebra is the important part.

I've done almost no checking of the code at any level except that it
compiles and builds properly. There is a tremendous amount of work
just to remaining to check the code so don't assume that this is a
finished release in any sense.

> 
> 2. Why is axiom NOT part of the gnu project? Wouldn't you at least ask 
> for something similar as (from aldor)

The code was released under modified BSD. That makes it both free and
open source in the technical sense. The GPL license was not chosen because
of various issues. Some of the discussion about the license still exists
at the original axiom site (www.earthlink.net/~jgg964/axiom.html).

Aldor is under its own license. Contact www.aldor.org for details.
The Aldor software is a separate effort of which, so far, I'm not a
developer. 

> 
> If you distribute a modified form of either source or binary code (a) you
> must make the source form of these modification available to Aldor.org (b)
> you grant Aldor.org a royalty-free license to use, modify or redistribute
> your modifications without limitation, and (c) you represent that you are
> legally entitled to grant these rights and that you are not providing
> Aldor.org with any code that violates any law or breaches any contract.

I'm confused about the intent of the above parpagraph. Axiom source code
is (or will be) freely available to everyone. It isn't usable in the
current state as it is just a "box of parts". I'm automating the build
process so anyone can build a working system. This involves many (8, so far)
layers of code, some of it self-referencing and circular.

As the build procedures begin to work I'm releasing the code. Aldor
developers, as well as anyone else, are free to do anything with that
code including NOT making their changes available (per mBSD). I expect
to work closely with the Aldor group in the future. 

> 
> (if there was a choice, I would find it great if axiom would become gpl, 
> but presumably there are lots of legal issues I do not know about...)
> 
> Martin
> 

Axiom code is 30 years worth of work by about 100 people and went thru
two companies (IBM and NAG). Both have been extremely generous in open
sourcing the code. The choice of modified BSD makes the code as free as
it can be. 

\start
Date: 16 Feb 2003 21:33:20 +0100
From: David Mentre
To: list
Subject:  Failed to build latest axiom tree

Hello,

I have tried to build the latest source tree of axiom (from
tenkan.org). I've update the source tree on "Sun Feb 16 21:13:37 CET
2003".

After a cd in the new/new/ directory, I have launched a :

david@oops:~/pub/axiom-libre/axiom/new/new$ make SPAD=`pwd`/mnt/linux SPD=`pwd` 
 :
[ some output ]
 :
make[2]: Entering directory `/home/david/pub/axiom-libre/axiom/new/new/lsp'
2 building gcl-2.4.1
3 applying EXTRAS patch to h/386-linux.defs
/bin/sh: /home/david/pub/axiom-libre/axiom/new/new/zips/gcl.h.386-linux.defs.patch: No such file or directory
make[2]: *** [gcldir] Error 1
make[2]: Leaving directory `/home/david/pub/axiom-libre/axiom/new/new/lsp'

So the build fails for a missing patch. Apparently, this file is not in
the CVS repository :

david@oops:~/pub/axiom-libre/axiom/new/new$ find . -name '*.patch'
./zips/gcl.init_gcl.lsp.patch
./zips/noweb.modules.c.patch
./zips/noweb.modules.nw.patch
./zips/noweb.src.Makefile.nw.patch
./zips/noweb.src.Makefile.patch
./obj/noweb/src/c/noweb.modules.c.patch

Other patch files missing in the repository are
zips/gcl.h.386-linux.h.patch and gcl.unixport.makefile.patch.

BTW, I think there is a small mistake in new/new/Makefile.pamphlet :

--- Makefile.pamphlet	Sun Feb 16 18:58:48 2003
+++ Makefile.pamphlet.fix	Sun Feb 16 21:20:42 2003
@@ -584,7 +584,7 @@
 shipped system. For example, if we want to build a linux system
 the {\bf SPAD} variable should look like:
 
-    `pwd`/new/mnt/{\bf linux}
+    `pwd`/mnt/{\bf linux}
 
 From the {\bf SPAD} variable we look at the last directory name
 and create a version of Axiom for that system. The {\bf SYS}


I hope it helps,

\start
Date: Sun, 16 Feb 2003 15:59:38 -0500
From: Tim Daly
To: David Mentre
Subject: Re:  Failed to build latest axiom tree

David,

I'm in the process of uploading the files and testing the upload.
Unfortunately, this takes hours on my modem connection. I've uploaded
and downloaded the files, am now in the process of building from the
download. There are a few missing files (the zips subdir is missing
a few patches). I should be done later today and then I'll announce
that the changes are available.

\start
Date: Sun, 16 Feb 2003 17:13:58 -0500
From: Tim Daly
To: list
Subject: Re:  Failed to build latest axiom tree

> > I should be done later today and then I'll announce that the changes
> > are available.
> 
> Oops sorry. Later time I'll wait for your announcement.
> 
> Best regards,
> d.

No problem. I'm going to need someone to test this version anyway. 
The fact that *I* can build it doesn't mean anyone else can. I'm
testing the build on a clean machine just to make sure I don't
announce something totally broken (which, clearly, the current upload is).

While I compile it here I'll take the time to tell you what has changed.
The bulk of the work is in the src/algebra directory. There are about
350 files (.spad.pamphlet) that contain approx 1100 domains, packages,
and categories. The task has been to get the algebra to compile from
scratch (previously you required a compiled version of the algebra).
In order to do that I need to determine which domains are primitive
(depend on nothing) and which depend on each other. This lattice currently
contains about 900 of the files. You can see the lattice by looking at
src/algebra/Makefile.pamphlet.

On my 1Ghz machine it takes about 90 minutes to compile. Great effort
is being made in the Makefiles to ensure that once you've compiled
something no redundant work occurs next time you type "make". All
useful intermediate work is cached (splitting out pamphlets for source,
compiling source to lisp, etc). It might not seem worth it now as it
greatly complicates the Makefiles but I can assure you from experience
that the Makefile complexity pays for itself immediately.

Further progress is blocked by a bug. The bug causes an infinite
recursion during "has" category lookup. It is not clear where the bug
occurs but I suspect it was an optimization for CCL. I've been
ignoring it as long as possible so I can make progress on the rest of
the lattice but the time has come to hunt it down.

Nevertheless, the system is now capable of doing limited algebra.
For example, 1+1 works. That exercises a fair piece of the machinery.
The failing algebra is, unfortunately, scattered around the lattice
so it keeps the system from doing marvelous things. The symptom is a
message about being unable to load "FOO.o" which means that "FOO.o" is
one of the algebra files that won't compile yet.

My next task is to find and fix the infinite recursion bug and finish
building the algebra lattice and the associated databases. At that
point we will have a working algebra system. Assuming the bug gets
fixed tonight (ha) the rest of the lattice build should go smoothly
(ha^2). I'll do another upload at that point. Once other people can
get the full algebra code running it will be time to switch to using
the real website rather than tenkan and switch to really using CVS
rather than my current "bulk" method.

This build does not have any "user" packaging yet so the "axiom" command
does not yet really exist. If the build works you should be able to run
(yourpath)/obj/linux/bin/interpsys

I should caution you that, before you build, you need to change 3 things:

1) modify the (yourpath)/Makefile and change the SPD=(yourpath)
2) modify the (yourpaht)/Makefile.pamphlet and change the SPD=(yourpath)
3) export AXIOM=(yourpath)/mnt/linux

missing any of these three things will cause obscure bugs. I'll include
a note to that effect in the top level directory but I doubt it will get
read until after the fact :-)

I want to drive the first two changes off the AXIOM shell variable but
"make" seems to have changed since last I used it and my old trick fails.

Once this gets "announced" please let me know if you can compile it.

\start
Date: Sun, 16 Feb 2003 21:25:11 -0500
From: Tim Daly
To: list
Subject:  Axiom next release

The Feb 16, 2002 release has been uploaded to tenkan.
If you want developer access please let me know and
I'll arrange it with my serive provider.

Please make the following changes BEFORE you type "make"

1) do the CVS checkout into (pathname)
    The build tree starts at (path)/axiom/new/new due to a 
    mistake in the original CVS checkin
2) cd (pathname)/axiom/new/new
3) edit Makefile 
     change 
       SPD=/home/axiomgnu/new
     to
       SPD=(pathname)/axiom/new/new
4) edit Makefile.pamphlet
     change 
       SPD=/home/axiomgnu/new
     to
       SPD=(pathname)/axiom/new/new
5) export AXIOM=(pathname)/axiom/new/new/mnt/linux
6) type "make"
7) if the make succeeds you will be able to run axiom as:
     (pathname)/obj/linux/bin/interpsys
   
interpsys should correctly compute 1+1 

Note that there are still more algebra files to compile. 
Once all of the algebra compiles the plan is to move everything
to the axiom website on savannah and start using CVS for more
than bulk uploads.

Questions and comments should be sent to me.

\start
Date: Tue, 18 Feb 2003 4:24:24 -0500
From: Bill Page
To: list
Subject:  Re: Axiom next release

Tim,

It is great to see all the progress that you have been making!

I checked the lastest version out of tenkan at about 10:00 PM
last night and followed the instructions in you email. After
about 45 minutes of chugging along on my 1.6 GHz machine under
RedHat 8.0, I got the following error message:

  614 copying /home/axiom/new/new/src/algebra/INTERP.EXPOSED to /home/new/new/mnt/linux/algebra
  make[3]: *** No rule to make target '/home/axiom/new/new/src/algebra/exposed.lsp' needed by '/home/axiom/new/new/mnt/linux/algebra/exposed.o'. Stop.
  make[3]: Leaving directory '/home/axiom/new/new/src/interp'
  make[2]: *** [interpdir] Error 2
  ...

etc.

Can you see what might be wrong? It appears that the appropriate
clause may be in /home/axiom/new/new/src/interp/Makefile.pamphlet
but not in the Makefile itself?

I am using the following versions of gcc and make (straight out
of the RedHat 8.0 distribution):

  # gcc -v
  gcc version 3.2 20020903 (Red Hat Linux 8.0.3.2-7)
  # make -v
  GNU Make version 3.79.1, by Richard Stallman and Roland McGrath.

\start
Date: Tue, 18 Feb 2003 6:04:50 -0500
From: Bill Page
To: list
Subject:  Re: Axiom next release

Tim,

After a few more false starts, I found that my problem was
probably caused by that fact that LaTex was not found in the
path for the account I was using for the install. Fixing that
and trying again, I got stuck once inside LaTex but after killing
the LaTex process, things progressed into the infinite loop of
compilations that you mentioned earlier. Once I got tired of
that and interrupted the loop. I found that I could indeed start
interpsys and add 2+2. Hooray! But unfortunately not much else.

Anyway, it is progress.

I'll try it again later today with a clean installation - this
time making sure that I have LaTex first.

> Tim,
> 
> It is great to see all the progress that you have been making!
> 
> I checked the lastest version out of tenkan at about 10:00 PM
> last night and followed the instructions in you email. After
> about 45 minutes of chugging along on my 1.6 GHz machine under
> RedHat 8.0, I got the following error message:
> 
>   614 copying /home/axiom/new/new/src/algebra/INTERP.EXPOSED to /home/new/new/mnt/linux/algebra
>   make[3]: *** No rule to make target '/home/axiom/new/new/src/algebra/exposed.lsp' needed by '/home/axiom/new/new/mnt/linux/algebra/exposed.o'. Stop.
>   make[3]: Leaving directory '/home/axiom/new/new/src/interp'
>   make[2]: *** [interpdir] Error 2
>   ...
> 
> etc.
> 
> Can you see what might be wrong? It appears that the appropriate
> clause may be in /home/axiom/new/new/src/interp/Makefile.pamphlet
> but not in the Makefile itself?
> 
> I am using the following versions of gcc and make (straight out
> of the RedHat 8.0 distribution):
> 
>   # gcc -v
>   gcc version 3.2 20020903 (Red Hat Linux 8.0.3.2-7)
>   # make -v
>   GNU Make version 3.79.1, by Richard Stallman and Roland McGrath.

\start
Date: Tue, 18 Feb 2003 07:02:43 -0500
From: Tim Daly
To: Bill Page
Subject: Re:  Re: Axiom next release

Can't imagine how Latex was involved in that error.
Usually that error is because the file is missing.
INTERP.EXPOSED is a list of "exposed constructors".
The interpreter does not see every possible constructor
because it does not need to. INTERP.EXPOSED is one of the
databases I build. You'll notice several copies of it as
it is in the process of moving directories and I haven't
had the time to kill all possible old references. NAG built
a "share" directory and did not follow the rule that all
shipped files live under "mnt". I'm reworking it so that
will again be true.

I'm glad to see that you can do arithmetic. More is coming
but I'm blocked by a bug which I have not yet found. Yet 2+2
exercises a fair amount of the machinery and is an encouraging
sign.

For documentation purposes I'm using a program called "dia" to
build a graph of the lattice. It has helped me find a bug already.
I'm snowed in at home today so progress will be made :-)

The most useful thing we need is to try to port the system to
another platform. It is a huge job in general as some of the
later parts are in C. This part should work provided you can
get latex, noweb, and gcl to compile and have various tools
like make, patch, ar, cp, tar, etc available. The MKS toolkit
is ideal for windows but rumor has it that various tools have
been successfully ported. Please take a few notes about porting
as, hopefully, you won't be the only one to try to move the
system around. If you find that we need various packages from
the web let me know.

On the porting front HP has a deal where you can sign up to use
their computing farm for porting. I wrote about it in another
email in this group. I'm signed up and itching to play with the
64-bit version but I fear that isn't going to happen until my
mother has one in her toaster. You might want to check into it
if you want a real challenge.

If you want a simple place to start you can look at the issue of
documenting the algebra. It needs work at two levels. First, if
you look at the .spad.pamphlet files you'll see that they are TeX
documents. Each domain (classes with representation structures),
package (classes without representation) and category requires
a long explanation of its mathematics or computer science (for
data structures), its details (such as cross references to other
domains with \cite, specifics of the algorithm, references to
the literature, etc). Look at dhmatrix.spad.pamphlet for an example.

Second, if you look at the algebra compiles you'll see a raft-load of
messages about functions that are not documented or have improper
words in their documentation. There is an automated build of online
documentation that occurs from the so-called "plus-plus" (lines
beginning with ++) documentation (lines beginning with -- are
ignored). The ++ comments require a certain style using special
keywords which, of course, themselves are not documented. It appears
that these were never maintained once the system left my hands.

Or choose what interests you to work on. There is work needed
everywhere.

\start
Date: 18 Feb 2003 21:30:42 +0100
From: David Mentre
To: Tim Daly
Subject: Re:  Axiom next release

Hello tim and bill,

Tim Daly writes:

> 1) do the CVS checkout into (pathname)
>     The build tree starts at (path)/axiom/new/new due to a 
>     mistake in the original CVS checkin
> 2) cd (pathname)/axiom/new/new
> 3) edit Makefile 
>      change 
>        SPD=/home/axiomgnu/new
>      to
>        SPD=(pathname)/axiom/new/new
> 4) edit Makefile.pamphlet
>      change 
>        SPD=/home/axiomgnu/new
>      to
>        SPD=(pathname)/axiom/new/new
> 5) export AXIOM=(pathname)/axiom/new/new/mnt/linux

I have done the following steps but there are several remaining issues :

 (i) in new/new/src/interp/debugsys.lisp.pamphlet, the path to
 /home/axiomgnu/new is hard coded (and it is indeed written in the
 documentation! :) so it should be changed with (pathname)/axiom/new/new 

 (ii) as bill page, a latex is called ("/usr/bin/latex Makefile.tex"),
 but I don't know for which Makefile.tex (is it because I have modified
 the pamphlet? Probably). Anyway, this latex compilation does not seem
 to work. I had to kill the latex process twice.

> 6) type "make"

It is still compiling on my old celeron 500 :) so: to be continued...

\start
Date: 18 Feb 2003 22:42:59 +0100
From: David Mentre
To: Tim Daly
Subject: Re:  Axiom next release

David MENTRE writes:

> It is still compiling on my old celeron 500 :) so: to be continued...

After more than three hours of compilation... it works!

(AXIOM Sockets) The AXIOM server number is undefined.
-----------------------------------------------------------------------------
   Issue )copyright to view copyright notices.
   Issue )summary for a summary of useful system commands.
   Issue )quit to leave AXIOM and return to shell.
-----------------------------------------------------------------------------
 
(1) -> 1+1
   Loading 
      /home/david/pub/axiom-libre/axiom-i386/new/new/mnt/linux/algebra/PI.o
      for domain PositiveInteger 
   Loading 
      /home/david/pub/axiom-libre/axiom-i386/new/new/mnt/linux/algebra/NNI.o
      for domain NonNegativeInteger 
   Loading 
      /home/david/pub/axiom-libre/axiom-i386/new/new/mnt/linux/algebra/INT.o
      for domain Integer 

   Loading 
      /home/david/pub/axiom-libre/axiom-i386/new/new/mnt/linux/algebra/OUTFORM.o      for domain OutputForm 
   Loading 
      /home/david/pub/axiom-libre/axiom-i386/new/new/mnt/linux/algebra/LIST.o
      for domain List 
   (1)  2
                                                        Type: PositiveInteger

Great work Tim. Some few bugs to iron out but your are definitely on the
right path.

\start
Date: Tue, 18 Feb 2003 17:36:23 -0500
From: Tim Daly
To: David Mentre
Subject: Re:  Axiom next release

Excellent. I'll look into the Latex issue. That's the kind of thing
that I can't test because I build systems the same way every time.

\start
Date: Tue, 18 Feb 2003 18:06:50 -0500
From: Bill Page
To: David Mentre
Subject: Re:  Axiom next release

David,

Did the compilation complete on it's own (besides from
the latex processes that you had to kill)? Or did you
terminate the compilation after it became quite clear
that there was very likely some kind of rather long
cycle of apparently repeated re-compilations of the
same algebra modules?

In my case, I waited and watched the compilation process
for more than an hour after it appeared to be nearing
the last phase. I was looking for an obvious repitition.
I recalled that Tim had said that there probably still
remained an unbroken cycle of dependencies that caused
an infinite loop, so I decided to stop it rather than
wait it out. Overall the make process ran for about 2
hours on my 1.6 GHz linux box before I stopped it.

After stopping it, I tried starting interpsys as Tim
instructed in his earlier message. That's when I declared
success and went to get some sleep.

Tim, is this consistent with what you see when you do
the full compilation? Do you have any hints as to were
to look for this cycle of dependencies?

> David MENTRE writes:
> 
> > It is still compiling on my old celeron 500 :) so: to be continued...
> 
> After more than three hours of compilation... it works!
> 
> ...

\start
Date: Tue, 18 Feb 2003 18:26:26 -0500
From: Tim Daly
To: Bill Page
Subject: Re:  Axiom next release

David,

There can't be a cycle of dependencies at the level of the Makefile.
I compile every file each time it is added to the Makefile and again
after I've added a set. Each file is independent, except at the
pamphlet level. So there can't be cycles in the algebra compile.

Axiom is huge. You're only building a portion of it so far. 
Non-linux compiles (or linux compiles on a machine with not a lot
of free memory) are going to take hours. Linux caches a LOT of the
disk transfers where Windows does not. The SUN build used to 
reload interpsys from disk every time as well as all of the loaded
files. Each algebra file compile would take 2 minutes or more which
gets into big numbers when you compile thousands of files. I've seen
an Axiom build on a SUN box take 24 hours even though I'd already
cached all possible work (hence the religious use of the intermediate
directory). I never built a SUN version from scratch.

\start
Date: Wed, 19 Feb 2003 07:51:18 +0000
From: Mike Dewar
To: Tim Daly
Subject: Re:  Axiom next release
cc: Bill Page

Tim is right that compiling Axiom can take a long time but on a more
modern machine and using CCL (which is *much* faster than gcc because it
compiles to byte code and doesn't launch a C compilation for every file)
it was taking us maybe 1 or 2 hours to rebuild all the algebra.  This
didn't include the bootstrap process which you guys are presumably doing
or any extra LaTeXing that the new pamphlets require of course.

I'd strongly recommend using CCL for your development system if you can.
Not only is it the platform for the last NAG build but its smaller,
leaner and as I've said compiles code much faster.  Although in theory
akcl/gcl code runs faster in practice we found that you rarely got the
benefit because the image was so much bigger and as a result the Axiom
interpsys process swapped a lot more.  WIth CCL you can compile
performance-critical parts of the libraries into the kernel (which
roughly doubles the execution speed unless the functions are very small
in which case the incresed overhead of calling compiled functions
cancels this out), and as a result the first CCL-based version we
shipped ran our test suite faster than the previous AKCL version.  Of
course machines are bigger and faster today so this may not be such an
issue any more.

The biggest drawback with CCL was debugging the byte-code interpreter
where neither a source-level tool looking at the C or the lisp-level
tools gave us much of a clue as to what was going on. 

Well done getting integer arithmetic to work!  It sounds like you're
making good progress.

On Tue, Feb 18, 2003 at 06:26:26PM -0500, Tim Daly wrote:
> David,
> 
> There can't be a cycle of dependencies at the level of the Makefile.
> I compile every file each time it is added to the Makefile and again
> after I've added a set. Each file is independent, except at the
> pamphlet level. So there can't be cycles in the algebra compile.
> 
> Axiom is huge. You're only building a portion of it so far. 
> Non-linux compiles (or linux compiles on a machine with not a lot
> of free memory) are going to take hours. Linux caches a LOT of the
> disk transfers where Windows does not. The SUN build used to 
> reload interpsys from disk every time as well as all of the loaded
> files. Each algebra file compile would take 2 minutes or more which
> gets into big numbers when you compile thousands of files. I've seen
> an Axiom build on a SUN box take 24 hours even though I'd already
> cached all possible work (hence the religious use of the intermediate
> directory). I never built a SUN version from scratch.

\start
Date: Wed, 19 Feb 2003 07:09:23 -0500
From: Tim Daly
To: Mike Dewar
Subject:  CCL development
cc: Bill Page

Mike,

CCL is on the list of lisps to build and, indeed, was the first effort
I made. You'll notice that the CCL code is in pamphlet format. The
Makefile tree exists and will properly build a working CCL image. There
were a few problems I ran into with CCL. Overall I did like the speed
of compiles and the byte-codes are very small. It took me a while to
create an image but that eventually succeeded. There were some difficult
problems however.

First, I had no documentation of the language and it is clearly not
common lisp. System commands, such as "save-system" and memory management
commands (like setting heap and stack sizes) are the kind of things I
NEED to know. I know these commands with GCL as I helped define them.

Second, I couldn't figure out how to debug code. The "break loop"
commands were not obvious to me. There is no documentation on the
structure of the runtime image or of the byte codes. These don't
exist either for GCL but I spent years with it and know the GCL
code well.

Third, there didn't seem to be a way to generate "intermediate"
code.o files from the build process. This lack breaks the whole
makefile tree and means I need to deeply reengineer the process.

Fourth, Several of the shipped algebra files will not compile in the
NAG CCL image and I couldn't figure out how to find the failure.

Fifth, the CCL image hangs if you run it long enough and I couldn't
figure out why. It isn't swapping and it uses 99% of the cpu. (The
easiest way to see this is to fire up the NAG axiom image and compile
the algebra libraries). I tried to debug this also without success.

Sixth, Size isn't the issue (anymore). Actually, it never was. I had
Axiom running on a 16Meg laptop years ago. Portability is an issue and
I expect that CCL is going to be very useful there. Speed also is not
an issue. Schelter built a special mechanism for Axiom that deeply
optimizes function calling (the so-called .fn files). We had many
discussions about it. Camm has also extended GCL with a function that
allows me to see the assembler code generated by pieces of lisp so I
can see exactly what is expensive (a more useful disassemble output).

Seventh, Common Lisp has moved since last I played with it. For instance,
you need to do a make-package before doing an in-package. Previously
in-package would do an implied make-package. The boot and spad compilers
need to be updated to handle this, and other, changes. Axiom code will
not run on the latest version of common lisp. I have several people 
who have proposed projects that depend on common lisp and Axiom so
I can't abandon common lisp. Languages change and code needs to keep
up to live (I also have a C++ code pile I maintain that won't compile
in the latest C++ because the language changed. sigh). I'll have to
retrofit the new functions into CCL when they get generated by Axiom.

So CCL is a target lisp that I plan to support but the first two
months were spent trying to get things running without success. 
Almost every problem above falls into the category of "I don't know
how to ...." and I was losing huge amounts of time trying to reverse
engineer CCL without making progress on getting Axiom running. I
switched to GCL, originally for debugging, now for first release
because I know the code so well. Once the system development moves
into the developer's hands I'll revisit the CCL path. It clearly
is an excellent lisp and I just need to find the time to learn it
in intimate detail. Unfortunately, the learning time was on the
critical path to getting Axiom out the door, which is my first
priority, and I couldn't afford to delay the effort any longer.

\start
Date: Wed, 19 Feb 2003 12:59:58 +0000
From: Mike Dewar
To: Tim Daly
Subject:  Re: CCL development
cc: Bill Page, Mike Dewar

Hi Tim,

> First, I had no documentation of the language and it is clearly not
> common lisp. System commands, such as "save-system" and memory management
> commands (like setting heap and stack sizes) are the kind of things I
> NEED to know. I know these commands with GCL as I helped define them.
Its pretty close to Common Lisp but not a full implementation.  From
memory (I'm at a meeting in Italy on the end of a slow connection so
can't experiment) the heap size is a command line option, probably "-K".
However this turns off the default behaviour which is to grow the heap
as necessary so shouldn't be used except in special circumstances.  nAll
the command line options are handled in src-cslbase-csl.c.  The stack
size is fixed and I think depends on the system stack size (the single
most common Axiom bug report used to be that the stack blew out on some
machines like HPs where the system stack size was low - a problem we
never had with CCL).  

I apprecaite that you know AKCL inside out - we didn't and found CCL
more intuitive and also found Arthur very helpful.

> Second, I couldn't figure out how to debug code. The "break loop"
> commands were not obvious to me. There is no documentation on the
> structure of the runtime image or of the byte codes. These don't
> exist either for GCL but I spent years with it and know the GCL
> code well.
Debugging in CCL is poor, you basically have a back trace and thats it.
In Axiom commands can be found by going ")set break break <cr>1/0" in
Axiom and following the instructions.

> Third, there didn't seem to be a way to generate "intermediate"
> code.o files from the build process. This lack breaks the whole
> makefile tree and means I need to deeply reengineer the process.
In CCL code.o files make no sense so you can't build them.  You can
build "uxx.c" files which are C versions of lisp functions which you
link into the kernel.  This is documented (I think) in either the
caxbase or axbase src directories.
 
> Fourth, Several of the shipped algebra files will not compile in the
> NAG CCL image and I couldn't figure out how to find the failure.
I find this very surprising since they were rebuilt as part of our
product build cycle - this is probably a Linux problem.  Do you have an
example - if I can reproduce it in Oxford then I could try and talk you
through what I'd do to fix the problem?

> Fifth, the CCL image hangs if you run it long enough and I couldn't
> figure out why. It isn't swapping and it uses 99% of the cpu. (The
> easiest way to see this is to fire up the NAG axiom image and compile
> the algebra libraries). I tried to debug this also without success.
Again, I've never seen this and no user has reported such a bug.  It
sounds like a Linux problem again - bear in mind that we never ported
Axiom to anything newer than RedHat 6 and Linux releases tend to throw
up portability issues.
 
> Sixth, Size isn't the issue (anymore). Actually, it never was. I had
> Axiom running on a 16Meg laptop years ago. Portability is an issue and
Actually it is an issue - real-world computer algebra applications in
robotics and molecular modelling are too big to run in any existing
system.  

> I expect that CCL is going to be very useful there. Speed also is not
> an issue. Schelter built a special mechanism for Axiom that deeply
> optimizes function calling (the so-called .fn files). We had many
> discussions about it. Camm has also extended GCL with a function that
> allows me to see the assembler code generated by pieces of lisp so I
> can see exactly what is expensive (a more useful disassemble output).
Speed is an issue for some people - look at Faugere's work for example.
There are profiling facilities in CCL which you should know about - I'll
try and send you details when I'm back in Oxford.  
 
Somehow we need to ensure that all the useful but undocumented
information about Axiom and CCL stored in Themos and my brains is made
explicit, so please let me know about any problems you encounter.
Although I can't promise to be able to help I will certainly try.

\start
Date: Wed, 19 Feb 2003 14:09:04 +0100 (CET)
From: Bertfried Fauser
To: list
Subject:  compiling axiom/error

Dear Tim,

just a few minutes ago, I downloaded a new version from cvs and tried to
compile AXIOM on my SuSE 8.0 Linux. I got the following error after about
20 min of compilation time....

Finished loading /home/fauser/axiom/new/new/obj/linux/interp/makeint.lisp
6 /home/fauser/axiom/new/new/obj/linux/bin/interpsys created
39 making /home/fauser/axiom/new/new/int/interp/debugsys.lisp from
/home/fauser/axiom/new/new/src/interp/debugsys.lisp.pamphlet
7 building debugsys

>
Loading /home/fauser/axiom/new/new/int/interp/debugsys.lisp

Error: Cannot open the file /home/axiomgnu/new/int/interp/sys-pkg.lisp.
Error signalled by LOAD.
Broken at SYSTEM::BREAK-LEVEL.  Type :H for Help.
>>8 /home/fauser/axiom/new/new/obj/linux/bin/debugsys created
make[3]: Leaving directory
`/Net/clifford/home/fauser/axiom/new/new/src/interp'
14 making /home/fauser/axiom/new/new/src/algebra/Makefile from
/home/fauser/axiom/new/new/src/algebra/Makefile.pamphlet

Any hint, whlie this file is missing?

[Looks like if there is still a hard coded path, since I guess the file
should be searched in my local path /home/fauser/axiom/.... and not in
/home/axiomgnu/....

\start
Date: 19 Feb 2003 18:48:51 +0100
From: David Mentre
To: Bill Page
Subject: Re:  Axiom next release

Hi bill,

Bill Page writes:

> Did the compilation complete on it's own (besides from
> the latex processes that you had to kill)? 

Yes, I just killed the two latex processes.

> Or did you terminate the compilation after it became quite clear that
> there was very likely some kind of rather long cycle of apparently
> repeated re-compilations of the same algebra modules?

No, no interruption. But I would like to underline that I also modified
the new/new/src/interp/debugsys.lisp.pamphlet file (see my previous
email).

Next compile, I'll try to make a log of the compilation process.

\start
Date: 19 Feb 2003 18:55:10 +0100
From: David Mentre
To: Bertfried Fauser
Subject: Re:  compiling axiom/error

Bertfried Fauser writes:

> Any hint, whlie this file is missing?

See my previous message on this mailing list:
http://mail.nongnu.org/archive/html/axiom-developer/2003-02/msg00008.html

\start
Date: Thu, 20 Feb 2003 00:21:57 -0500
From: Tim Daly
To: Bertfried Fauser
Subject:  debugsys absolute pathnames

Bertfried Fauser wrote:
> >
> Loading /home/fauser/axiom/new/new/int/interp/debugsys.lisp
> 
> Error: Cannot open the file /home/axiomgnu/new/int/interp/sys-pkg.lisp.

After much headscratching I finally figured out what the problem was.
David's message was quite clear but I was suffering from "familiarity
blindness". I couldn't see why debugsys wouldn't compile. After all
it worked fine for me every time. I've tried hard to keep pathnames
out of the game but sometimes I can't see them even when I'm looking
at them. Sorry about that.

Debugsys is a new image that was never created before. It basically
loads all of the system as interpreted lisp code. It is useful if
you have to trace a bug all the way down to the last expression
which is what I'm doing at the moment. It is optional and should be
commented out of the build as I'm the only one who is likely to be
chasing bugs in such detail. 

\start
Date: Thu, 20 Feb 2003 00:26:21 -0500
From: Tim Daly
To: David Mentre
Subject:  latex processes
cc: Bill Page

Can you give me a clue about where the latex proceses are started?

Latex hangs if you have a syntax error. It puts up a console prompt.
In order to minimize the amount of console noise the latex output
is put in a file called ${TMP}/trace 
In order to find the hung process you can run the build using
  make NOISE=

This is documented in the Makefile.dvi in the root directory (see
FAQ item number 4).

\start
Date: Thu, 20 Feb 2003 00:33:25 -0500
From: Tim Daly
To: Mike Dewar
Subject: Re:  Re: CCL development
cc: Bill Page
 
> Somehow we need to ensure that all the useful but undocumented
> information about Axiom and CCL stored in Themos and my brains is made
> explicit, so please let me know about any problems you encounter.
> Although I can't promise to be able to help I will certainly try.

It would be most useful if you were to either create a new pamphlet file
(they are really just TeX files with <<foo>>= tags added). You can write
about anything you think is useful; possibly creating a developer's notes
document or some such. The whole idea of the pamphlet files is to capture
the knowledge so it survives, something we didn't do while I worked at IBM.
Since most of my effort is going into writing Makefiles at the moment 
the Makefile.pamphlet files contains most of the new documentation. However
every file in the system has the machinery wrapped around it to allow us
to document every detail from the large to the small. 

In fact, I'd encourage all of the developers (and users) to create 
pamphlet files.

\start
Date: Thu, 20 Feb 2003 00:39:16 -0500
From: Tim Daly
To: Mike Dewar
Subject: Re:  Re: CCL development
cc: Bill Page
 
> Somehow we need to ensure that all the useful but undocumented
> information about Axiom and CCL stored in Themos and my brains is made
> explicit, so please let me know about any problems you encounter.
> Although I can't promise to be able to help I will certainly try.

One particular problem I'm chasing is that algebra files that contain
lookups like:
        if S has SetCategory then SetCategory
fail by going into an infinite look in knownInfo (info.boot).
It appears that this code has been rewritten and no longer works in
common lisp although it must somehow work in CCL. knownInfo does in
fact find that (has s SetCategory) exists but then appears to call
itself again. The offending code appears to be:


    -- This line caused recursion on predicates which are no use in deciding
    -- whether a category was present.
    -- or/[AncestorP(cat,LIST CAR u) for u in CADR catlist | knownInfo CADR u] => true
    or/[AncestorP(cat,LIST CAR u) and knownInfo CADR u for u in CADR catlist] => true
    false

The call to knownInfo in this line recurses infinitely. This bug is holding
up the final algebra build. Any insight as to why this was rewritten and/or
why it recurses when it has clearly found the property would be most welcome.

\start
Date: 20 Feb 2003 20:32:46 +0100
From: David Mentre
To: list
Subject:  Re: latex processes

Hi tim,

Tim Daly writes:

> Latex hangs if you have a syntax error. It puts up a console prompt.
> In order to minimize the amount of console noise the latex output
> is put in a file called ${TMP}/trace 
> In order to find the hung process you can run the build using
>   make NOISE=

Here is the error message I have:

14 making /home/david/pub/axiom-libre/axiom-dm1/new/new/src/algebra/Makefile from /home/david/pub/axiom-libre/axiom-dm1/new/new/src/algebra/Makefile.pamphlet
This is TeX, Version 3.14159 (Web2C 7.3.7)
(./Makefile.tex
LaTeX2e <2001/06/01>
Babel <v3.7h> and hyphenation patterns for american, french, german, ngerman, n
ohyphenation, loaded.
(/usr/share/texmf/tex/latex/base/article.cls
Document Class: article 2001/04/21 v1.4e Standard LaTeX document class
(/usr/share/texmf/tex/latex/base/size10.clo))

! LaTeX Error: File `noweb.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: 

\start
Date: Thu, 20 Feb 2003 16:30:45 -0500
From: Tim Daly
To: David Mentre
Subject:  Re: latex processes

David,

The missing noweb.sty file should reside in mnt/linux/bin/tex
It is installed there as part of the build. It originally resides
in the src/scripts subdirectory but is copied (almost immediately)
to the mnt subdirectory. 

\start
Date: Thu, 20 Feb 2003 17:17:19 -0500
From: Bill Page
To: David Mentre
Subject: RE:  Re: latex processes

Tim,

I think the problem is that you have not given the full
path for the noweb.sty file in the make file. By default
Tex only looks in the /usr/share/texmf/tex/latex/base
(or equivalent) directory for it's style files. If
some process has previously copied the noweb.sty file
to that default directory (as it is on my system
from previously playing with noweb itself and probably
on your system too), then the make will succeed using
the file that it finds there. But in a virgin installation
the noweb file will not be found.

This same error occurred in your previous (partial) Axiom
release. I think you fixed most of the places where noweb
was referenced without a full path. Maybe this one just
was missed. Perhaps you should remove noweb.sty from the
Tex default directory on your system so you can catch this
problem. Or alternatively you could change the location
where you copy noweb.sty to the Tex default and then
modify the makefile.pamphlet files so that they all
use this default.

> -----Original Message-----
> 
> David,
> 
> The missing noweb.sty file should reside in mnt/linux/bin/tex 
> It is installed there as part of the build. It originally 
> resides in the src/scripts subdirectory but is copied (almost 
> immediately) to the mnt subdirectory. 

\start
Date: Fri, 21 Feb 2003 09:47:24 +0000
From: Mike Dewar
To: Tim Daly
Subject: Re:  latex processes
cc: Bill Page

Tim,

The version of LaTeX I use has various command-line modes such as
"-interaction=batchmode" which stops LaTeX hanging on error.  You might
want to check these out in case they can make your build process more
robust.

On Thu, Feb 20, 2003 at 12:26:21AM -0500, Tim Daly wrote:
> Can you give me a clue about where the latex proceses are started?
> 
> Latex hangs if you have a syntax error. It puts up a console prompt.
> In order to minimize the amount of console noise the latex output
> is put in a file called ${TMP}/trace 
> In order to find the hung process you can run the build using
>   make NOISE=
> 
> This is documented in the Makefile.dvi in the root directory (see
> FAQ item number 4).

\start
Date: Fri, 21 Feb 2003 11:45:21 +0100 (CET)
From: Bertfried Fauser
To: list
Subject:  compilation on SuSE 8.0

Hi Tim,

after I have moved noweb.sty manually to a place where LaTeX can find it,
the compilation finished. I was able to start interpsys and to do integer
arithmetic (but nothing else, since it reported error messages like
Float.o not found) but I guess, these abilities are not yet present...

After all, very amazing and encouraging to see the thing really starting
to work, great...

\start
Date: Fri, 21 Feb 2003 06:32:22 -0500
From: Tim Daly
To: Mike Dewar
Subject: Re:  latex processes
cc: Bill Page

Thanks for the latex tip. I did a strings on my latex and it does
support that option. Good catch. I'll fix that in the next release.
This has been a thorn for a long time and I've never found that option.

\start
Date: Fri, 21 Feb 2003 06:40:13 -0500
From: Tim Daly
To: Bertfried Fauser
Subject: Re:  compilation on SuSE 8.0

Bertfried, 
 
Glad that build succeeded. Actually, symbols should probably work also
but very little else does because of a property lookup bug I'm chasing.
Currently I know where the bug occurs but not why (which is somewhat
harder to infer). There have been a few changes in the code and it is
broken in a subtle way. I've been ignoring this bug in order to make
progress but it is now "front and center" on the problem list and should
fall shortly. *IF* that is the only remaining bug then the rest of the
algebra should compile cleanly and we'll have a fully running system.

\start
Date: Fri, 21 Feb 2003 11:56:18 +0000
From: Mike Dewar
To: Tim Daly
Subject: Re:  Re: CCL development
cc: Bill Page

According to our logs that particular fix came from Barry Trager but I
don't have all my mail files from 1996 any more so can't find the
message saying why.  

Sorry I can't be of more help,

On Thu, Feb 20, 2003 at 12:39:16AM -0500, Tim Daly wrote:
>  
> > Somehow we need to ensure that all the useful but undocumented
> > information about Axiom and CCL stored in Themos and my brains is made
> > explicit, so please let me know about any problems you encounter.
> > Although I can't promise to be able to help I will certainly try.
> 
> One particular problem I'm chasing is that algebra files that contain
> lookups like:
>         if S has SetCategory then SetCategory
> fail by going into an infinite look in knownInfo (info.boot).
> It appears that this code has been rewritten and no longer works in
> common lisp although it must somehow work in CCL. knownInfo does in
> fact find that (has s SetCategory) exists but then appears to call
> itself again. The offending code appears to be:
> 
> 
>     -- This line caused recursion on predicates which are no use in deciding
>     -- whether a category was present.
>     -- or/[AncestorP(cat,LIST CAR u) for u in CADR catlist | knownInfo CADR u] => true
>     or/[AncestorP(cat,LIST CAR u) and knownInfo CADR u for u in CADR catlist] => true
>     false
> 
> The call to knownInfo in this line recurses infinitely. This bug is holding
> up the final algebra build. Any insight as to why this was rewritten and/or
> why it recurses when it has clearly found the property would be most welcome.

\start
Date: Fri, 21 Feb 2003 07:04:05 -0500
From: Tim Daly
To: Mike Dewar
Subject:  SetCategory infinite loop
cc: Bill Page

Thanks, Mike. Once I have more information, if I haven't solved it,
I'll ask Barry. -- Tim

\start
Date: Mon, 24 Feb 2003 07:14:57 -0500
From: Tim Daly
To: Bill Page
Subject:  porting Axiom

Bill, re: porting issues.

Apparently you can run the MS VC++ compiler under GNU/Wine. See:
http://codingstyle.com/articles/using-ms-vcpp-with-gnu-wine.html

When you built the latest version did you do it on Windows?

\start
Date: Mon, 24 Feb 2003 11:16:30 -0500
From: Bill Page
To: list
Subject:  RE: porting Axiom

Tim,

No, I have not tried to build the latest version
on Windows yet. I have only built it under Linux
Red Hat 8.0.

I wasn't planning to use the M$ VC++ compiler since it
seemed both possible and perhaps make more sense to
remain entirely within the "free software" environment
by using MSYS/MinGW. I think this will minimize the C
language compatibility issues and also take advantage
of the growing expertise available on porting to
Windows in this community.

I am very interested in the work going on in porting
GCL to windows. Mike Thomas has produced a very slick
installation package for GCL 2.5 on windows using the
free Innosetup package

  http://www.jrsoftware.org/isinfo.php

which has apparently also been used for Maxima. I think
this would be very nice way to install the "run time"
version of Axiom (once we get to that stage).

Anyway, over the next few days (or at most, weeks) I
will be trying to build your latest Axiom under Windows
(probably using GCL 2.5 if that isn't a big deal) and
with MSYS/MinGW. MSYS allows one to build applications
directly under Windows (no cross compile).

Wine is a "windows-like" environment under Linux intended
mostly for doing things like running Microsoft Office
(shudder) under Linux. It sounds like asking for trouble
to try to use this environment to build Linux apps using
a Microsoft compiler.

> Bill, re: porting issues.
> 
> Apparently you can run the MS VC++ compiler under GNU/Wine. 
> See: http://codingstyle.com/articles/using-ms-vcpp-with-gnu-wine.html
> 
> When you built the latest version did you do it on Windows?