Lakai - Linux Akai sampler tools
================================

Author:       Frank Neumann
Version:      0.1
Date:         05.02.2004
Homepage:     http://lakai.sourceforge.net


What is it?
-----------
Lakai is a small set of tools (+ a link library) used to transfer sampler 
data (programs, samples) between a PC with a SCSI host adapter and an AKAI 
sampler (S1000, S2000..). The current tools allow an easy way to create a
full backup of the sampler's memory contents to the PC and a full restore
of this data back from the PC to the sampler.
Future versions might contain more fine-grained control over the data you
exchange, but this is still in planning stage.


Additions/changes in this release
---------------------------------

This is the first release. Enter at your own risk, stranger!


Supported hardware
------------------

Currently, this software has only been tested on my hardware, which is a
1 GHz Athlon desktop PC with an Adaptec 2904CD SCSI host adapter connected 
to an S2000 sampler. Other hardware might work, but people need to test 
and report this first with their respective systems. Hopefully the list
of supported hardware will then soon grow.


Requirements
------------

- Hardware
  Naturally, you need a PC with SCSI host adapter and an AKAI sampler. Also,
  this software has only been written with Linux in my mind, and has been
  developed solely on ia32 hardware - but in theory it should also work on
  e.g. MIPS, UltraSparc or Alpha. Still to be tested.

- Cabling
  You might know about this already, but SCSI requires termination.
  That means: SCSI is a bus; a bus has two ends; and these two ends need
  to be terminated correctly. If you fail to do that, you might get strange
  errors sometimes. My setup looks like this:

  ---       ---       ---
 X| | <---> | | <---> | |X
  ---       ---       ---
  PC      CD-ROM     Sampler

  The two "X" indicate that in my case the bus is terminated at the PC side
  (inside the SCSI host adapter) and on the sampler. The external CD-ROM drive
  is in the middle, and thus needs no termination. The sampler only has one
  SCSI connection, so it has to be one end of the bus anyway. When you bought 
  it, it should have come with termination resistors installed inside.
  To check for sure, you need to open the sampler and look for 2 resistor
  arrays plugged in.
  
- To compile, you will need the usual developer tools:
  - glibc 2.x with its -dev package (include files etc)
  - gcc, the Gnu C compiler. 2.95 or 3.x, both should work.
  - binutils - should have been installed in parallel with gcc
  - make

- For analysis of your SCSI setup, the "scsiadd" tools is quite useful -
  so I suggest to install it (see "Links" at the end for download resources)
  
- There is also a graphical frontend to scsiadd called "scsiaddgui" for those
  who rather point-click-point-click-knock-over-my-coffee-cup than
  tippety-tappety-tip. Again, see the "Links" section at the end of this 
  file for URLs.

- Kernel: You will need a scsi generic driver ('sg'), either built statically
  into the kernel or as a loadable module. It should provide the
  "sg3" driver interface which comes with kernel 2.4 series and above.
  I am using a 2.4.16 kernel successfully, but I sometimes see warnings
  from the kernel about "scsi: Someone has reset channel A". These warnings
  are, as far as I can tell, harmless. However, I made a few tests with a
  2.4.20 kernel once, and things went a little more smooth there.

  I saw that in 2.4.16, after booting the PC, the sampler would not always
  immediately "see" the SCSI CD-ROM - perhaps because the PC has reset
  the entire SCSI bus at kernel startup, and the sampler first has to
  find out about this (and reset the bus again himself). Normally, just
  trying to access the CD-ROM once more after a few seconds worked for me.

  No statement on 2.6 kernels yet - I still have to try them. I wouldn't
  expect any big problems here, though.

  When you need to build a new kernel, you will find the SCSI Generic (or
  "sg") option under "SCSI generic support" in the "SCSI support" submenu 
  of a  "make menuconfig" or "make xconfig" run.
  If you build the sg module statically into the kernel, you need to reboot,
  but after the reboot everything is already "in place". If you opt for an
  "sg" module instead, you need to load it (e.g. through "insmod sg.o" or
  with modprobe - this requires root privileges), and you will have to let
  your kernel search for SCSI devices with scsiadd.

  After a restart (and/or possibly having loaded the sg.o module manually),
  you should see the sg device driver is listed in the available devices 
  when you do a 
  $ cat /proc/devices
  It will show up there under "Character devices" as "21 sg".

  If the module was already loaded when the kernel boots up (that is, if
  the sg driver was compiled into the kernel), any connected (and powered-up)
  SCSI sampler should appear in the list of probed SCSI devices. For instance,
  on my system where the sg driver is built into the kernel, I see the
  following info when the kernel comes up:

   Vendor: AKAI EMI  Model: S2000 SAMPLER     Rev: 2.00
   Type:   Processor                          ANSI SCSI revision: 01 CCS

   If you have installed the "scsiadd" tool, you can also use "scsiadd -p"
   to see a more verbose listing of the available SCSI devices, with their
   channel/unit/lun numbers, perhaps similar to this:

   franky@faramir:~> scsiadd -p
   Attached devices: 
   Host: scsi0 Channel: 00 Id: 05 Lun: 00
     Vendor: SONY     Model: CD-ROM CDU-8003A Rev: 1.9a
     Type:   CD-ROM                           ANSI SCSI revision: 02
   Host: scsi0 Channel: 00 Id: 06 Lun: 00
     Vendor: AKAI EMI Model: S2000 SAMPLER    Rev: 2.00
     Type:   Processor                        ANSI SCSI revision: 01 CCS
   Host: scsi1 Channel: 00 Id: 00 Lun: 00
     Vendor: PLEXTOR  Model: CD-R   PX-W1210A Rev: 1.05
     Type:   CD-ROM                           ANSI SCSI revision: 02

  This shows you an old Sony SCSI-CDROM at (0,5,0), followed by the 
  sampler at (0,6,0). The Plextor CD-RW is only an IDE device with
  emulated SCSI command set.

  If you loaded the sg module manually, you can ask that the SCSI bus is
  to be probed again at any time by issueing (as root) the command
  "scsiadd -s". This is especially useful when you did not turn on the
  sampler before the Linux kernel came up.

  Finding out what SCSI generic devices are available (and online):
  Do a "cat /proc/scsi/sg/device_strsi /proc/scsi/sg/devices". This
  will yield an output something like this:
  
  franky@faramir:/proc/scsi/sg> cat device_hdr devices
  host    chan    id      lun     type    opens   qdepth  busy    online
  0       0       5       0       5       0       2       0       1
  0       0       6       0       3       0       2       0       1
  1       0       0       0       5       0       5       0       1

  This is basically the same data as above, but in a more condensed form.
  Please take a note of the position of your sampler in this list; this
  is required for later when you want to use the backup/restore tools.
  The first device in this list (the one with (0,5,0) as chan/id/lun)
  becomes SCSI node /dev/sg0, the second becomes /dev/sg1 and so on.
  So, in this case my sampler is "/dev/sg1" at this time.

  The last column ("online") is interesting here: When for some reason the
  sampler does not answer to kernel commands in time, the device gets marked
  as "offline" by the Linux kernel, and any more attempts to access this 
  device will fail. In this case, you can put the device online again by 
  issuing a command like

  scsiadd -r <host> <id> <lun>
  scsiadd -a <host> <id> <lun>
  
  In my example above, the sampler is connected with host (or channel) 0,
  id 6, lun 0, so
  scsiadd -r 0 6 0  removes the sampler from the device list, and
  scsiadd -a 0 6 0  puts it in again, and sets it to "online".
  
  Generally, avoid accessing the sampler while it is busy, e.g. when it is
  booting up, loading a sample set from CD/harddisk etc. The sampler does
  not answer to SCSI commands in this time, and the PC will believe it's
  unavailable, and will set it to "offline".

  One trap you can easily fall into is also that MIDI communications between
  the PC and the sampler over SCSI will happen on a MIDI channel which has
  to be the same on both sides. The lakai tools will only use the lowest
  MIDI communications channel (1), so it should be set to this on the sampler
  as well. When you load a sample set from a CD or harddisk, it can change
  the sampler's channel, and suddenly you find lakai tools not to work
  anymore. In that case, you will have to
  a) Set the sampler's MIDI channel to 1
  b) Re-'online' it on the PC with the above mentioned scsiadd command as
     it will most likely have been set to offline at this time.
    

Building
--------

autoconf/automake is not supported - so there is NO ./configure script
to call here. But the Makefile is there and is very simple, so just doing

make
make install

should do all you need. "make install" copies the files to /usr/local/ by
default - if you do not want that, please edit the simple Makefile yourself.


Usage
-----

Users:

Three small tools are coming with this Lakai release, namely lakclear,
lakbak and lakres:
- lakclear - erase all samples and programs currently resident in the sampler
- lakbak   - performs a complete backup of sampler's data to PC.
- lakres   - performs a restore of data from PC to the sampler.

Typically, you would work e.g. like this:
- boot the sampler;
- erase all data in it with lakclear;
- record some samples through its LineIn connectors
- adjust programs, keygroups etc through the sampler's console
- back up all of the sampler to the PC with lakbak;
- (on the next day, when you return to your work): boot the sampler
  again, clear it with lakclear and restore yesterday's data with lakres.

In order to use these tools, you will need privileged access to a SCSI
generic device, located in /dev/sg* (as mentioned above).

Normally, those device nodes have these permissions:
crw-------    1 root     disk      21,   0 May 25  2001 /dev/sg0
crw-------    1 root     disk      21,   1 May 25  2001 /dev/sg1
...

To use one of them as a non-root user, you could do the following:
- Add yourself to the "disk" group in /etc/group. One line in there reads
  like this before:
  disk:x:6:
  and for me it now reads:
  disk:x:6:franky

  After having made this change, you might need to re-login. Use the "id"
  command to see if you are really member of the disk group afterwards.
  
- Modify the permission for the affected /dev/sg* device entry as root:

# chmod 660 /dev/sg1
.. do the same for possible other sg devices.

I know this presents a possible security risk, but I assume you are not
using these tools on a networked 24/7 multi-user database/server machine.

When you have done this, you should be able to get a complete backup from
the sampler like this:

lakbak /dev/sg1 <name.list>

where /dev/sg1 is my SCSI node name for the sampler, and <name.list> is
a (text) file that will be created and to which lakbak will write the 
names and types of all data it is downloading from the sampler. 
All retrieved files (see below) will be written into the current directory -
so please create and enter a subdirectory first if you want to avoid file
clutter.

Similarly, a

lakclear /dev/sg1

clears out all samples and programs from the sampler (actually, there will
always be at least one "Test program" in the sampler because it must
not contain 0 programs), and

lakres /dev/sg1 <name.list>

will try to send all files listed in <name.list> back to the sampler.

All programs spit out some diagnostic information as they run.

The <name.list> file is a simple ASCII text file that contains:
- empty lines
- comment lines (starting with the "#" character to indicate this is a 
  comment)
- "SAMP" or "PROG" lines which refer to sample (.s) and program (.p) files.

The .p files contain program data: How to arrange samples into keygroups,
splits, velocity switches, what filter to apply, LFO settings etc.

The .s files contain the actual PCM data, but they also include some
AKAI-specific header info, like root note, loop points etc. To convert
a .s file into, say, a standard .wav file, you need to cut off these
header bytes (for an S2000, this is 192 bytes), and stuff the remaining
data through sox (SOund eXchange) which can then create a .wav from it.
Example: Assume you have a file named "COOLDRUM.s", to convert it into
a WAV you would do:

dd if=COOLDRUM.s bs=192 skip=1 | sox -t raw -r 44100 -sw - cooldrum.wav

Developers:

If you are a developer, you will want to know about using the link library,
liblakai.a. Please read the lakai.h header file and the example applications'
source code to understand how it works. There is still lots of work to be
done, but the current code is (I think) quite simple to understand and start
with.


Performance
-----------

My measurements so far gave up/download speed of up to 620 KBytes/sec which
does not sound all that bad. The uploading of programs take relatively long,
and clearing out the sampler when it carries a lot of data also takes quite
long (at the beginning; the more samples get deleted, the faster it gets :-).
I don't know right now if performance can still be increased, but I was told
that with Windows or Mac software the speeds were similar or even lower,
so I have quite a good feeling. This still means that entirely filling a
fully equipped (32MByte RAM) S2000 takes about 1 minute.


Known problems
--------------

- This is new code, only tested on MY equipment so far. For other setups 
  it may work or not, but I hope for people to tell me about their success
  stories. Do not use it for production systems yet until you feel somewhat
  confident with these tools.
  I am quite sure that at this point an S1000 will not work because it uses
  a different "block" size for its basic data structures (keygroups, sample
  headers, programs). This needs both testers and somecode changes. This 
  might also be try for S2800/S3000/S3200 etc, none of which I own or 
  could test.

- Multis are not handled yet, neither in backup nor restore procedures.

- I have seen problems in transferring large samples back to the PC (files
  larger than about 7 MBytes). After some point, I only get 0 data. To
  be examined. For smaller samples it has worked fine so far, though.

- When sending a new sample, after creating the sample header but before
  beginning the bulk data upload, I needed to put in a sleep(1) for now
  which ruins upload speeds. Working on it.
  Comparison: Loading a 16 MB sample bank (67 samples) from a 2x Apple
  CD-ROM: 48 seconds; uploading through lakai: 1min48sec. Subtract the
  67*1sec artifical delay, and I am close to the CD-ROM speed. Is even
  higher throughput possible? To be determined...

- No signal handling yet - specifically, when hitting Ctrl-C while a
  backup or restore procedure is running, the sampler might stay in "MIDI
  over SCSI" mode. Just don't do that for now.

- Memory consumption is a little "generous" - to be optimized later.


Reporting bugs
--------------

If you discover bugs in the source, documentation or elsewhere, or if you
have something to add, please let me know: franky@users.sourceforge.net

There is no mailing list or other discussion forum for this project yet; 
also, there is no CVS repository yet. If the need arises, all of this 
will probably be created, courtesy of SourceForge.


Authors
-------
Frank Neumann <franky@users.sourceforge.net>


Links
-----

Things you might be interested in:

http://www.mda-vst.com/akai/index.htm
Paul Kellett's pages on Akai disk formats etc

http://www.nal.ics.es.osaka-u.ac.jp/~oosaki/akaitools/index.html
Akaitools - Perl scripts to extract samples, programs and other
data from Akai sample CD-ROMs

http://llg.cubic.org/tools/
The scsiadd tool

http://www.8ung.at/klappnase/scsiaddgui/scsiaddgui.html
scsiaddgui, a graphical frontend to scsiadd

http://www.torque.net/sg
The source of the kernel's sg driver

http://www.linuxdj.com/audio/lad
Home of the Linux Audio Developers (LAD)

http://www.linuxsampler.org
The Linuxsampler project - somewhat related

http://lmuse.sourceforge.net
The MusE MIDI/Audio sequencer - my tool of choice.

http://www.alsa-project.org
The Advanced Linux Sound Architecture. You want this.

http://www.macanet.com
Holds (among others) a nice selection of downloads drumkit set of old
analog drum machines.

http://home.sprynet.com/sprynet/cbagwell/projects.html
sox, a universal sound sample translator


Thanks
------

My thanks go to:
- The people at SourceForge for putting up this great service of offering
  free webspace, logistics etc. to open source projects.
- 'Laslo' (you know who you are :-) for providing me with invaluable
  information and source code to start this project.
- Paul Kellett for his very helpful pages on the Akai disk and file formats
- Akai themselves for actually making available some SysEx communication
  protocols. Not as much as was really required, but a start.