Before delving into the code, you need to do some high-level conceptual thinking about how to organize the address space of a UML process. UML maps its executable, physical memory, and kernel virtual memory into the address space of each of its processes. You need to decide where to put each of these so as to minimize the likelihood of a process trying to allocate that memory for its own use.

The only arch hook at this point is where in the address space the UML binary is going to load. The other addresses are still hard-coded because they happen to work for both i386 and ppc. UML puts its own memory in the area starting at 0x5000000 and process stacks 4M below its own process stack. These choices may not work on all architectures, so feel free to generalize them. To locate a likely area on your arch, staring at /proc/<pid>/maps of various processes on the host has been the technique so far.

You need to create arch/um/Makefile-$(SUBARCH), which contains the following definitions:

• START_ADDR - The address where the UML executable will load in memory. This address must be chosen so that it won't conflict with any memory that a UML process is going to want to use. The i386 definition is

                      START_ADDR = 0x10000000
                    

• ARCH_CFLAGS - Anything that needs to be added to CFLAGS goes here. Both the i386 and ppc ports use this to turn off definitions that would pull hardware-specific code into the kernel. The ppc definition is

                      ARCH_CFLAGS = -U__powerpc__ -D__UM_PPC__
                    

• ELF_SUBARCH - This is the name of the ELF object format for the architecture. On i386, it's 'i386', but on ppc, it's not 'ppc' (it's 'powerpc'). The i386 definition is

                      ELF_SUBARCH = $(SUBARCH)
                    

There are three headers, which go in arch/um/include/sysdep-$(SUBARCH), which need to be written, and each needs to contain a certain set of definitions.

ptrace.h

ptrace.h contains a number of definitions which insulate generic UML code, and its calls to ptrace, from details of the machine's register set.

First, the register set needs to be defined as struct sys_pt_regs. This is a structure which contains the machine's register set as defined by ptrace. This may be different from the machine's register set - i386 adds ORIG_EAX, which doesn't correspond to a register on the chip. The generic code won't look inside the structure directly, but will do structure copies and access it through the macros described below. The i386 definition is

                
#define UM_MAX_REG (17)

struct sys_pt_regs {
  unsigned long regs[UM_MAX_REG];
};

              

Along with this, you need a definition of EMPTY_REGS, which is a default initialization of a struct sys_pt_regs:

                
#define EMPTY_REGS { { [ 0 ... UM_MAX_REG - 1 ] = 0 } }

              

Then, you need a number of macros to allow the generic code to access information in a register set. These come in two varieties, UM_* and UM_*_OFFSET. The UM_* macros return a field from a struct sys_pt_regs which can be assigned to or its value taken, while the UM_*_OFFSET macros return a number suitable for a call to ptrace(PTRACE_SETREG, ...). Below are the required macros and their i386 definitions.

• UM_IP(regs) - the instruction pointer from regs
• UM_IP_OFFSET - the instruction pointer offset

                      
  #define UM_REG(r, n) ((r)->regs[n])
  #define UM_IP(r) UM_REG(r, EIP)
  #define UM_IP_OFFSET (EIP * sizeof(long))
  
                    

• UM_SP(regs) - the stack pointer from regs
• UM_SP_OFFSET - the stack pointer offset

                      
  #define UM_SP(r) UM_REG(r, UESP)
  #define UM_SP_OFFSET (UESP * sizeof(long))
  
                    

• UM_ELF_ZERO(regs)
• UM_ELF_ZERO_OFFSET - the register and offset that needs to be zeroed before branching to the entry point of a new process. This is a bit of a kludge and may be replaced with a more general UM_ELF_INIT(regs) macro.

                      
  #define UM_ELF_ZERO(r) UM_REG(r, EDX)
  #define UM_ELF_ZERO_OFFSET (EDX * sizeof(long))
  
                    

• UM_FIX_EXEC_STACK - A hook which is called after loading an ELF image into memory. On i386, this is empty, but on ppc, for example, the stack needs to be moved to sit on a 16 byte boundary.

                      
  extern void shove_aux_table(unsigned long sp);
  #define UM_FIX_EXEC_STACK(sp) shove_aux_table(sp);
  
                    

• UM_SYSCALL_RET(regs)
• UM_SYSCALL_RET_OFFSET - the register and offset that contains the system call return value as it finishes.

                      
  #define UM_SYSCALL_RET(r) UM_REG(r, EAX)
  #define UM_SYSCALL_RET_OFFSET (EAX * sizeof(long))
  
                    

• UM_SYSCALL_NR(regs)
• UM_SYSCALL_NR_OFFSET - the register and offset that contains the system call number at the start of system call execution.

                      
  #define UM_SYSCALL_NR(r) UM_REG(r, ORIG_EAX)
  #define UM_SYSCALL_NR_OFFSET (ORIG_EAX * sizeof(long))
  
                    

• UM_SET_SYSCALL_RETURN - stores the result of a system call in the right place in the register set. On some architectures special handling is needed, eg setting a flag on error returns on ppc.

                      
  #define UM_SET_SYSCALL_RETURN(_regs, result)	        \
  do {                                                    \
          if (result < 0) {				\
  		(_regs)->regs[PT_CCR] |= 0x10000000;	\
  		UM_SYSCALL_RET((_regs)) = -result;	\
          } else {					\
  		UM_SYSCALL_RET((_regs)) = result;	\
          }                                               \
  } while(0)
  
                    

• UM_SYSCALL_ARG1(regs)
• UM_SYSCALL_ARG1_OFFSET
• UM_SYSCALL_ARG2(regs)
• UM_SYSCALL_ARG2_OFFSET
• UM_SYSCALL_ARG3(regs)
• UM_SYSCALL_ARG3_OFFSET
• UM_SYSCALL_ARG4(regs)
• UM_SYSCALL_ARG4_OFFSET
• UM_SYSCALL_ARG5(regs)
• UM_SYSCALL_ARG5_OFFSET
• UM_SYSCALL_ARG6(regs)
• UM_SYSCALL_ARG6_OFFSET - the appropriate system call argument or offset

                      
  #define UM_SYSCALL_ARG1(r) UM_REG(r, EBX)
  #define UM_SYSCALL_ARG2(r) UM_REG(r, ECX)
  #define UM_SYSCALL_ARG3(r) UM_REG(r, EDX)
  #define UM_SYSCALL_ARG4(r) UM_REG(r, ESI)
  #define UM_SYSCALL_ARG5(r) UM_REG(r, EDI)
  #define UM_SYSCALL_ARG6(r) UM_REG(r, EBP)
  
  #define UM_SYSCALL_ARG1_OFFSET (EBX * sizeof(long))
  #define UM_SYSCALL_ARG2_OFFSET (ECX * sizeof(long))
  #define UM_SYSCALL_ARG3_OFFSET (EDX * sizeof(long))
  #define UM_SYSCALL_ARG4_OFFSET (ESI * sizeof(long))
  #define UM_SYSCALL_ARG5_OFFSET (EDI * sizeof(long))
  #define UM_SYSCALL_ARG6_OFFSET (EBP * sizeof(long))
  
                    

syscalls.h

syscalls.h needs to define a number of things related to system calls.

• syscall_handler_t - a typedef describing the interface to the system call functions. The i386 port declares a syscall_handler_t as taking a struct sys_pt_regs as its only argument

                      
  typedef long syscall_handler_t(struct sys_pt_regs regs);
  
                    

  and the ppc port declares it as taking six unsigned longs

                      
  typedef long syscall_handler_t(unsigned long arg1, unsigned long arg2,
  			       unsigned long arg3, unsigned long arg4,
  			       unsigned long arg5, unsigned long arg6);
  
                    

• EXECUTE_SYSCALL - a macro which takes a system call number and a register set and invokes the system call. This should match the declaration of syscall_handler_t, so the i386 port sticks the entire register set on the stack

                      
  #define EXECUTE_SYSCALL(syscall, regs) (*sys_call_table[syscall])(regs)
  
                    

  while the ppc port extracts the individual arguments

                      
  #define EXECUTE_SYSCALL(syscall, regs) \
          (*sys_call_table[syscall])(UM_SYSCALL_ARG1(&regs), \
  			           UM_SYSCALL_ARG2(&regs), \
  				   UM_SYSCALL_ARG3(&regs), \
  				   UM_SYSCALL_ARG4(&regs), \
  				   UM_SYSCALL_ARG5(&regs), \
  				   UM_SYSCALL_ARG6(&regs))
  
                    

• ARCH_SYSCALLS - a fragment of an array initialization. This will be incorporated into the system call vector, which has the entry point for each system call in the appropriate slot. This defines system calls which are present on Linux for this architecture, but not for all of the others. An empty value is OK, and the ppc port used to define this as such. When it is not empty, it should be written with a trailing comma. Here is a piece of the i386 definition:

                      
  #define ARCH_SYSCALLS \
          [ __NR_vm86old ] = sys_ni_syscall, \
          [ __NR_modify_ldt ] = sys_modify_ldt, \
          [ __NR_lchown32 ] = sys_lchown, \
          [ __NR_madvise ] = sys_madvise,
  
                    

• LAST_SYSCALL - this is the number of the last defined system call for this architecture, which may be one of the generic system calls, so check the numbers. It is used to pad out the otherwise undefined end part of the system call vector.

                      
  #define LAST_SYSCALL __NR_madvise
  
                    

sigcontext.h

sigcontext.h defines a few sigcontext access macros.

• UM_ALLOCATE_SC(name) - Declares a sigcontext struct and any other data which is part of a sigcontext. It's put at the end of the locals where it's used, so it may also initialize things. The ppc definition declares a separate register set and assigns a sigcontext field to point to it:

                      
  #define UM_ALLOCATE_SC(name) \
  	struct sys_pt_regs name##_regs; \
  	struct sigcontext name; \
  	name.regs = &name##_regs
  
                    

  This will be filled in by fill_in_sigcontext, which has to match whatever is set up here.
• SC_FAULT_ADDR(sc) - Extracts the page fault address from the sigcontext

                      
  #define SC_FAULT_ADDR(sc) ((sc)->cr2)
  
                    

• SC_FAULT_WRITE(sc) - Zero if the fault was a read, non-zero if it was a write

                      
  #define SC_FAULT_WRITE(sc) (((sc)->err) & 2)
  
                    

• SC_IP(sc) - Extracts the faulting ip from the sigcontext

                      
  #define SC_IP(sc) ((sc)->eip)
  
                    

• SC_SP(sc) - Extracts the stack pointer from the sigcontext

                      
  #define SC_SP(sc) ((sc)->esp_at_signal)
  
                    

• SEGV_IS_FIXABLE(sc) - Returns non-zero or zero, depending on whether a particular seg fault was caused by a page or protection fault, which can be fixed, or some other fault, and can't. In the latter case, the UML process will be segfaulted.

                      
  #define SEGV_IS_FIXABLE(sc) (((sc)->trapno == 14) || ((sc)->trapno == 13))
  
                    

The actual implementation of the port is contained in sys-$(SUBARCH). You have complete freedom in this directory, except that when it is built, it must produce an object file named sys.o which contains all the code required by the generic kernel.

Here is a list of the files used by the existing ports, along with what they define.

ptrace.c

                
int putreg(struct task_struct *child, unsigned long regno, unsigned long value)

              

This does any needed validity checking on the register and the value, and assigns the value to the appropriate register in child->thread.process_regs. If it fails, it returns -EIO.

This may be changed in the future so that it is provided with just the register set rather than the whole task structure.

                
unsigned long getreg(struct task_struct *child, unsigned long regno)

              

getreg fetches the value of the requested register from child->thread.process_regs, doing any required masking of registers which don't use all their bits. This may also be changed to take the register set rather than the task structure.

ptrace_user.c

Linux doesn't implement PTRACE_SETREGS and PTRACE_GETREGS on all architectures. This file contains definitions of ptrace_getregs and ptrace_setregs to hide this difference from the generic code. Architectures which define PTRACE_SETREGS and PTRACE_GETREGS will implement these functions as follows

                
int ptrace_getregs(long pid, struct sys_pt_regs *regs_out)
{
	return(ptrace(PTRACE_GETREGS, pid, 0, regs_out));
}

int ptrace_setregs(long pid, struct sys_pt_regs *regs)
{
	return(ptrace(PTRACE_SETREGS, pid, 0, regs));
}

              

Architectures which don't will implement them as loops which call ptrace(PTRACE_GETREG, ...) or ptrace(PTRACE_SETREG, ...) for each register.

semaphore.c

This implements the architecture's semaphore primitives. It is highly recommended to steal this from the underlying architecture by having the Makefile make a link from arch/$(SUBARCH)/kernel/semaphore.c to arch/um/sys-$(SUBARCH)/semaphore.c.

checksum.c or checksum.S

This implements the architecture's ip checksumming. This is stolen from the underlying architecture in the same manner as semaphore.c.

sigcontext.c

This defines fill_in_sigcontext, which copies data from a struct_pt_regs into a struct sigcontext. The sigcontext will be whatever was declared by UM_ALLOCATE_SC.

Other files

If Linux on your architecture defines any private system calls, you will need to implement them here. Normally, you can take the code from the underlying architecture, and you might get away with linking to the files in the other architecture that implement them.

After implementing the stuff described above, you will build your shiny new UML, and the make will blow up horribly. The reason is that almost all of the headers in include/asm-um simply include their counterparts from include/asm-$(SUBARCH), like this:

                
#ifndef __UM_POLL_H
#define __UM_POLL_H

#include "asm/arch/poll.h"

#endif

              

Some of these headers will define things in ways that are incompatible with their use in UML and some will need different definitions on different architectures.

This has been handled in two ways. Where there are definitions that differ from architecture to architecture or where simple modifications need to be made to a header, those arch-dependent definitions are put in include/asm/archparam-$(SUBARCH).h, which is linked to by include/asm/archparam.h. archparam.h, in turn, is included by the headers which need those changes. At this point, they are

• elf.h - has a number of differences between i386 and ppc
• delay.h - i386 and ppc declare a delay value differently
• hw_irq.h - ppc defines hard_irq_{enter,exit}, but not irq_{enter,exit}, so ppc defines irq_{enter,exit} as hard_irq_{enter,exit}
• string.h - ppc blows up unless __HAVE_ARCH_STRRCHR is defined, which is done here

When archparam.h isn't sufficient, then the header is replaced by a symlink which points to an architecture-specific header. This header includes the generic UML header, but is able to put whatever it wants around that include, which enables it to do things like hide definitions and wrap ifdefs around the header. This has been done to system.h, processor.h, and sigcontext.h.

As an example, on ppc, include/asm-um/processor.h is now a symbolic link to include/asm-um/processor-ppc.h. processor-ppc.h wraps an ifdef around the UML generic processor.h and turns on a cpp symbol:

                
#ifndef __UM_PROCESSOR_PPC_H
#define __UM_PROCESSOR_PPC_H

#if defined(__ASSEMBLY__)
#define CONFIG_ALL_PPC

#include "asm/processor-generic.h"

#endif

#endif

              

There's no algorithm for doing this stage of the port, so I'll just describe a number of useful tricks.

gdb is available. Use it. It's usable for any part of the kernel after the beginning of start_kernel. If you need to debug anything before that, the 'debugtrace' option is handle. It causes the tracing thread to stop and wait to be attached with gdb. Then you can step through the very early boot before start_kernel.

If you're post-mortem-ing a bug and you want to see what just happened inside UML, there are some arrays which store some useful recent history:

• signal_record - stores the last 1024 signals seen by the tracing thread, including the host pid of the process getting the signal, the time, and the IP at which the signal happened. This is a circular buffer and the latest entry is at index signal_index - 1.
• syscall_record - the same, except it stores process system calls. It stores the system call number, the return value (and 0xdeadbeef is stored there if it hasn't returned), the UML pid of the process, and the time. It is indexed by syscall_index, so the most recent entry is at index syscall_index - 1.

These provide a decent picture of what UML has been doing lately. Looking for unusual things here immediately before a bug happened is a useful debugging technique. Correlating timestamps between the two arrays is also sometimes useful.

If you have reproducable memory corruption, an extremely useful way to track it down is to set the page that it happens on read-only and to see what seg faults when it tries writing to that page. Obviously, this only works if there aren't legitimate writes happening to that page at the same time.