1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
|
How to set up RAID
------------------
Using filesystems with RAID (Redundant Array of Inexpensive Disks)
has many advantages. First there is speed. RAID combines several
disks and reads/writes chunks from the disks in a sequence. That way
it can reach transfer speeds up to three times that of the slowest
disk, maybe even more. Second you are able to get bigger
filesystems than your largest disk (useful for /var/spool/news,
/home/ftp/pub etc.). Third there is the possibility to get
redundancy so a disk failure won't hurt.
For technical information on RAID please refer to
<URL:ftp://ftp.infodrom.north.de/pub/doc/tech/raid/>.
To do RAID with Linux you first need a kernel with appropriate
support. Linux 2.0.x supports linear and striping modes (the latter
is also known as RAID-0). Linux kernel 2.1.63 also supports RAID-4
and RAID-5. To use either of them you need to have special tools
installed. For linear and RAID-0 you need the mdutils package. To
use RAID 4/5 you need to have the raidtools package installed and a
kernel version higher than 2.1.62.
With RAID (not linear) you'll get best results if you use partitions
with exactly the same sizes. The RAID driver will work with
different sizes, too, but is less efficient as you may imagine after
reading some RAID documents.
1 Setting up RAID
Setting up RAID for normal filesystems such as /var, /home or /usr
is quite simple. First you need to partition your disks. After
you've done that you need to tell the RAID subsystem how you want to
organize the partitions, e.g. with
mdcreate -c4k raid0 /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1
mdcreate -c8k raid0 /dev/md1 /dev/sdd1 /dev/sde1 /dev/sdf1
This creates two RAIDs, each of them consisting of three partitions.
The first one has a chunk size of 4k while the second one uses 8k
chunks. These commands will create appropriate entries in /etc/mdtab.
The next step is to activate these devices with:
mdadd -ar
From now on you may refer to /dev/md0 and /dev/md1 as block devices
carrying your filesystems. Now you may create your filesystem on
the new devices just and add them to /etc/fstab just as usual.
Debian GNU/Linux is configured to initialize and activate any RAID
at boot time so you should not run into problems. Please note that
you need to have the RAID drivers compiled into the kernel, modules
might not work.
2 Swapping over RAID
The kernel has native support for distributing swap space over
several disks. Just add all swap partitions to /etc/fstab and use
'swapon -a' to activate all of them. The kernel uses striping
(RAID-0) for them. Here's a sample setup:
/dev/sda3 none swap sw
/dev/sdb3 none swap sw
/dev/sdc3 none swap sw
3 Root filesystem on RAID
The use of RAID for the root filesystem is a little bit tricky. The
problem is that LILO can't read and boot the kernel if it is not
stored linear on the disk (like it is on ext2 or dos). The solution
is to put the kernel on a different filesystem that doesn't use
RAID.
This way LILO would boot the kernel but the kernel itself would be
unable to mount the root filesystem because its RAID subsystem isn't
initialized.
For late 2.1.x kernels there's a kernel parameter that can be used
to load the kernel from a RAID. This is
md=<md device no.>,<raid level>,<chunk size factor>,<fault level>,dev0,dev1,...,devn
This needs to be added to lilo using the append="" option or
directly at the lilo prompt during boot stage. You'll find more
information in Documention/md.txt in the Linux source tree.
For stable kernels (2.0.x) and "not soo late" development kernels
(2.1.x) you need a mechanism to call some programs, at least mdadd,
before the kernel tries to mount the root file system and after the
kernel is loaded.
The only way to achive this is to use the initial ramdisk also known
as initrd. General information about initrd may be found in the
Documentation directory inside of the kernel source tree. If the
Linux kernel uses initrd it mounts the ramdisk as root file system
and executes /linuxrc if it is around. After this is finished the
kernel continues its boot process and mounts the real root
filesystem. The old / (from the initrd) will be moved to /initrd if
that directory is available or umounted otherwise.
The initrd file is a simple rootdisk. It should contain all the files
that are needed for processing the /linuxrc file. This includes a
working shell if it's a shell script and all tools that are used in
this script. This might include a working libc with ld.so and
tools, too.
After you have initialized RAID from /linuxrc you need to tell the
kernel where its root filesystem resides. As it uses the initrd it
might not know. There is an easy interface for this using the
proc filesystem. You only need to echo the appropriate device number to
/proc/sys/kernel/real-root-dev and the kernel continues with that
setting.
As lilo isn't able to boot from a non-linear block device (such as
RAID) you need to reserve a small partition with the kernel on it.
I've decided to use a 10MB partition which I use as /boot and put
stuff on it. 10MB is plenty of space for only one kernel and
initrd, currently my system only uses 2.5 MB of it. So
/etc/lilo.conf still points to /boot/vmlinuz-2.0.34 in this setup.
Now, decide what needs to be done in the /linuxrc script. You only
need to activate RAID and tell the kernel where your root filesystem
resides. The following script should do it:
#! /bin/ash
if [ -s /etc/mdtab -a -f /sbin/mdadd ]
then
echo "Preparing system for rootfs raid."
/sbin/mdadd -ar
/bin/mount -t proc /proc /proc
echo 0x900 > /proc/sys/kernel/real-root-dev
/bin/umount /proc
else
echo "No mdtab or mdadd found."
fi
You may use any block device as root filesystem. 0x900 stands for
major number 9 and minor number 0 which is /dev/md0.
Now make a list of binaries needed and additional files. Of course
you need some device files in /dev/ as well. To get the /linuxrc
script working at all, you need to have /dev/tty1. The other devices
depend on your /etc/mdtab file. You will at least need /dev/md0.
Binaries: ash, mount, umount, mdadd
Files: mdtab, fstab and mtab and for safety passwd
Devices: tty1, depending on /etc/mdtab
I use this mdtab:
# mdtab entry for /dev/md0
/dev/md0 raid0,4k,0,93f5553f /dev/hda2 /dev/hdb2
# mdtab entry for /dev/md1
/dev/md1 raid0,8k,0,3ffaa1d8 /dev/hda4 /dev/hdb4
Therefore I have created these block devices:
/dev/hda2
/dev/hda4
/dev/hdb2
/dev/hdb4
/dev/md0
/dev/md1
/dev/md2
/dev/md3
You can use the mknod program to create the device files, e.g. with
the following command for tty1:
mknod dev/tty1 c 4 1
Ok, but how does one create the initrd file? The best thing you can
do is to create the directory /tmp/initrd and install everything in
it. When you're finished you determine the diskspace it uses (du -s)
and create the initrd itself. The following command would create a
1M initial ramdisk. This is what I use.
dd if=/dev/zero of=/tmp/initrd.bin bs=1k count=1024
mke2fs /tmp/initrd.bin
mount -o loop /mnt /tmp/initrd.bin
As you probably use dynamic linked binaries you need to make sure
that the Linker and the dynamic libraries are installed, too. You
need to copy at least /lib/libc*.so and /lib/ld-linux.so.2 as link
to /lib/ld-2.0.6.so. You also need an appropriate /etc/ld.so.config
file. Appropriate here means that "/lib" should be the only line in
it. You need to create a new library cache /etc/ld.so.cache file
with "ldconfig -r /initrd". Of course you also have to install the
needed binaries in appropriate directories /sbin and /bin.
Don't forget to create the /proc directory or mount will fail. The
fstab and mtab files can be empty. They will only be read, not
written to, but they need to exist. For the /etc/passwd file it's
sufficient to include the root user.
After you have copied everything from /tmp/initrd to the ramdisk,
umount it (e.g. with the command "umount /mnt") and move the file to
/boot/initrd.bin. Now you need to tell lilo to load the kernel and
the ramdisk. That's no problem, just use a record in /etc/lilo.conf
similar to the following:
image=/boot/vmlinuz-2.0.34
initrd=/boot/initrd.bin
label=linux
read-only
Issue the command "lilo" and you're nearly done. As the RAID
subsystem is now configured at boot stage before any /etc/init.d
scripts were issued you should disable the mdadd call in /etc/init.d
scripts.
|
