File: README.ISOLCPUS

package info (click to toggle)
rtai 3.6.1-1
  • links: PTS
  • area: main
  • in suites: lenny
  • size: 18,448 kB
  • ctags: 13,674
  • sloc: ansic: 64,656; sh: 11,769; cpp: 10,387; makefile: 1,324; yacc: 575; lex: 366; php: 313; asm: 173; perl: 108; xml: 73
file content (78 lines) | stat: -rw-r--r-- 4,754 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
                      *** EXPLOITING CPUs ISOLATION ***

RTAI can take advantage of the possibility Linux affords of isolating CPUs 
from any of its scheduling activity on MultiProcessors (MP) machines. 
There should be no better way of explaining what it is than the following
excerpts from Linux "Documentation/kernel-parameters.txt":
        isolcpus=       [KNL,SMP] Isolate CPUs from the general scheduler.
                        Format: <cpu number>,...,<cpu number>
                        This option can be used to specify one or more CPUs
                        to isolate from the general SMP balancing and scheduling                        algorithms. The only way to move a process onto or off
                        an "isolated" CPU is via the CPU affinity syscalls.
                        <cpu number> begins at 0 and the maximum value is
                        "number of CPUs in system - 1".

                        This option is the preferred way to isolate CPUs. The
                        alternative -- manually setting the CPU mask of all
                        tasks in the system -- can cause problems and
                        suboptimal load balancer performance.
Add the following to the above, from the very same source:
        noirqbalance    [IA-32,SMP,KNL] Disable kernel irq balancing
Irq balancing can be disabled directly at kernel configuration and has little
effect on what will follow. Nonetheless RTAI prefers to be sure that Linux 
does not manipulate hardware related stuff on its own. So if you forgot to 
disable irq balancing when you made your kernel there is no need to remake it,
just set "noirqbalance" at boot time.

From what above it should be easy to infer that by using "isolcpus" you can 
be sure that Linux will have none of its tasks running on the isolated cpus.
That is not entirely true since many kernel threads (deamons) will have a
copy of each deamon repeated and assigned to all available CPUs. Nonetheless 
they will do nothing if no interrupt arrives on the isolated CPUs. Thus if 
RTAI is able to care avoiding any hard interrupt on the isolated CPUs they
will be isolated from Linux _completely_. Finally, combining all of the above 
with the possibility of forcing RTAI enabled Linux process/thread/kthreads 
to stay on the isolated CPUs the latter will find themselves processing just 
RTAI real time stuff, with a significant reduction of latencies/jitters. 
In the case of many RTAI tasks running on the isolated CPUs all issues 
producing latency/jitter will still have an effect, but it will be reduced 
to the least possible, without any added bus/pipe/cache interference from 
Linux.

It should be noticed however that there will remain a single interrupt still
managed by Linux, i.e. the one to the local APIC timer. Such an interrupt
will not come from the hardware but from a software interprocessor interrupt 
(IPI) generated by RTAI to keep Linux happy. It is likely that such an IPI 
could be sent only to non isolated CPUs but no testing of such a solution 
has been done up to now. The possible change requires just the substitution 
of a single line of code but I'm afraid it could damage Linux somehow.

Then what you have to do on the RTAI side is to load the core RTAI HAL module
using something like: "insmod rtai_hal.ko IsolCpusMask=<xxx>", where <xxx> 
is the mask of isolated CPUs. Please notice that Linux uses a list if isolated
CPUs while RTAI requires the corresponding mask (I'm lazy and let you do it).

What will happen next is that RTAI will divert all of Linux interrupts away
from isolated CPUs, caring of keeping those requested for RTAI (rt_request_irq)
on the isolated CPUs, without you having to care for it. In such a way there 
will be no Linux activity whatsoever on the isolated CPUs and they will remain 
within complete RTAI ownership.
Finally it is up to you to exploit such a feature by assigning all of your
tasks to the isolated CPUs, according to your needs, by using: 
"rt_task_init_cpuid", "rt_thread_init_cpuid", "rt_task_init_schmod".

Notice that you have also the possibility of further specialising your CPUs 
isolation scheme by diverting any real time interrupt you use to a single 
specific CPU, or CPU cluster within the isolated CPUs, by using the RTAI 
function: rt_assign_irq_to_cpu.

Naturally you can set RTAI "IsolCpusMask" even without setting the Linux 
"isolcpus" list, still with some beneficial effects though, for sure, they 
will not be as good as with the complete isolation setting described 
previously.

By appropriately exploiting what illustrated above it is mostly possible to 
reduce latencies to single microsec digit figures, especially if the hard 
timer periodic mode is adopted. 

Paolo.