
   
   
   
   
   
     _________________________________________________________________
   
                IRQTUNE -- A LINUX X86 IRQ PRIORITY OPTIMIZER
                                       
   Copyright 1996, 1997 by Craig Estey.
   irqtune version 0.5 
   Last updated: Wed Apr 23 06:08:54 PDT 1997
   See the Changes section at the bottom of this document.
   Table of Contents
   
   
   
   
   irqtune changes the IRQ priority of devices to allow devices that
   require high priority and fast service (e.g. serial ports, modems) to
   have it.
   
   With irqtune, a 3X speedup of serial/modem throughput is possible. 
   
   
   
   
   
   
     _________________________________________________________________
   
                           WHERE DO I GET IRQTUNE?
                                       
   irqtune is free software under the terms and conditions of the GNU
   Public License. See the file COPYING, included in the distribution,
   for details.
   
     * The author is Craig Estey, (cae@best.com).
       
     * This FAQ is available online via http://www.best.com/~cae/irqtune.
       This is the most authoritative reference and will always contain
       the most up-to-date information.
       
     * The distribution is a gzipped tar archive that contains all
       programs, sources, and this FAQ (in both HTML and text versions).
       It is available from:
       
          + http://www.best.com/~cae/irqtune/irqtune.tgz
          + ftp://shell5.ba.best.com/pub/cae/irqtune.tgz
            
   
   
   
   
   
   
   
     _________________________________________________________________
   
                      HOW DO I KNOW IF I NEED IRQTUNE?
                                       
   You are running Linux on an x86 PC, other architectures to be
   implemented later--Sorry.
   
   You probably need irqtune, if you are experiencing any of the
   following: 
   
     * SLIP/PPP transfers seem slow. For example, using a 28.8 (or
       better) modem, the effective throughput is approximately 700
       bytes/second instead of the expected 2500 bytes/second.
       
     * A running SLIP/PPP dies at random times.
       
     * Serial connections are slow or drop data.
       
     * Netscape hangs mysteriously or stalls when trying to access a web
       page.
       
     * Equivalent serial/PPP programs under Windoze run much faster than
       under Linux.
       
     * Disk accesses seem to interfere with SLIP/PPP.
       
     * Interrupt handlers for specialized, time critical devices don't
       get control when they need to.
       
   
   
   
   
   
     _________________________________________________________________
   
             WHAT IS ACTUALLY HAPPENING TO CAUSE THESE PROBLEMS?
                                       
   
   
   When the PC boots Linux, the timer is given, by default, the highest
   IRQ priority in the system (it's IRQ 0 and thus, priority 0). On a
   standard configuration, the serial ports are priority 11 and 12!!!
   This means that 10 other devices have higher priority.
   
   
   
   
   
  Q: SO WHAT DOES IRQ PRIORITY DO?
  
   
   
   When multiple devices are in contention to interrupt the CPU, their
   priority decides which interrupts will occur in what order.
   
   
   
   
   
  Q: WHEN DOES THIS CONTENTION OCCUR?
  
   
   
   After an arbitrary period of having interrupts disabled (e.g after a
   cli), at the point where they're reenabled (sti). This can happen in
   several places:
   
     * In an ISR that runs with interrupts locked, it happens in the
       epilog, just before attempting to execute the bottom-half.
       
     * The bottom-half itself may do a lock and unlock.
       
     * When a task that enters the kernel to do a system call, the system
       call handler may lock and unlock interrupts briefly.
       
     * Almost anywhere in the kernel are brief periods where it does a
       cli to lock interrupts and then an sti to unlock them again.
       
     * Under Linux, from the moment the first instruction of the ISR
       prolog is executed until an sti is done (This occurs even in
       slow interrupts where the prolog does an sti after just a few
       instructions).
       
   
   
   
   
  Q: IF THERE ARE MULTIPLE INTERRUPTS NOW PENDING, WHICH ONE GETS THE SERVICE,
  THE SERIAL OR SOME OTHER?
  
   
   
   In the default configuration, the serial ISR will usually lose as it's
   priority 11.
   
   
   
   
   
   
     _________________________________________________________________
   
                         HOW DOES IRQTUNE HELP THIS?
                                       
   
   
   irqtune gives priority 0 to the whatever device we specify. If we
   specify a serial device, irqtune guarantees that the serial ISR gets
   control whenever a contention occurs.
   
   
   
   
   
   
     _________________________________________________________________
   
     WHY DOES THE SERIAL INTERRUPT SERVICE REQUIRE THE HIGHEST PRIORITY?
                                       
   
   
   
   
   
   
  Q: WHY DOES THE SERIAL DEVICE MERIT SUCH SPECIAL TREATMENT?
  
   
   
   Serial devices are somewhat unique. Even though they have one of the
   slowest data rates (relative to a disk), they are the largest consumer
   of interrupts and are extremely sensitive to interrupt latency (the
   time from when a device yanks the IRQ line until its ISR is executed).
   
   
   
   
   
   
  Q: COULD YOU GIVE A CONCRETE EXAMPLE OF THIS?
  
   
   
   
   
     * For a modem running at 33.6, we can have a peak maximum (with
       compression) of 6700 bytes/second. The serial driver programs the
       silo to interrupt after 8 bytes, leaving a latency window of 8
       bytes. This means that when the serial port yanks its IRQ line, it
       can still absorb 8 more characters before its buffer will overflow
       and data will be dropped
       
     * In terms of time, this means that the maximum that the serial ISR
       may be delayed is (8 / 6700) seconds or 1194 microseconds. It also
       means that the serial ISR will require 838 interrupts/second.
       
     * Currently there are 52 devices that install their ISR with the
       SA_INTERRUPT option. This means that they wish to run with
       interrupts disabled. I have not looked at all the devices, but
       just assume for the moment that all will run with interrupts
       locked for an arbitrary period.
       
     * Assume we've got 4 of them pending (remember, at higher priority).
       Assume that they will lock interrupts individually for 320, 300,
       190, and 500 microseconds, respectively. Assume that the serial
       ISR also wants an interrupt.
       
     * This means that the serial ISR will have to wait for (320 + 300 +
       190 + 500) or 1310us for these other higher priority ISR's to run
       to completion. This is greater than the maximum of 1194 us.
       
     * With internal modems, we probably just see start/stop behavior
       causing a slowdown. With an external modem, data will be dropped.
       This will cause PPP to see a CRC error and request transmission.
       Remember, the entire packet must be retransmitted, which means we
       just wasted 296-1500 bytes (depending on the selected MRU). With
       an MRU of 1500 bytes, we just wasted 23.4% of the bandwidth in a
       given second--all for the loss of a single byte!
       
     * Worse yet, consider a 115 Kbps ISDN card that is masquerading as a
       serial device (Yes, I know it should be a DMA device). In this
       case, the maximum delay the serial ISR can wait is 695 us. And
       that's just one ISDN channel. Try adding the other and, well,
       golly :-).
       
   
   
   
   
   
   
  Q: IN THIS EXAMPLE, HOW WOULD BOOSTING SERIAL IRQ PRIORITY HELP?
     * If the serial IRQ had priority 0, it would get in before the
       others. It could also re-interrupt after any individual lockout
       window. Thus, the maximum it would be forced to wait would be
       largest individual time, not the summation of all of these times.
       In this example, this means 500 us. which is now within the 1194
       us. maximum for the serial ISR.
       
     * It has been my experience that the serial device must always win
       these battles for contention. When serial devices don't get what
       they need, when they need it, they slow horribly or drop data
       outright.
       
   
   
   
   
   
   
   
     _________________________________________________________________
   
                ISN'T THAT EXAMPLE VERY UNLIKELY UNDER LINUX?
                                       
   
   
   Unlikely doesn't mean never. There are places were the contention
   period must occur, no matter how we program the CPU.
   
   Variations in CPU speed, RAM size, RAM speed, cache size, disk speed,
   disk rotational position, number and type of other devices, system
   workload, etc, etc etc, all contribute to variations of order and
   timing of internal OS events.
   
   Unlikely on one system may mean 1/1000th % chance. On another system,
   it may mean happens 50 times/second. Beyond a certain point, it all
   comes down to measurement.
   
   Tight, accurate, repeatable measurement is the key to system tuning.
   If we can't measure it, we can't tune it. Once we can measure these
   things, we can then try various combinations until we achieve our
   desired results. It can often be pointless to guess at performance.
   
   In fact, the interrupt disable windows themselves are used to prevent
   data corruption caused by various parts of the kernel that try to
   update a data structure simultaneously and get corrupted. Many of
   these windows are put there in the unlikely case that such corruption
   would occur. In this context, unlikely is never considered a reason to
   forego interrupt locking. Eliminating a necessary interrupt lock
   window could result in kernel panics, RAM corruption, disk data
   corruption, etc.
   
   
   
   
   
   
     _________________________________________________________________
   
             DOESN'T THIS HURT THE PERFORMANCE OF OTHER DEVICES?
                                       
   
   
   Not really.
   
   In actual practice, most devices don't even notice the difference.
   Most other devices (e.g. disks, tape, ethernet) are DMA devices. The
   DMA does most of the work, thus greatly reducing their need for
   interrupts. If the device allows a request queue, it may function
   autonomously on several requests, producing only one interrupt for the
   entire batch.
   
   Furthermore, serial interrupt services are, themselves, very fast.
   They slam their data as quickly as possible and get out ASAP. No fancy
   calculations, just the minimum, mindless data transfer. Almost
   everything else is handled later, in the bottom-half with interrupts
   enabled. In fact, a serial ISR may have to re-interrupt it's own
   bottom-half several times.
   
   Those devices that do experience some slight slowdown are more likely
   to have long interrupt disable windows themselves. Having several
   smaller cli/sti windows is much better than one large cli/sti
   window--It's just harder to program.
   
   
   
   
   
  Q: BUT SUPPOSE I WANT A BALANCED PRIORITY SYSTEM?
  
   
   
   Well, actually a prioritized system behaves like a balanced
   system--most of the time. This occurs when all devices have short
   interrupt lockout windows and short ISR execution times. The priority
   mechanism is like a safety valve--it only really matters when some
   device or combination of devices has held interrupts locked for an
   extended period of time.
   
   
   
   
   
  Q: BUT DOESN'T THAT SEEM UNFAIR TO OTHER DEVICES?
  
   
   
   When a disk ISR gets delayed, that's all that happens, a slight
   delay--disks and tapes are not real-time devices. When a serial ISR
   gets delayed, data is destroyed--serial devices are real-time devices
   that dictate the cadence of the entire system.
   
   It may sound cruel, but you just can't be fair. 
   
   And speaking of sound, if the sound card gets delayed, we hear an
   annoying pop right in the middle of our favorite piece of music.
   
   These are real-time devices that can not tolerate excessive delays. If
   we must defer some less time critical devices to meet the minimum
   real-time criteria of devices, then that's a bargain.
   
   
   
   
   
  Q: BUT SUPPOSE I REALLY WANT BOTH FAST SERIAL AND FAST DISK?
  
   
   
   Ultimately, it's a bit of a compromise. Which is better:
     * Reliable serial and slightly slower disk.
     * Slightly faster disk and unreliable serial/modem support.
       
   When paying an ISP for Internet access in $$$/hour, it's an easy
   decision :-). 
   
   
   
   
   
   
     _________________________________________________________________
   
             ISN'T THIS IRQ PRIORITY THING A BIT OF A NEW IDEA?
                                       
   
   
   No. It's actually an old idea. I've been doing device drivers since
   1977 and Unix kernel work since 1981. I've personally written 8 serial
   drivers have used this many times commercially. Giving the serial
   device the highest priority is actually standard practice in many
   systems. With a 4Mhz CPU, these problems used to occur at 1200 baud
   :-)
   
   
   
   
   
   
     _________________________________________________________________
   
                          HOW DO I INSTALL IRQTUNE?
                                       
   
   
   
   
   
   
  Q: WHERE SHOULD I PLACE IRQTUNE FILES?
     * Decide what directory you would like to contain irqtune's home
       directory. Some good choices are: /usr/local, /usr/lib,
       /usr/lib/hwtools, /home.
     * Exactly where depends upon your local conventions.
     * In this example, we'll use /usr/local.
       
   
   
   
   
   
   
  Q: HOW DO I UNPACK THE ARCHIVE?
     * Go to the containing directory.
       cd /usr/local
     * Unpack the tar file:
       gzip -d < irqtune.tgz | tar xvf -
     * This will create a directory, irqtune in /usr/local. This is
       /usr/local/irqtune. We will call this IRQTUNE_HOME.
       
   
   
   
   
   
   
  Q: HOW DO I DO A SIMPLE INSTALLATION?
     * Go to the IRQTUNE_HOME directory:
       cd /usr/local/irqtune
     * To install irqtune, enter:
       make install
     * This will install the files:
       /sbin/irqtune
       /sbin/irqtune_mod.o
       /sbin/irqtune_npr.o
       
   
   
   
   
   
   
  Q: WHY ARE .O FILES BEING PLACED IN THE /SBIN DIRECTORY?
  
   
   
   As a note to purists, the installation directory is completely
   arbitrary. /sbin is short, sweet, and easy to type. 
   
   The standard convention would be to install irqtune as /sbin/irqtune
   and the .o files as, say, /usr/lib/irqtune/*.o. irqtune uses a
   slightly different convention. It wants all installed files to be in
   the same directory. This works fine because argv[0] will point to the
   .o files. This also allows:
     * Multiple versions of irqtune to be installed simultaneously (in
       different directories).
     * Simplifies testing of new versions.
     * Provides better assurance that the built version is correct.
     * Using one directory produces less clutter.
     * One directory is more portable across all the myriad kernel
       revisions.
     * Future versions of irqtune may imbed the .o's directly into
       irqtune.
       
   
   
   
   
   
   
  Q: WHAT IF I REALLY DON'T WANT .O FILES IN /SBIN?
  
   
   
   If placing .o files in /sbin is deemed to be an anathema, we have two
   options:
     * Simply use another directory in place of /sbin:
       make SBIN=/whatever install
     * Install with a stub (This will install /sbin/irqtune as a shell
       script and use IRQTUNE_HOME/sbin/irqtune_*.o):
       make INSTALL=sh install
       
   
   
   
   
   
   
  Q: ARE THERE ANY SPECIAL CONSIDERATIONS FOR THIS ALTERNATE INSTALLATION
  METHOD?
  
   
   
   Yes. If the shell/stub installation method is used:
     * IRQTUNE_HOME must not be removed.
     * If using irqtune from an /etc/rc.d/rc.* file and irqtune will be
       invoked before secondary mounts are done, IRQTUNE_HOME should be
       placed on the root device.
       
   
   
   
   
   
   
   
     _________________________________________________________________
   
          HOW DO I USE IRQTUNE? DON'T I HAVE TO REBUILD MY KERNEL?
                                       
   No, we do not have to rebuild the kernel. irqtune uses insmod and
   rmmod to dynamically load and unload a kernel module. But it is
   correct to sense that irqtune is a kernel patch.
   
   
   
   
   
  Q: OK, IF IT'S A KERNEL PATCH, WHY NOT JUST ISSUE A KERNEL PATCH LIKE
  EVERYBODY ELSE DOES (E.G. DIFF -U OUTPUT)?
  
   
   
   irqtune will work even if we don't have the kernel source loaded. It
   uses insmod to load the patch, invoke it, and then unload it. The IRQ
   priority changes will last so long as the kernel is booted.
   
   
   
   
   
  Q: HOW DO WE INVOKE IT?
  
   
   
   irqtune takes two arguments optional arguments:
     * irqtune master slave
       
   The only caveats are:
     * If we did a simple installation, we must specify the full
       pathname, even if irqtune is placed in a directory that is in
       $PATH. This is required because irqtune uses argv[0] to locate its
       irqtune_mod.o file.
     * /sbin should be in $PATH--this is the probable place for insmod.
       As a convenience, irqtune will add /sbin to $PATH automatically.
       
   
   
   The default is 3 14 which will work for many standard configurations.
   More on this later.
   
   
     * Here is the usage output from irqtune:
       usage: irqtune [options] [master] [slave]
       version: 0.5
       options:
       -e -- show verbose errors
       -h -- display help
       -n -- nogo mode (just show what would happen)
       -o -- reset to original values (0/8)
       -v -- display irqtune version
       -F<file> -- use <file> instead of /proc/interrupts
       -q -- suppress priority table printing
       -s -- sort table by priority
       -u -- force module unload
       -x -- show inactive devices in table
       arguments:
       master -- high priority IRQ on PIC master (DEFAULT: 3)
       slave -- high priority IRQ on PIC slave (DEFAULT: 14)
       
   
   
   
   
   
   
  Q: COULD WE DO THIS FROM MY /ETC/RC.D/RC.LOCAL FILE?
  
   
   
   Yes. Just add a /sbin/irqtune line to this file and we're in business.
   You may also issue another irqtune command at any time.
   
   
   
   
   
  Q: WHAT IF MY KERNEL REVISION IS DIFFERENT FROM THE KERNEL REVISION THAT YOU
  BUILT IT ON?
  
   
   
   irqtune is 99.44% kernel revision independent. It is built using ELF
   binaries, so your insmod must understand them. Also, make sure that
   you have the correct insmod for your kernel. But that's really about
   it. There are some slight caveats--these are discussed later.
   
   
   
   
   
  Q: BUT WHAT IF I DON'T HAVE ELF BINARY SUPPORT, HOW CAN I STILL DO THINGS?
  
   
   
   Well, I'd recommend that you upgrade your kernel as ELF binaries are
   cool :-) But if you insist, you'll just have to recompile irqtune.
   Just be sure that /usr/src/linux/include is installed. The exact
   procedure for building a.out binaries can vary with compiler revision,
   so be sure to check your documentation on this (You may need to add a
   parameter or two).
     * Then, just type:
       make warp9
     * Here is the makefile's own documentation:
       INSTALLATION:
       install install prebuilt binaries in /sbin
       custom create custom installation -- Normally _not_ required --
       Try
       install first (EQUIVALENT: kvers sbin install)
       uninstall remove prebuilt binaries from /sbin
       
       INSTALLATION OVERRIDES:
       SBIN SBIN=/whatever specify installation directory (DEFAULT:
       /sbin)
       INSTALL INSTALL=simple simple installation (DEFAULT: simple)
       INSTALL=sh install as shell stub
       
       REBUILDING FROM SOURCE:
       kvers create kernel_version from local system (NOTE: this is
       normally
       _not_ required because we use the insmod -f). You can rebuild
       _without_ /usr/src/linux/include
       sbin rebuild binaries from source (full or partial)
       
       FORCE FULL REBUILD (requires /usr/src/linux/include):
       clean remove .o files to force full recompile)
       warp9 rebuild all (EQUIVALENT: clean kvers sbin install)
       
   That should do the trick. But, if it doesn't, look at revision numbers
   on insmod, gcc, ld, and the kernel for incompatibilities. Upgrade as
   necessary.
   
   
   
   
   
   
     _________________________________________________________________
   
             WHAT ABOUT MY NON-STANDARD HARDWARE CONFIGURATION?
                                       
   irqtune defaults for a standard IRQ configuration. It assumes that the
   highest priority device should be on IRQ 3. This is normally the first
   serial port on standard configurations, which is what you want.
   
   
   
   
   
  Q: HOW DO I DETERMINE WHAT MY IRQ CONFIGURATION IS?
  
   
   
   NOTE: For brevity, we've combined the non-sorted and sorted output in
   these examples. 
   
     * Just type /sbin/irqtune -n -o and we'll get something like:
       

SORTED BY IRQ:                          SORTED BY PRIORITY:

I00/P00:   8578913   timer              I00/P00:   8578913   timer

I01/P01:    109547   keyboard           I01/P01:    109547   keyboard

I02/P02:         0 + cascade            I02/P02:         0 + cascade

I03/P10:     86470 + serial             I11/P05:    197648 + sermux

I04/P11:    197648 + serial             I12/P06:     17968 + eth

I11/P05:    197648 + sermux             I13/P07:         1   math error

I12/P06:     17968 + eth                I14/P08:     93123 + Ux4F

I13/P07:         1   math error         I03/P10:     86470 + serial

I14/P08:     93123 + Ux4F               I04/P11:    197648 + serial

   
   
   NOTE: /proc/interrupts, and therefore irqtune, only reports on active
   devices. So to scope out the serial IRQ's, ideally, you'd have X
   Windows up with your serial mouse and be connected via PPP to the net.
   
   
   
   
   
   
  Q: OK, WE'VE GOT THE OUTPUT FROM /SBIN/IRQTUNE -N -O, WHAT DO WE DO WITH IT?
  
   
   
   The leftmost number is the IRQ number. The next number is the
   priority. The rightmost column is the internal device name (not to be
   confused with /dev names). In the above case, the two serial ports are
   on IRQ 3 and IRQ 4. Just use the lower number, in this case 3:
   
     * /sbin/irqtune 3
       
   This sets IRQ 3 to the highest priority.
   
   
   
     * After this command, the IRQ priorities are now:
       

SORTED BY IRQ:                          SORTED BY PRIORITY:

I00/P05:   8578913   timer              I03/P00:     86470 + serial

I01/P06:    109547   keyboard           I04/P01:    197648 + serial

I02/P07:         0 + cascade            I00/P05:   8578913   timer

I03/P00:     86470 + serial             I01/P06:    109547   keyboard

I04/P01:    197648 + serial             I02/P07:         0 + cascade

I11/P12:    197648 + sermux             I14/P07:     93123 + Ux4F

I12/P13:     17968 + eth                I11/P12:    197648 + sermux

I13/P14:         1   math error         I12/P13:     17968 + eth

I14/P07:     93123 + Ux4F               I13/P14:         1   math error

   
   
   
   
   
   
  Q: BTW, WHAT'S THE CASCADE DEVICE I SAW IN THE OUTPUT OF IRQTUNE?
  
   
   
   Glad you asked. There are actually two interrupt controllers, a master
   and a slave. The slave is cascaded to the master via its IRQ 2. The
   master controls IRQ's 0-7 and the slave controls IRQ's 8-15.
   
   You actually may select two high IRQ priorities, one for the master
   and one for the slave. irqtune defaults the slave to IRQ 14, which is
   normally the disk controller.
   
   In fact, cascade is sort of a "zero width" device as it does not
   contribute to interrupt latency. Setting the cascade to top priority
   on the master has an interesting effect which we'll see shortly.
   
   
   
   
   
  Q: BUT WE'VE ALSO GOT AN ETHERNET CONTROLLER ON IRQ 12. WHAT ABOUT THAT?
  
   
   
   In this case, we might want to use:
   
     * /sbin/irqtune 3 12
       
   because we want our ethernet card to have a higher priority than the
   disk controller. Actually if we did have this configuration, setting 3
   14 (the default) would make the ethernet card, the lowest priority
   device in the system.
   
   
   
     * The resulting priority would be:
       

SORTED BY IRQ:                          SORTED BY PRIORITY:

I00/P05:   8578913   timer              I03/P00:     86470 + serial

I01/P06:    109547   keyboard           I04/P01:    197648 + serial

I02/P07:         0 + cascade            I00/P05:   8578913   timer

I03/P00:     86470 + serial             I01/P06:    109547   keyboard

I04/P01:    197648 + serial             I02/P07:         0 + cascade

I11/P14:    197648 + sermux             I12/P07:     17968 + eth

I12/P07:     17968 + eth                I13/P08:         1   math error

I13/P08:         1   math error         I14/P09:     93123 + Ux4F

I14/P09:     93123 + Ux4F               I11/P14:    197648 + sermux

   
   
   
   
   
   
  Q: WHAT ABOUT THE SERIAL MULTIPLEXER CARD ON IRQ 11?
  
   
   
   This is a bit tricky because now we've got a serial device on the
   slave controller. It would be much better to put all serial cards on
   the master controller. Things would stay much simpler.
   
   In this case we would want to use:
   
     * /sbin/irqtune 11
       
     * The resulting priorities would be more complex and would result in
       something like:
       

SORTED BY IRQ:                          SORTED BY PRIORITY:

I00/P13:   8578913   timer              I02/P00:         0 + cascade

I01/P14:    109547   keyboard           I11/P00:    197648 + sermux

I02/P00:         0 + cascade            I12/P01:     17968 + eth

I03/P08:     86470 + serial             I13/P02:         1   math error

I04/P09:    197648 + serial             I14/P03:     93123 + Ux4F

I11/P00:    197648 + sermux             I03/P08:     86470 + serial

I12/P01:     17968 + eth                I04/P09:    197648 + serial

I13/P02:         1   math error         I00/P13:   8578913   timer

I14/P03:     93123 + Ux4F               I01/P14:    109547   keyboard

   
   
   
   
   
   
  Q: WAIT A MINUTE. DIDN'T WE JUST SPECIFY A SLAVE IRQ NUMBER AS THE MASTER TO
  IRQTUNE?
  
   
   
   Yes, this is shorthand way of saying 2 11. You can make a slave device
   top priority, but we get no options for the master IRQ. It will always
   be 2, the cascade device. Remember, the cascade device contributes no
   latency delay by itself.
   
   
   
   
   
  Q: SO WHY IS THIS CONFIGURATION SO BAD?
  
   
   
   Well, we boosted the priority of the serial multiplexer at the expense
   of the regular serial ports. The only way to allow all serial ports
   equally high priority is to group them on consecutive IRQ's and set
   the high priority for the lowest of those IRQ's.
   
   
   
   
   
  Q: HOW CAN WE FIX THIS WITH SOFTWARE?
  
   
   
   We can't.
   
   We're limited by the architecture of the PC and its interrupt
   controllers. We must change the IRQ of a device by physical
   restrapping--we can't do it by reprogramming the priority alone. 
   
   We'll go back to the earlier example. We'll wave a magic wand and
   Poof!--assume we just restrapped the serial multiplexer to IRQ 5:
   
     * /sbin/irqtune 3
       
     * The resulting priorities would be:
       

SORTED BY IRQ:                          SORTED BY PRIORITY:

I00/P05:   8578913   timer              I03/P00:     86470 + serial

I01/P06:    109547   keyboard           I04/P01:    197648 + serial

I02/P07:         0 + cascade            I05/P02:    197648 + sermux

I03/P00:     86470 + serial             I00/P05:   8578913   timer

I04/P01:    197648 + serial             I01/P06:    109547   keyboard

I05/P02:    197648 + sermux             I02/P07:         0 + cascade

I12/P13:     17968 + eth                I14/P07:     93123 + Ux4F

I13/P14:         1   math error         I12/P13:     17968 + eth

I14/P07:     93123 + Ux4F               I13/P14:         1   math error

   
   
   
   
   
   
  Q: COULD WE DO MORE OF THIS RESTRAPPING, SAY, WITH THE ETHERNET CONTROLLER?
  
   
   
   Sure. Waving the wand again, we restrap the ethernet card to IRQ 6.
   
     * /sbin/irqtune 3
       
     * The resulting priorities would be:
       

SORTED BY IRQ:                          SORTED BY PRIORITY:

I00/P05:   8578913   timer              I03/P00:     86470 + serial

I01/P06:    109547   keyboard           I04/P01:    197648 + serial

I02/P07:         0 + cascade            I05/P02:    197648 + sermux

I03/P00:     86470 + serial             I06/P03:     17968 + eth

I04/P01:    197648 + serial             I00/P05:   8578913   timer

I05/P02:    197648 + sermux             I01/P06:    109547   keyboard

I06/P03:     17968 + eth                I02/P07:         0 + cascade

I13/P14:         1   math error         I14/P07:     93123 + Ux4F

I14/P07:     93123 + Ux4F               I13/P14:         1   math error

   
   
   
   
   
   
  SO IN ORDER TO GET THE BEST OVERALL SYSTEM, WE MAY NEED TO CHANGE IRQ
  PRIORITY AND PHYSICALLY CHANGE THE HARDWARE IRQ CONFIGURATION?
  
   
   
   Exactly.
   
   Different systems may have highly different criteria for what is
   optimum. It is, ultimately, a choice that each system administrator
   must make based upon the specific requirements for the particular
   system in question. We can only provide tools to do the job, but the
   final choice is ultimately decided on a case-by-case basis. There is
   no one size fits all solution.
   
   
   
   
   
   
     _________________________________________________________________
   
             WHAT ABOUT INCOMPATIBILITIES WITH KERNEL REVISIONS?
                                       
   
   
   
   
   
   
  Q: WHAT ABOUT IRQ SHARING OF SERIAL PORTS?
  
   
   
   Under the 2.0.X kernels, use of IRQ sharing will defeat IRQ priority
   because the serial port ISR's are installed as slow rather than fast
   interrupts (e.g. they don't use the SA_INTERRUPT flag).
   
   
   
     * Change the serial port IRQ configuration so that ports don't share
       IRQ's
       
     * The other option is a kernel source patch to
       drivers/char/serial.c:
       
       Change the following line:
       

#define IRQ_T(info) ((info->flags & ASYNC_SHARE_IRQ) ? SA_SHIRQ : SA_INTERRUPT)
   and replace it with:

#define IRQ_T(info) (((info->flags & ASYNC_SHARE_IRQ) ? SA_SHIRQ : 0) | SA_INTE
RRUPT)

   
   
   Under earlier kernels this is not a problem because the serial ISR was
   always installed with SA_INTERRUPT.
   
   
   
   
   
  Q: ARE THERE ANY KERNEL REVISIONS THAT IRQTUNE WON'T WORK WITH?
  
   
   
   When irqtune was first released, some experimental changes were made
   to the kernel to solve the IRQ priority problem by use of a
   round-robin, balanced priority system that was incompatible with
   irqtune. These changes were ultimately removed but irqtune will not
   work with kernel revisions 2.0.15 to 2.0.18.
   
   
   
   
   
  Q: WHAT IS THE DIFFERENCE BETWEEN IRQTUNE_MOD.O AND IRQTUNE_NPR.O?
  
   
   
   The only kernel symbol that irqtune's kernel module (irqtune_mod.o)
   uses is printk (to print a confirmation message to syslog). Some
   kernels that use MODVERSIONS have some difficulty loading
   irqtune_mod.o.
   
   The irqtune_npr.o module is exactly the same as irqtune_mod.o except
   that it does not use printk. irqtune first trys to load irqtune_mod.o
   and falls back to irqtune_npr.o if it detects a loader error. Since
   irqtune pre-checks all parameters before attempting to load the
   kernel module, the confirmation message is a nicety but not a
   necessity.
   
   
   
   
   
   
     _________________________________________________________________
   
                    WHAT ABOUT HARDWARE/CONFIG PROBLEMS?
                                       
   
   
   
   
   
   
  Q: WHAT IF THE SERIAL PORT DOESN'T WORK?
     * The First Law of Computing: Make sure it's plugged in.
       
     * Double check the strapping for port and IRQ assignments. Verify
       that the port/IRQ desired is actually the one that is strapped.
       Some serial ports will appear to work even if the IRQ is
       wrong--they will just be extremely slow. Some Plug-N-Play (PnP)
       ports may need to be explicitly strapped.
       
     * Check for port/IRQ conflicts with other devices. Serial ports may
       share IRQ assignments but sharing an IRQ between a serial port and
       something else (e.g. an ethernet card) is fraught with peril. See
       the previous section about serial IRQ sharing under 2.0.X kernels.
       
       
     * Verify that the IRQ that the kernel reports as a result of
       auto-detection is the IRQ that was strapped. Some serial ports or
       hardware configurations can fool the auto-detect. If necessary,
       use the setserial program (in /etc/rc.d/rc.local) to force the
       kernel port/IRQ settings to match the hardware strapping.
       
     * Verify that irqtune got the correct IRQ numbers for the specific
       configuration. For example, suppose our primary serial port is on
       IRQ 3, but we gave irqtune the value 4. We just made the serial
       device on IRQ 3 into priority 14, not priority 0.
       
   
   
   
   
   
   
  Q: WHAT IF PPP DOESN'T WORK?
  
   
   
   See the PPP man page and PPP-Howto for best information, but some
   recommended options:
     * asyncmap 0
     * crtscts
     * defaultroute
     * modem
     * mru 296 mtu 296
     * passive
       
   
   
   AT-like modems have a special character to escape from data mode to
   command mode. To avoid confusion here, we'll call this modem escape
   character a guard character.
   
   The default guard character is '+' (decimal 43, hex 2B). Normally, 3
   such characters are required within a special timing sequence.
   
   Although it is unlikely, it is still possible that some PPP packets
   could generate the guard sequence inadvertantly. To prevent this, we
   may want to inhibit the generation of the guard character in a data
   sequence. To do this, we would add the additional PPP option:
     * escape 2B
       
   
   
   Since '+' is a common ASCII character (PPP escaped characters generate
   two characters), we may wish to use a less common value for the guard
   character. For example, a less common value might be (decimal 200, hex
   C8). We would add an ATS2=200 command to our modem dialer script and
   change the PPP escape option to C8.
   
   
   
   
   
   
     _________________________________________________________________
   
      WHAT ABOUT OTHER PERFORMANCE SOFTWARE REMEDIES I'VE HEARD ABOUT?
                                       
   
   
   
   
   
   
  Q: WHAT ABOUT USING HDPARM -U TO SET THE INTERRUPT-UNMASK FLAG IN THE HARD
  DISK DRIVER?
  
   
   
   This is only necessary for the IDE driver. The SCSI driver has short
   disable windows by default. This will shorten the IDE interrupt
   disable windows.
   
   Beware: Without this option, IDE disk activity will almost certainly
   cause serial data dropouts.
   
   
   
   
   
  Q: WHAT ABOUT DISABLING VAN JACOBSEN HEADER COMPRESSION IN PPP?
  
   
   
   This reduces the amount of bottom-half processing the system has to do
   at the expense of larger packets being sent. This may be helpful on
   slower CPU's or heavily loaded configurations.
   
   
   
   
   
  Q: WHAT ABOUT ADJUSTING THE MRU/MTU NUMBERS IN PPP?
  
   
   
   Reducing the MRU/MTU to a minimum (296) reduces the bottom-half
   processing and flip-buffer latency at the expense of adding extra
   overhead bytes due to the reduced packet size. The optimal value will
   vary from configuration to configuration.
   
   Beware: Start with 296 as the optimal may not be 1500.
   
   The flip-buffer is a double buffer mechanism in the serial/tty drivers
   through which all data must pass. It has a fixed size of only 512
   bytes. MRU/MTU greater than the flip-buffer size may create an
   internal race condition that may cause dropouts on slower CPU's or
   heavily loaded configurations. 
   
   
   
   
   
  Q: WHAT ABOUT GOING TO NEWER KERNEL REVISIONS?
  
   
   
   Although irqtune will work surprisingly well with just about any
   kernel revision, the low level IRQ handlers and device drivers have
   been vastly improved in the 2.0.X kernels. This will only improve
   irqtune's effect.
   
   
   
   
   
   
     _________________________________________________________________
   
           HOW CAN I TELL IF IRQTUNE ACTUALLY DID ANYTHING FOR ME?
                                       
   
   
   Well, first off, if PPP/SLIP was dying mysteriously, it will probably
   be more reliable.
   
   
   
   
   
  Q: HOW CAN WE BENCHMARK IRQTUNE?
  
   Run without it and get a feel for the transfer rate:
   
     * Hit many favorite web sites and note the transfer rates in
       bytes/second. Make life easy. Netscape is at least one browser
       that reports transfer rates in bytes/second in the status line.
       
     * FTP reports the transfer time of a file in bytes/second. Download
       (or upload) a few files (300K or greater to smooth out the
       benchmark) and note the transfer rates. Use a mix of ascii and
       binary files that to see the effect of modem compression. Binary
       won't compress too well so the numbers will reflect the real
       transfer rate. Ascii files which typically compress 2:1 will up
       the effective rate by 2X, making it more stressful.
       
     * Try several things of varying duration, different times of day,
       different sites to accomodate variations in network loading. Don't
       stop until there is an average set of numbers that are more or
       less repeatable.
       
   
   
   Repeat this using irqtune and note the transfer times again.
   
   NOTE: IRQTUNE just won't quit--if you want to test in the original
   mode again, reboot the system first.
   
   
   
   
   
  Q: WHAT IF WE STILL DON'T SEE ANY REAL IMPROVEMENT?
  
   
   
   It's a matter of probability. Performance measurement is as much art
   as science.
   
     * We're much more likely to see improvement on a DX2/66 than a
       Pentium/166. With the 166, we may be overpowering the problem.
       We'll still have a problem, we just won't notice it as much
       because the Pentium has the extra speed to burn. A badly tuned
       Ferrari may still outperform a well-tuned VW :-)
       
     * What problem was occurring before? Which of the symptoms listed
       earlier is happening? If performance was 2500 bytes/second before
       using irqtune, we're less likely to notice the smaller jump to,
       say, 2800.
       
     * System loading is very light. The problems that irqtune will fix
       are more likely to happen when more devices and more work are
       added. On a certain configuration, irqtune had little effect.
       However, when a brand new SCSI DAT drive was added, serial
       performance nosedived. Using irqtune brought performance back to
       the correct level.
       
     * We may have a rogue interrupt service that disables interrupts for
       something outrageous, say 2 ms. This is singularly longer than the
       1194 us. in the earlier example. irqtune will still help, but the
       real solution here is to reduce interrupt lockout times in the
       other device below the 1194 us. threshold.
       
   
   
   
   
   
   
   
     _________________________________________________________________
   
                                   CHANGES
                                       
   
   
   
   
   
   
  REVISION 0.2 CHANGES:
     * No code changes
     * Major rewrite and expansion of the problem explanation section
     * More thorough explanation of how and why irqtune works
     * Explanation of why serial devices must be highest priority
     * Impact on other devices
     * Cleaner and better installation instructions
     * Better benchmarking section
     * Problem resolution section
     * Explanation of my prior misread on the EOI thing
       
   
   
   
   
   
   
  REVISION 0.3 CHANGES:
     * irqtune automatically prints interrupt priority table
     * Significantly improved error detection and reporting
     * Improved loading under different kernel revisions
     * insmod now invoked with -x to improve loading
     * Added module without printk's -- irqtune_npr.o
     * Special makefile options for difficult or custom rebuilds
     * Corrected error in hdparm -u note.
     * Improved the non-standard configuration section
     * Added note and patch about IRQ sharing under 2.0.X kernels
       
   
   
   
   
   
   
  REVISION 0.4 CHANGES:
     * No code changes
     * Updated links in the FAQ to reflect configuration changes at my
       ISP, best.com. Specifically, the FTP address changed from
       ftp://www.best.com/pub/cae/irqtune.tgz to
       ftp://ftp.best.com/pub/cae/irqtune.tgz
     * irqtune distribution now available via HTTP as well as FTP.
     * Consolidated and improved trouble resolution sections.
     * Added explanation of irqtune_mod.o vs. irqtune_npr.o.
     * Added explanation of /sbin usage.
     * Corrected some typos.
       
   
   
   
   
   
   
  REVISION 0.5 CHANGES:
     * Added alternate installation procedure with "INSTALL=" option.
     * /sbin is added to $PATH automatically.
     * Added "-u" option.
     * Consolidated and improved installation documentation.
     * Additional trouble resolution documentation.
     * Again, updated links in the FAQ to reflect configuration changes
       at my ISP, best.com. Specifically, the FTP address changed from
       ftp://ftp.best.com/pub/cae/irqtune.tgz to
       ftp://shell5.ba.best.com/pub/cae/irqtune.tgz
       
   
   
   
   
   
   
   
     _________________________________________________________________
   
                              TABLE OF CONTENTS
                                       
   
           IRQTUNE -- A Linux x86 IRQ Priority Optimizer
     * Where do I get irqtune?
     * How do I know if I need irqtune?
     * What is actually happening to cause these problems?
     * How does irqtune help this?
     * Why does the serial interrupt service require the highest
       priority?
     * Isn't that example very unlikely under Linux?
     * Doesn't this hurt the performance of other devices?
     * Isn't this IRQ priority thing a bit of a new idea?
     * How do I install irqtune?
     * How do I use irqtune? Don't I have to rebuild my kernel?
     * What about my non-standard hardware configuration?
     * What about incompatibilities with kernel revisions?
     * What about hardware/config problems?
     * What about other performance software remedies I've heard about?
     * How can I tell if irqtune actually did anything for me?
     * Changes
     * Table of Contents
