my package of the day: irqbalance

Maybe you are in the nice situation of running machine with multiple cpu cores having to crunch a lot of numbert or providing network daemons. Multiple cpu cores can boost your performance dramatically but there are, of course, possible issues. On is the fact, that interrupts by default are mainly called by the first cpu. As applications that aren’t able to thread correctly can stick to this cpu you might notice by a „cat /proc/interrupts“ that something goes wrong. The following is a (compressed) /proc/interrupts from a live server running for about ten weeks:

           CPU0       CPU1
  0: 1855681242        574  timer
  7:          0          0  parport0
  8:          1          0  rtc
  9:          1          0  acpi
 14:         65          0  ide0
 58:          4          0  ehci_hcd:usb1, uhci_hcd:usb2
 66:  212905125         72  3w-xxxx
 74: 1094755762          0  eth0
185:    6223686          0  uhci_hcd:usb4, eth1
193:       1978          0  uhci_hcd:usb3, libata

You can see, that actually all interrupts are called by CPU0. We want to brush this up! How? Just run a „aptitude install irqbalance“ as irqbalance promises:

Daemon to balance interrupts across multiple CPUs, which can lead to better performance and IO balance on SMP systems. This package is especially useful on systems with multi-core processors, as interrupts will typically only be serviced by the first core.

So let’s check after about one week by another „cat /proc/interrupts“:

           CPU0       CPU1
  0: 1887385089   33827155     timer
  7:          0          0     parport0
  8:          1          0     rtc
  9:          1          0     acpi
 14:         65          0     ide0
 58:          4          0     ehci_hcd:usb1, uhci_hcd:usb2
 66:  212950265   11810501     3w-xxxx
 74: 1310191290          0     eth0
169:          0          0     uhci_hcd:usb5
185:    6223686     228881     uhci_hcd:usb4, eth1
193:       1978          0     uhci_hcd:usb3, libata

Nice, isn’t it? CPU1 started to grab interrupts also. If we would reboot the server, most of the irqs would look balanced over time. (most, not all)

Please notice:

Before installing irqbalance, check your /proc/interrupts. It might be possible, that you don’t need it though you have multiple cores as there is a value „CONFIG_IRQBALANCE“ in 2.6 kernels that can be turned on.

[update]

The comments (thank you!) pointed out the following:

  • There are reports on crashed systems using irqbalance. (Though I have never seen anyone by myself)
  • Note that irqbalance is not in main – if you are using it on an important server.
  • CONFIG_IRQBALANCE seems to be enabled in Bbuntu Hardy by default.
  • There are discussions about removing CONFIG_IRQBALANCE as it is said that irqbalance is more reliable.

So it is up to you to decice which one to use!

[update2]

Actually a glance on  /boot/config-2.6.24-17-generic shows, that CONFIG_IRQBALANCE seems not to be enabled in Hardy though the balancing seems to work. Actually I am not one of the kernel guys so my investigation will take it’s time. Any hints welcome (thank you lissyx).