tools: add softirqslower

[chenyuezhou]

Summary

Adds softirqslower tool to trace softirq execution performance issues by measuring:

1. SoftIRQ Runtime: Handler execution duration
2. HardIRQ-to-SoftIRQ Latency: Delay between wakeup (trigger) and execution start

Motivation

Diagnosing network latency/stalls requires observing:

• Slow softirq executions (e.g., >10ms NET_RX processing)
• Scheduling delays where softirqs get preempted after wakeup

This closes #5353

[yonghong-song]

I tried and the below is the format:

Tracing softirq latency higher than 10 us... Hit Ctrl-C to end.

23:09:54 irq(hard) to softirq: rcu took 10 us (0 ms), on CPU:1, comm:swapper/1
23:09:54 softirq runtime: rcu took 12 us (0 ms), on CPU:1, comm:swapper/1
23:09:54 softirq runtime: net_rx took 25 us (0 ms), on CPU:1, comm:ThriftIO106
23:09:55 softirq runtime: sched took 294 us (0 ms), on CPU:1, comm:swapper/1

Two things. You need to have a header with field names and for each event, they need to be aligned properly for each field.

[chenyuezhou]

I tried and the below is the format:

Tracing softirq latency higher than 10 us... Hit Ctrl-C to end.

23:09:54 irq(hard) to softirq: rcu took 10 us (0 ms), on CPU:1, comm:swapper/1
23:09:54 softirq runtime: rcu took 12 us (0 ms), on CPU:1, comm:swapper/1
23:09:54 softirq runtime: net_rx took 25 us (0 ms), on CPU:1, comm:ThriftIO106
23:09:55 softirq runtime: sched took 294 us (0 ms), on CPU:1, comm:swapper/1

Two things. You need to have a header with field names and for each event, they need to be aligned properly for each field.

Hi @yonghong-song , I've made the adjustments per your suggestions

Tracing softirq latency higher than 10 us... Hit Ctrl-C to end.
TIME     STAGE                SOFTIRQ  LAT(us)        CPU    COMM  
14:44:09 irq(hard) to softirq rcu      16             5      swapper/5
14:44:09 softirq runtime      rcu      16             5      swapper/5
14:44:09 irq(hard) to softirq sched    16             0      swapper/0
14:44:09 softirq runtime      net_rx   12             0      ksoftirqd/0
14:44:09 softirq runtime      net_rx   38             4      python3
14:44:09 irq(hard) to softirq rcu      10             1      swapper/1
14:44:09 irq(hard) to softirq rcu      10             1      swapper/1
14:44:09 softirq runtime      net_rx   13             0      swapper/0
14:44:09 irq(hard) to softirq net_rx   15             0      swapper/0
14:44:09 softirq runtime      net_rx   14             6      swapper/6
14:44:09 softirq runtime      net_rx   18             0      swapper/0
14:44:09 irq(hard) to softirq net_rx   14             0      swapper/0
14:44:09 softirq runtime      net_rx   15             6      swapper/6