[Bugme-new] [Bug 6682] New: BUG: soft lockup detected on CPU#0! / ksoftirqd takse 100% CPU

bugme-daemon at bugzilla.kernel.org bugme-daemon at bugzilla.kernel.org
Mon Jun 12 16:11:22 PDT 2006


http://bugzilla.kernel.org/show_bug.cgi?id=6682

           Summary: BUG: soft lockup detected on CPU#0! / ksoftirqd takse
                    100% CPU
    Kernel Version: 2.6.15.6
            Status: NEW
          Severity: normal
             Owner: acme at conectiva.com.br
         Submitter: alchemyx at uznam.net.pl


Most recent kernel where this bug did not occur: (unknown)
Distribution: Gentoo
Hardware Environment: 2x Xeon 2.66, 1 GB RAM, NICS: 2 x e1000, and one double
port e100. Based on Intel E7501 architecture (2U rack Intel chassis).
Software Environment: quagga 0.98.6
Problem Description: ksoftirqd/0 takes 100% of CPU. further investigation shows
no sing of network flood or something (and also 2 of 3 NICs are e1000 with
NAPI). Ocassionaly there are "BUG: soft lockup detected on CPU#0!".


Steps to reproduce:

There is no simple way to reproduce. I think that everythint started when we
attached second provider with BGP support. We are using quagga which injects
about 186 000 routes into kernel. When running for a while (at least few hours,
sometimes a day) we get 100% usage on ksoftirqd/0 and following messages in logs:

BUG: soft lockup detected on CPU#0!

Pid: 6506, comm:                zebra
EIP: 0060:[<c027f6fd>] CPU: 0
EIP is at _spin_lock+0x7/0xf
 EFLAGS: 00000286    Not tainted  (2.6.15.6)
EAX: f6203180 EBX: e6fbf000 ECX: 00000000 EDX: f6bec000
ESI: f6203000 EDI: eddb4b80 EBP: fffffff4 DS: 007b ES: 007b
CR0: 8005003b CR2: aca6dff0 CR3: 361ad000 CR4: 000006d0
 [<c02396f9>] dev_queue_xmit+0xe0/0x203
 [<c0250de8>] ip_output+0x1e1/0x237
 [<c024f3f5>] ip_forward+0x181/0x1df
 [<c024e21a>] ip_rcv+0x40c/0x485
 [<c0239bd0>] netif_receive_skb+0x12f/0x165
 [<f885aa4c>] e1000_clean_rx_irq+0x389/0x410 [e1000]
 [<f885a1ca>] e1000_clean+0x94/0x12f [e1000]
 [<c0239d5a>] net_rx_action+0x69/0xf0
 [<c011a305>] __do_softirq+0x55/0xbd
 [<c011a39a>] do_softirq+0x2d/0x31
 [<c011a3f8>] local_bh_enable+0x5a/0x65
 [<c024a0a1>] rt_run_flush+0x5f/0x80
 [<c027623f>] fn_hash_insert+0x352/0x39f
 [<c027364c>] inet_rtm_newroute+0x57/0x62
 [<c02413ed>] rtnetlink_rcv_msg+0x1a8/0x1cb
 [<c0241245>] rtnetlink_rcv_msg+0x0/0x1cb
 [<c0247c1e>] netlink_rcv_skb+0x3a/0x8b
 [<c0247cb1>] netlink_run_queue+0x42/0xc3
 [<c0241245>] rtnetlink_rcv_msg+0x0/0x1cb
 [<c0241245>] rtnetlink_rcv_msg+0x0/0x1cb
 [<c0241227>] rtnetlink_rcv+0x22/0x40
 [<c0241245>] rtnetlink_rcv_msg+0x0/0x1cb
 [<c024764c>] netlink_data_ready+0x17/0x54
 [<c0246a99>] netlink_sendskb+0x1f/0x39
 [<c0247449>] netlink_sendmsg+0x27b/0x28c
 [<c0231467>] sock_sendmsg+0xce/0xe9
 [<c0112b36>] __wake_up+0x27/0x3b
 [<c01a6216>] copy_to_user+0x38/0x42
 [<c01a625a>] copy_from_user+0x3a/0x60
 [<c01a625a>] copy_from_user+0x3a/0x60
 [<c0126be2>] autoremove_wake_function+0x0/0x3a
 [<c0236bcd>] verify_iovec+0x49/0x7f
 [<c02327f2>] sys_sendmsg+0x152/0x1a8
 [<c0147a62>] do_sync_read+0xb8/0xeb
 [<c01a6216>] copy_to_user+0x38/0x42
 [<c0126be2>] autoremove_wake_function+0x0/0x3a
 [<c0122b7a>] getrusage+0x34/0x43
 [<c0168504>] inotify_dentry_parent_queue_event+0x29/0x7c
 [<c01a625a>] copy_from_user+0x3a/0x60
 [<c0232b6b>] sys_socketcall+0x167/0x180
 [<c0102433>] sysenter_past_esp+0x54/0x75

BUG: soft lockup detected on CPU#0!

Pid: 6506, comm:                zebra
EIP: 0060:[<f8952052>] CPU: 0
EIP is at u32_classify+0x52/0x170 [cls_u32]
 EFLAGS: 00000206    Not tainted  (2.6.15.6)
EAX: e2fbd020 EBX: f48649c0 ECX: 00000010 EDX: 29b09d5a
ESI: f48649ec EDI: 00000001 EBP: e2fbd020 DS: 007b ES: 007b
CR0: 8005003b CR2: 08154004 CR3: 361ad000 CR4: 000006d0
 [<f88462fa>] ipt_do_table+0x2de/0x2fd [ip_tables]
 [<f883b523>] ip_nat_fn+0x177/0x185 [iptable_nat]
 [<f88e159f>] ip_refrag+0x23/0x5f [ip_conntrack]
 [<c0244d82>] tc_classify+0x2c/0x3f
 [<f895514b>] htb_classify+0x14b/0x1dd [sch_htb]
 [<f8955638>] htb_enqueue+0x1d/0x13a [sch_htb]
 [<c02396fd>] dev_queue_xmit+0xe4/0x203
 [<c0250de8>] ip_output+0x1e1/0x237
 [<c024f3f5>] ip_forward+0x181/0x1df
 [<c024e21a>] ip_rcv+0x40c/0x485
 [<c0239bd0>] netif_receive_skb+0x12f/0x165
 [<f885aa4c>] e1000_clean_rx_irq+0x389/0x410 [e1000]
 [<f885a1ca>] e1000_clean+0x94/0x12f [e1000]
 [<c0239d5a>] net_rx_action+0x69/0xf0
 [<c011a305>] __do_softirq+0x55/0xbd
 [<c011a39a>] do_softirq+0x2d/0x31
 [<c011a3f8>] local_bh_enable+0x5a/0x65
 [<c024a0a1>] rt_run_flush+0x5f/0x80
 [<c027623f>] fn_hash_insert+0x352/0x39f
 [<c027364c>] inet_rtm_newroute+0x57/0x62
 [<c02413ed>] rtnetlink_rcv_msg+0x1a8/0x1cb
 [<c0241245>] rtnetlink_rcv_msg+0x0/0x1cb
 [<c0247c1e>] netlink_rcv_skb+0x3a/0x8b
 [<c0247cb1>] netlink_run_queue+0x42/0xc3
 [<c0241245>] rtnetlink_rcv_msg+0x0/0x1cb
 [<c0241245>] rtnetlink_rcv_msg+0x0/0x1cb
 [<c0241227>] rtnetlink_rcv+0x22/0x40
 [<c0241245>] rtnetlink_rcv_msg+0x0/0x1cb
 [<c024764c>] netlink_data_ready+0x17/0x54
 [<c0246a99>] netlink_sendskb+0x1f/0x39
 [<c0247449>] netlink_sendmsg+0x27b/0x28c
 [<c0231467>] sock_sendmsg+0xce/0xe9
 [<c0112b36>] __wake_up+0x27/0x3b
 [<c01a625a>] copy_from_user+0x3a/0x60
 [<c01a625a>] copy_from_user+0x3a/0x60
 [<c0126be2>] autoremove_wake_function+0x0/0x3a
 [<c0236bcd>] verify_iovec+0x49/0x7f
 [<c02327f2>] sys_sendmsg+0x152/0x1a8
 [<c0147a62>] do_sync_read+0xb8/0xeb
 [<c0122b7a>] getrusage+0x34/0x43
 [<c011d72f>] update_wall_time+0xa/0x32
 [<c011d9e5>] do_timer+0x33/0xa9
 [<c01a625a>] copy_from_user+0x3a/0x60
 [<c0232b6b>] sys_socketcall+0x167/0x180
 [<c0102433>] sysenter_past_esp+0x54/0x75

It happens on 2.6.15.6. Tonight I will try 2.6.16.16 with FIB_TRIE instead of
FIB_HASH. I am submitting that bug under network but I am not completly sure if
it belongs here.

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.



More information about the Bugme-new mailing list