[Bugme-new] [Bug 10890] New: Heavy iSCSI traffic triggers memory allocation complaints and system lockup

bugme-daemon at bugzilla.kernel.org bugme-daemon at bugzilla.kernel.org
Tue Jun 10 02:56:46 PDT 2008


http://bugzilla.kernel.org/show_bug.cgi?id=10890

           Summary: Heavy iSCSI traffic triggers memory allocation
                    complaints and system lockup
           Product: Other
           Version: 2.5
     KernelVersion: 2.6.25.6
          Platform: All
        OS/Version: Linux
              Tree: Mainline
            Status: NEW
          Severity: high
          Priority: P1
         Component: Other
        AssignedTo: other_other at kernel-bugs.osdl.org
        ReportedBy: bart.vanassche at gmail.com


Latest working kernel version: (not known)
Earliest failing kernel version: fails at least with 2.6.24.3-ubuntu and
2.6.25.6
Distribution: Ubuntu Linux 8.04 server
Hardware Environment: x86_64
Software Environment:
Problem Description:

System starts behaving strange after about an hour of heavy iSCSI traffic: the
console locks up (keys typed in are not echoed) and error messages are logged
about problems with memory allocation.

Steps to reproduce:

This is how my setup looks like:
- One server is running OpenSolaris build 90 and there have been 64 iSCSI
targets configured on this system, where each target is 10 GB in size.
- On the second server I have installed Ubuntu 8.04 server 64-bit, and I
replaced the Ubuntu kernel by the 2.6.25.6 kernel.
- Both systems are connected via a 4x SDR InfiniBand network (10 Gbit/s).
- Open-iscsi was configured on the Linux server such that it connects to all 64
iSCSI targets and such that I/O to all 64 targets was active simultaneously. I
used the following command (see below for the scripts):
  ./iscsi-logout-and-login && ./iscsi-stress-test &

# cat iscsi-logout
#!/bin/bash
target_ip="${1:-192.168.67.10}"
iscsiadm -m session \
  | while read proto num ip iqn
    do
      iscsiadm -m node -p ${ip} -T $iqn --logout
    done

# cat iscsi-logout-and-login
#!/bin/bash
target_ip=192.168.67.10
./iscsi-logout "${target_ip}"
i=0
iscsiadm -m discovery -t st -p ${target_ip} \
  | grep '\.67\.' \
  | cut -f2 -d' ' \
  | sort -k 1.60 \
  | while read iqn
    do
      if [ "$1" != "" -a "$i" -ge "${1:-0}" ]; then
        break
      fi
      iscsiadm -m node -p ${target_ip} -T ${iqn} --login
      iscsiadm --mode node -p ${target_ip} -T ${iqn} \
        --op update -n "node.session.timeo.replacement_timeout" -v 10
      iscsiadm --mode node -p ${target_ip} -T ${iqn} \
        --op update -n "node.conn[0].timeo.noop_out_interval" -v 0
      iscsiadm --mode node -p ${target_ip} -T ${iqn} \
        --op update -n "node.conn[0].timeo.noop_out_timeout" -v 0
      i=$((i+1))
    done

# cat iscsi-stress-test
#!/bin/bash
while true
do
  date
  for d in /dev/disk/by-path/ip-*:3260-iscsi-iqn.1986-03.com.sun*
  do
    dd if=/dev/zero of=$d bs=100K &
  done
  wait
done


-- 
Configure bugmail: http://bugzilla.kernel.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.


More information about the Bugme-new mailing list