[vserver] opteron server dies with vserver patch.

From: Pawel Sikora <pluto_at_pld-linux.org>
Date: Mon 08 Aug 2011 - 12:24:48 BST
Message-ID: <1573855.vSOPxPpqde@pawels>

Hi,

i have few dual opteron-6128 machines which are part of eda software auto-test farm.
these machines don't host any vserver guests but installed kernel has vserver-patch
applied by vendor. on vanilla kernel 2.6.38 (and newer) with vserver patch applied
machines lock in some way after few hours of quite heavy computing + i/o (avg. load ~16).
on 2.6.37.y with vserver everything works fine and on pure vanilla kernels also everything works fine.

on the ipmi console i've logged only kernel flood: https://lkml.org/lkml/2011/5/23/398
this crash is reproducible on 2.6.38.y/2.6.39.y/3.0.y with the latest vserver patches.

# cat /proc/version
Linux version 3.0.0-vs2.3.1-pre8-dirty (pawels@hal) (gcc version 4.5.3 20110515 (release) (PLD-Linux) ) #2 SMP Fri Aug 5 09:48:25 CEST 2011

# cat /proc/cpuinfo

processor : 0...15
vendor_id : AuthenticAMD
cpu family : 16
model : 9
model name : AMD Opteron(tm) Processor 6128
stepping : 1
cpu MHz : 2000.149
cache size : 512 KB
physical id : 1
siblings : 8
core id : 3
cpu cores : 8
apicid : 39
initial apicid : 23
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext
3dnow constant_tsc rep_good nopl nonstop_tsc extd_apicid amd_dcm pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs
skinit wdt nodeid_msr npt lbrv svm_lock nrip_save pausefilter
bogomips : 4001.84
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate

# cat /proc/modules
ext2 72951 0 - Live 0xffffffffa04d8000
loop 27624 1 - Live 0xffffffffa04ec000
nfs 312073 1 - Live 0xffffffffa048a000
fscache 46134 1 nfs, Live 0xffffffffa0476000
binfmt_misc 17456 1 - Live 0xffffffffa03fa000
nfsd 310366 13 - Live 0xffffffffa0429000
lockd 84640 2 nfs,nfsd, Live 0xffffffffa0413000
nfs_acl 12741 2 nfs,nfsd, Live 0xffffffffa03f5000
auth_rpcgss 47785 2 nfs,nfsd, Live 0xffffffffa0406000
sunrpc 216671 22 nfs,nfsd,lockd,nfs_acl,auth_rpcgss, Live 0xffffffffa03bf000
exportfs 12851 1 nfsd, Live 0xffffffffa03a6000
ipmi_si 52811 0 - Live 0xffffffffa03b1000
ipmi_devintf 17522 0 - Live 0xffffffffa03ab000
ipmi_msghandler 43683 2 ipmi_si,ipmi_devintf, Live 0xffffffffa039a000
sch_sfq 17212 2 - Live 0xffffffffa038d000
iptable_nat 13182 1 - Live 0xffffffffa0395000
nf_nat 24588 1 iptable_nat, Live 0xffffffffa037e000
nf_conntrack_ipv4 19478 3 iptable_nat,nf_nat, Live 0xffffffffa0387000
nf_conntrack 76827 3 iptable_nat,nf_nat,nf_conntrack_ipv4, Live 0xffffffffa036a000
nf_defrag_ipv4 12673 1 nf_conntrack_ipv4, Live 0xffffffffa0365000
iptable_filter 12810 0 - Live 0xffffffffa0360000
xt_TCPMSS 12715 2 - Live 0xffffffffa02f0000
xt_tcpudp 12603 2 - Live 0xffffffffa02e2000
iptable_mangle 12695 1 - Live 0xffffffffa02dd000
ip_tables 26705 3 iptable_nat,iptable_filter,iptable_mangle, Live 0xffffffffa02d1000
ip6table_filter 12815 0 - Live 0xffffffffa0217000
ip6_tables 26996 1 ip6table_filter, Live 0xffffffffa027a000
x_tables 27634 8 iptable_nat,iptable_filter,xt_TCPMSS,xt_tcpudp,iptable_mangle,ip_tables,ip6table_filter,ip6_tables, Live 0xffffffffa0262000
ext4 403044 1 - Live 0xffffffffa02fc000
jbd2 84403 1 ext4, Live 0xffffffffa02bb000
crc16 12559 1 ext4, Live 0xffffffffa0212000
raid0 17106 1 - Live 0xffffffffa0245000
dm_mod 75794 0 - Live 0xffffffffa0296000
autofs4 28311 11 - Live 0xffffffffa01e6000
dummy 12955 0 - Live 0xffffffffa01a0000
usbhid 46436 0 - Live 0xffffffffa02ae000
hid 93560 1 usbhid, Live 0xffffffffa01fa000
ide_cd_mod 35415 0 - Live 0xffffffffa01dc000
cdrom 41847 1 ide_cd_mod, Live 0xffffffffa01d0000
ata_generic 12899 0 - Live 0xffffffffa019b000
pata_acpi 13027 0 - Live 0xffffffffa0186000
pata_atiixp 13165 0 - Live 0xffffffffa015d000
sp5100_tco 13697 0 - Live 0xffffffffa014f000
ide_pci_generic 12866 0 - Live 0xffffffffa014a000
ohci_hcd 31523 0 - Live 0xffffffffa017d000
psmouse 68641 0 - Live 0xffffffffa0284000
ghes 13625 0 - Live 0xffffffffa01cb000
ssb 56258 1 ohci_hcd, Live 0xffffffffa026b000
mmc_core 85395 1 ssb, Live 0xffffffffa024c000
igb 115008 0 - Live 0xffffffffa0227000
i2c_piix4 13167 0 - Live 0xffffffffa0158000
amd64_edac_mod 28622 0 - Live 0xffffffffa01f2000
pcmcia 43683 1 ssb, Live 0xffffffffa0171000
dca 14562 1 igb, Live 0xffffffffa00de000
pcspkr 12702 0 - Live 0xffffffffa00e3000
i2c_core 28557 1 i2c_piix4, Live 0xffffffffa0142000
hed 13102 1 ghes, Live 0xffffffffa01a9000
edac_core 51596 6 amd64_edac_mod, Live 0xffffffffa0134000
button 13611 0 - Live 0xffffffffa011c000
pcmcia_core 21861 1 pcmcia, Live 0xffffffffa02f5000
atiixp 12917 0 - Live 0xffffffffa02eb000
processor 30161 0 - Live 0xffffffffa021e000
ide_core 108026 3 ide_cd_mod,ide_pci_generic,atiixp, Live 0xffffffffa01af000
serio_raw 13166 0 - Live 0xffffffffa012f000
ehci_hcd 44738 0 - Live 0xffffffffa018f000
edac_mce_amd 22882 1 amd64_edac_mod, Live 0xffffffffa016a000
k10temp 13119 0 - Live 0xffffffffa0162000
hwmon 12886 1 k10temp, Live 0xffffffffa0117000
sg 36330 0 - Live 0xffffffffa0125000
usbcore 159860 4 usbhid,ohci_hcd,ehci_hcd, Live 0xffffffffa00ee000
evdev 17753 0 - Live 0xffffffffa00e8000
sd_mod 44764 16 - Live 0xffffffffa00d2000
crc_t10dif 12483 1 sd_mod, Live 0xffffffffa0083000
raid1 30545 1 - Live 0xffffffffa00ac000
md_mod 106323 4 raid0,raid1, Live 0xffffffffa00b5000
ext3 138195 1 - Live 0xffffffffa0089000
jbd 55191 1 ext3, Live 0xffffffffa0074000
mbcache 14317 3 ext2,ext4,ext3, Live 0xffffffffa0031000
ahci 25854 12 - Live 0xffffffffa006c000
libahci 25686 1 ahci, Live 0xffffffffa0029000
libata 197408 5 ata_generic,pata_acpi,pata_atiixp,ahci,libahci, Live 0xffffffffa003a000
scsi_mod 161130 3 sg,sd_mod,libata, Live 0xffffffffa0000000

# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md0 7.6G 3.6G 3.7G 50% /
/dev/md1 1.9T 122G 1.8T 7% /home/atest
none 32G 0 32G 0% /dev/shm
nexus:/R0 1.8T 531G 1.2T 31% /remote/nexus/R0

# cat /proc/mdstat
Personalities : [raid1] [raid0]
md1 : active raid0 sda3[0] sdd3[3] sdc3[2] sdb3[1]
      2047993856 blocks super 1.2 512k chunks

md0 : active raid1 sda1[0] sdd1[3] sdc1[2] sdb1[1]
      8000256 blocks [4/4] [UUUU]

# /bin/bash testme.sh
Linux-VServer Test [V0.17] Copyright (C) 2003-2006 H.Poetzl
utility 'vserver' could not be found.

# ./testfs.sh -t -x -y -z -D /dev/loop0 -M /mnt
Linux-VServer FS Test [V0.23] Copyright (C) 2005-2009 H.Poetzl
Linux 3.0.0-vs2.3.1-pre8-dirty x86_64/
VCI: 0002:0308 236 13011f11 (ID24)

---
testing ext2 filesystem ...
[000]. [001]. [002]. (ext2 format)
tag related tests ...
[011]. [012]* [014]* [015]* [019]. 
[020]* [021]. [022]* [023]. [024]* [025]* [026]* [027]* [028]* 
[033]* [034]* [035]* [037]* [045]* [047]* 
xattr related tests ...
[101]. [103]* [104]* [106]* [107]* [109]
[112]* [113]* [114]. [115]* [116]* [118]* [119]*
[122]* [123]* [124]* [125]. [127]* [128]* [129]*
[131]* [132]. [133]* [134]* [135]* [138]. [139].
[148]. [149].
disk limit related tests ...
[201]* [202]* [203]. [204]* [205]. [206]. [207]. [208].
[211]* [212]* [213]. [222]. [223]. [231]. [232]. [233]. [239].
[999].
---
testing ext3 filesystem ...
[000]. [001]. [002]. (ext3 format)
tag related tests ...
[011]. [012]* [014]* [015]* [019].
[020]* [021]. [022]* [023]. [024]* [025]* [026]* [027]* [028]*
[033]* [034]* [035]* [037]* [045]* [047]*
xattr related tests ...
[101]. [103]* [104]* [106]* [107]* [109]*
[112]* [113]* [114]. [115]* [116]* [118]* [119]*
[122]* [123]* [124]* [125]. [127]* [128]* [129]*
[131]* [132]. [133]* [134]* [135]* [138]. [139].
[148]. [149].
disk limit related tests ...
[201]* [202]* [203]. [204]* [205]. [206]. [207]. [208].
[211]* [212]* [213]. [222]. [223]. [231]. [232]. [233]. [239].
[999].
---
testing ext4 filesystem ...
[000]. [001]. [002]. (ext4 format)
tag related tests ...
[011]. [012]* [014]* [015]* [019].
[020]* [021]. [022]* [023]. [024]* [025]* [026]* [027]* [028]*
[033]* [034]* [035]* [037]* [045]* [047]*
xattr related tests ...
[101]. [103]* [104]* [106]* [107]* [109]*
[112]* [113]* [114]. [115]* [116]* [118]* [119]*
[122]* [123]* [124]* [125]. [127]* [128]* [129]*
[131]* [132]. [133]* [134]* [135]* [138]. [139].
[148]. [149].
disk limit related tests ...
[201]* [202]* [203]. [204]* [205]. [206]. [207]. [208].
[211]* [212]* [213]. [222]. [223]. [231]. [232]. [233]. [239].
[999].
---
testing xfs filesystem ...
[000]* (xfs format failed)
---
testing reiser filesystem ...
[000]* (reiserfs format failed)
---
testing jfs filesystem ...
[000]* (jfs format failed)
Received on Mon Aug 8 12:25:00 2011
[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Mon 08 Aug 2011 - 12:25:00 BST by hypermail 2.1.8