From: Brian Ipsen (Brian.Ipsen-dated-1058525216.0d6b9d_at_andebakken.dk)
Date: Fri 11 Jul 2003 - 11:46:55 BST
Hi!
This night I had a kernel crash on my vserver. The check_disk program is
from nagios_plugins, and runs on the root-server. Qmail is only installed in
one vserver/context...
The output from ksymoops (in case anyone is able to analyze it and tell me
what program or similar that caused the kernel panic):
ksymoops 2.4.5 on i686 2.4.20ctx-17smp. Options used
-V (default)
-k /proc/ksyms (default)
-l /proc/modules (default)
-o /lib/modules/2.4.20ctx-17smp/ (default)
-m /boot/System.map-2.4.20ctx-17smp (specified)
Error (expand_objects): cannot stat(/lib/ext3.o) for ext3
Error (expand_objects): cannot stat(/lib/jbd.o) for jbd
Error (expand_objects): cannot stat(/lib/lvm-mod.o) for lvm-mod
Error (expand_objects): cannot stat(/lib/dpt_i2o.o) for dpt_i2o
Error (expand_objects): cannot stat(/lib/sd_mod.o) for sd_mod
Error (expand_objects): cannot stat(/lib/scsi_mod.o) for scsi_mod
Warning (map_ksym_to_module): cannot match loaded module ext3 to a unique
module object. Trace may not be reliable.
Warning (map_ksym_to_module): cannot match loaded module dpt_i2o to a unique
module object. Trace may not be reliable.
Jul 11 02:15:41 vserver kernel: Unable to handle kernel NULL pointer
dereference at virtual address 00000035
Jul 11 02:15:41 vserver kernel: c0286b85
Jul 11 02:15:41 vserver kernel: *pde = 00000000
Jul 11 02:15:41 vserver kernel: Oops: 0002
Jul 11 02:15:41 vserver kernel: CPU: 0
Jul 11 02:15:41 vserver kernel: EIP: 0010:[<c0286b85>] Tainted: P
Using defaults from ksymoops -t elf32-i386 -a i386
Jul 11 02:15:41 vserver kernel: EFLAGS: 00010246
Jul 11 02:15:41 vserver kernel: eax: 00000000 ebx: 00000025 ecx:
00000025 edx: 40014025
Jul 11 02:15:41 vserver kernel: esi: 40014000 edi: 00000035 ebp:
f50adec0 esp: ed3d9f44
Jul 11 02:15:41 vserver kernel: ds: 0018 es: 0018 ss: 0018
Jul 11 02:15:41 vserver kernel: Process check_disk (pid: 7092,
stackpage=ed3d9000)
Jul 11 02:15:41 vserver kernel: Stack: 00000001 00000025 d9900a00 00000025
fffffff2 c015aefd 00000035 40014000
Jul 11 02:15:41 vserver kernel: 00000025 f5341f44 00000025 00000035
eb3bd680 d9900a68 00000000 00000025
Jul 11 02:15:41 vserver kernel: 00000000 f50adec0 ffffffea 00000025
c014f027 f50adec0 40014000 00000025
Jul 11 02:15:41 vserver kernel: Call Trace: [<c015aefd>] [<c014f027>]
[<c010beef>]
Jul 11 02:15:41 vserver kernel: Code: f3 aa 58 59 e9 73 68 ff ff ba f2 ff ff
ff e9 bf 68 ff ff ba
>>EIP; c0286b85 <zlib_inflate_flush+3805/3842> <=====
>>edx; 40014025 Before first symbol
>>esi; 40014000 Before first symbol
>>ebp; f50adec0 <_end+34ce8900/38447aa0>
>>esp; ed3d9f44 <_end+2d014984/38447aa0>
Trace; c015aefd <pipe_write+22d/2d0>
Trace; c014f027 <sys_write+97/140>
Trace; c010beef <system_call+33/38>
Code; c0286b85 <zlib_inflate_flush+3805/3842>
00000000 <_EIP>:
Code; c0286b85 <zlib_inflate_flush+3805/3842> <=====
0: f3 aa repz stos %al,%es:(%edi) <=====
Code; c0286b87 <zlib_inflate_flush+3807/3842>
2: 58 pop %eax
Code; c0286b88 <zlib_inflate_flush+3808/3842>
3: 59 pop %ecx
Code; c0286b89 <zlib_inflate_flush+3809/3842>
4: e9 73 68 ff ff jmp ffff687c <_EIP+0xffff687c>
Code; c0286b8e <zlib_inflate_flush+380e/3842>
9: ba f2 ff ff ff mov $0xfffffff2,%edx
Code; c0286b93 <zlib_inflate_flush+3813/3842>
e: e9 bf 68 ff ff jmp ffff68d2 <_EIP+0xffff68d2>
Code; c0286b98 <zlib_inflate_flush+3818/3842>
13: ba 00 00 00 00 mov $0x0,%edx
Jul 11 02:16:24 vserver kernel: <1>Unable to handle kernel NULL pointer
dereference at virtual address 0000080e
Jul 11 02:16:24 vserver kernel: c0286b85
Jul 11 02:16:24 vserver kernel: *pde = 00000000
Jul 11 02:16:24 vserver kernel: Oops: 0002
Jul 11 02:16:24 vserver kernel: CPU: 1
Jul 11 02:16:24 vserver kernel: EIP: 0010:[<c0286b85>] Tainted: P
Jul 11 02:16:24 vserver kernel: EFLAGS: 00010246
Jul 11 02:16:24 vserver kernel: eax: 00000000 ebx: 00000051 ecx:
00000051 edx: 08057e71
Jul 11 02:16:24 vserver kernel: esi: 08057e20 edi: 0000080e ebp:
f57c3e20 esp: f22c1f44
Jul 11 02:16:24 vserver kernel: ds: 0018 es: 0018 ss: 0018
Jul 11 02:16:24 vserver kernel: Process qmail-remote (pid: 7104,
stackpage=f22c1000)
Jul 11 02:16:24 vserver kernel: Stack: 00000001 00000051 cc5c2e00 00000051
fffffff2 c015aefd 0000080e 08057e20
Jul 11 02:16:24 vserver kernel: 00000051 00000006 00000051 0000080e
f22c1f64 cc5c2e68 00000000 00000051
Jul 11 02:16:24 vserver kernel: 00000000 f57c3e20 ffffffea 00000051
c014f027 f57c3e20 08057e20 00000051
Jul 11 02:16:24 vserver kernel: Call Trace: [<c015aefd>] [<c014f027>]
[<c010beef>]
Jul 11 02:16:24 vserver kernel: Code: f3 aa 58 59 e9 73 68 ff ff ba f2 ff ff
ff e9 bf 68 ff ff ba
>>EIP; c0286b85 <zlib_inflate_flush+3805/3842> <=====
>>edx; 08057e71 Before first symbol
>>esi; 08057e20 Before first symbol
>>edi; 0000080e Before first symbol
>>ebp; f57c3e20 <_end+353fe860/38447aa0>
>>esp; f22c1f44 <_end+31efc984/38447aa0>
Trace; c015aefd <pipe_write+22d/2d0>
Trace; c014f027 <sys_write+97/140>
Trace; c010beef <system_call+33/38>
Code; c0286b85 <zlib_inflate_flush+3805/3842>
00000000 <_EIP>:
Code; c0286b85 <zlib_inflate_flush+3805/3842> <=====
0: f3 aa repz stos %al,%es:(%edi) <=====
Code; c0286b87 <zlib_inflate_flush+3807/3842>
2: 58 pop %eax
Code; c0286b88 <zlib_inflate_flush+3808/3842>
3: 59 pop %ecx
Code; c0286b89 <zlib_inflate_flush+3809/3842>
4: e9 73 68 ff ff jmp ffff687c <_EIP+0xffff687c>
Code; c0286b8e <zlib_inflate_flush+380e/3842>
9: ba f2 ff ff ff mov $0xfffffff2,%edx
Code; c0286b93 <zlib_inflate_flush+3813/3842>
e: e9 bf 68 ff ff jmp ffff68d2 <_EIP+0xffff68d2>
Code; c0286b98 <zlib_inflate_flush+3818/3842>
13: ba 00 00 00 00 mov $0x0,%edx
Jul 11 02:16:25 vserver kernel: <1>Unable to handle kernel NULL pointer
dereference at virtual address 0000080e
Jul 11 02:16:25 vserver kernel: c0286b85
Jul 11 02:16:25 vserver kernel: *pde = 00000000
Jul 11 02:16:25 vserver kernel: Oops: 0002
Jul 11 02:16:25 vserver kernel: CPU: 1
Jul 11 02:16:25 vserver kernel: EIP: 0010:[<c0286b85>] Tainted: P
Jul 11 02:16:25 vserver kernel: EFLAGS: 00010246
Jul 11 02:16:25 vserver kernel: eax: 00000000 ebx: 0000008f ecx:
0000008f edx: 08057eaf
Jul 11 02:16:25 vserver kernel: esi: 08057e20 edi: 0000080e ebp:
cb32d960 esp: ce96df44
Jul 11 02:16:25 vserver kernel: ds: 0018 es: 0018 ss: 0018
Jul 11 02:16:25 vserver kernel: Process qmail-remote (pid: 7101,
stackpage=ce96d000)
Jul 11 02:16:25 vserver kernel: Stack: 00000003 0000008f cc5c2a40 0000008f
fffffff2 c015aefd 0000080e 08057e20
Jul 11 02:16:25 vserver kernel: 0000008f 00000006 0000008f 0000080e
ce96df64 cc5c2aa8 00000000 0000008f
Jul 11 02:16:25 vserver kernel: 00000000 cb32d960 ffffffea 0000008f
c014f027 cb32d960 08057e20 0000008f
Jul 11 02:16:25 vserver kernel: Call Trace: [<c015aefd>] [<c014f027>]
[<c010beef>]
Jul 11 02:16:25 vserver kernel: Code: f3 aa 58 59 e9 73 68 ff ff ba f2 ff ff
ff e9 bf 68 ff ff ba
>>EIP; c0286b85 <zlib_inflate_flush+3805/3842> <=====
>>edx; 08057eaf Before first symbol
>>esi; 08057e20 Before first symbol
>>edi; 0000080e Before first symbol
>>ebp; cb32d960 <_end+af683a0/38447aa0>
>>esp; ce96df44 <_end+e5a8984/38447aa0>
Trace; c015aefd <pipe_write+22d/2d0>
Trace; c014f027 <sys_write+97/140>
Trace; c010beef <system_call+33/38>
Code; c0286b85 <zlib_inflate_flush+3805/3842>
00000000 <_EIP>:
Code; c0286b85 <zlib_inflate_flush+3805/3842> <=====
0: f3 aa repz stos %al,%es:(%edi) <=====
Code; c0286b87 <zlib_inflate_flush+3807/3842>
2: 58 pop %eax
Code; c0286b88 <zlib_inflate_flush+3808/3842>
3: 59 pop %ecx
Code; c0286b89 <zlib_inflate_flush+3809/3842>
4: e9 73 68 ff ff jmp ffff687c <_EIP+0xffff687c>
Code; c0286b8e <zlib_inflate_flush+380e/3842>
9: ba f2 ff ff ff mov $0xfffffff2,%edx
Code; c0286b93 <zlib_inflate_flush+3813/3842>
e: e9 bf 68 ff ff jmp ffff68d2 <_EIP+0xffff68d2>
Code; c0286b98 <zlib_inflate_flush+3818/3842>
13: ba 00 00 00 00 mov $0x0,%edx
2 warnings and 6 errors issued. Results may not be reliable.
Any idea what went wrong ??
Regards,
/Brian