Re: [vserver] Re: Again: [vserver] Linux vServer: general protection fault with apache2 and kernel 2.6.38.6

From: Herbert Poetzl <herbert_at_13thfloor.at>
Date: Thu 15 Sep 2011 - 02:21:28 BST
Message-ID: <20110915012128.GI9137@MAIL.13thfloor.at>

On Wed, Sep 14, 2011 at 12:11:00PM +0200, Jean Weisbuch wrote:
> On 12/09/2011 16:15, Herbert Poetzl wrote:
>>> Hello,
>>> I encountered the problem on several servers (mainly dual
>>> 6cores Istanbul Opteron and Xeon E3 (Sandy Bridge)).
>>> As the kernel.org bugzilla is down, i dont know in which
>>> kernel versions the bug has been corrected.
>> well, the second url linked above says 2.6.35 to 3.0-rc7,
>> so I'd presume it was fixed in 3.0 final ....

>>> Which kernel versions are immune to the bug?
>> again, handwaving here based on the same url, a kernel
>> before 2.6.35 or after 3.0 should not expose the issue,
>> if that's what you call 'immune' ...

>>> Are the 3.0x kernels from the psand.info repository safe
>>> for this (i got the problem on the latest 2.6.36 and 2.6.38)?
>> once again, if it was fixed in 3.0, the kernels from there
>> should be as safe as any other, in general the latest
>> kernel + patch is advised ...

>> best,
>> Herbert

>>> Thanks in advance.
> Unfortunately, i had the problem again on the
> 3.0.4-vs2.3.1-pre10.1-beng amd_x64 kernel after one day of
> uptime on a server with only one VServer (an apache2 node) :

please run the addresses (those numbers in angle brackets
like <ffffffff81331f32>) through addr2line -e vmlinux
(where vmlinux is the file from the build tree) and
annotate the bug trace with those ... it should be rather
simple to pinpoint and feed upstream if not already fixed

(hopefully kernel.org will be back soon)

thanks in advance,
Herbert

> Sep 14 03:48:36 localhost kernel: [55435.551844] BUG: unable to handle
> kernel NULL pointer dereference at 0000000000000050
> Sep 14 03:48:36 localhost kernel: [55435.551901] IP:
> [<ffffffff81331f32>] mutex_lock+0x22/0x22
> Sep 14 03:48:36 localhost kernel: [55435.551936] PGD 10c379067 PUD
> 1d7faf067 PMD 0
> Sep 14 03:48:36 localhost kernel: [55435.551970] Oops: 0002 [#1] SMP
> Sep 14 03:48:36 localhost kernel: [55435.552000] CPU 0
> Sep 14 03:48:36 localhost kernel: [55435.552006] Modules linked in:
> nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_tcpudp
> iptable_filter ip_tables x_tables nfs lockd auth_rpcgss nfs_acl sunrpc
> cpufreq_conservative speedstep_lib acpi_cpufreq cpufreq_userspace
> cpufreq_powersave cpufreq_stats mperf ext3 jbd coretemp ipmi_si
> ipmi_msghandler loop snd_pcm snd_timer snd i2c_i801 tpm_tis soundcore
> intel_agp tpm snd_page_alloc tpm_bios intel_gtt i2c_core ghes pcspkr
> joydev button hed processor evdev thermal_sys psmouse serio_raw ext4
> mbcache jbd2 crc16 usbhid hid sd_mod crc_t10dif ahci libahci libata
> scsi_mod ehci_hcd usbcore e1000e [last unloaded: scsi_wait_scan]
> Sep 14 03:48:36 localhost kernel: [55435.552408]
> Sep 14 03:48:36 localhost kernel: [55435.552430] Pid: 28772, comm:
> httpd Not tainted 3.0.4-vs2.3.1-pre10.1-beng #1 Supermicro
> X9SCL/X9SCM/X9SCL/X9SCM
> Sep 14 03:48:36 localhost kernel: [55435.552488] RIP:
> 0010:[<ffffffff81331f32>] [<ffffffff81331f32>] mutex_lock+0x22/0x22
> Sep 14 03:48:36 localhost kernel: [55435.552537] RSP:
> 0000:ffff88010586be80 EFLAGS: 00010202
> Sep 14 03:48:36 localhost kernel: [55435.552565] RAX: 00000000fffffffe
> RBX: ffff880172e99500 RCX: ffff880085c04d90
> Sep 14 03:48:36 localhost kernel: [55435.552597] RDX: 0000000040000200
> RSI: ffff880172e99500 RDI: 0000000000000038
> Sep 14 03:48:36 localhost kernel: [55435.552629] RBP: ffff8801adbe0618
> R08: 0000000031210000 R09: ffffffffa02b5f30
> Sep 14 03:48:36 localhost kernel: [55435.552661] R10: ffff880200000003
> R11: 0000000000000000 R12: 0000000000000000
> Sep 14 03:48:36 localhost kernel: [55435.552692] R13: 0000000000000000
> R14: 0000000000000000 R15: 0000000000000000
> Sep 14 03:48:36 localhost kernel: [55435.552725] FS:
> 0000000000000000(0000) GS:ffff88023fc00000(0063) knlGS:00000000f73f56b0
> Sep 14 03:48:36 localhost kernel: [55435.552774] CS: 0010 DS: 002b
> ES: 002b CR0: 000000008005003b
> Sep 14 03:48:36 localhost kernel: [55435.552803] CR2: 0000000000000050
> CR3: 00000001d7e7a000 CR4: 00000000000406f0
> Sep 14 03:48:36 localhost kernel: [55435.552834] DR0: 0000000000000000
> DR1: 0000000000000000 DR2: 0000000000000000
> Sep 14 03:48:36 localhost kernel: [55435.552866] DR3: 0000000000000000
> DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Sep 14 03:48:36 localhost kernel: [55435.552899] Process httpd (pid:
> 28772, threadinfo ffff88010586a000, task ffff88002f6d6790)
> Sep 14 03:48:36 localhost kernel: [55435.552946] Stack:
> Sep 14 03:48:36 localhost kernel: [55435.552968] ffffffff81101a83
> 000000000acb5084 00000000fffffffe 0000000000000000
> Sep 14 03:48:36 localhost kernel: [55435.553022] ffff880172e99500
> 0000000000000000 ffffffff81104248 ffff88011d0e5ec0
> Sep 14 03:48:36 localhost kernel: [55435.553077] ffff88004c84aa80
> 000000030028e821 ffff880104d6000e 0000000000000000
> Sep 14 03:48:36 localhost kernel: [55435.553131] Call Trace:
> Sep 14 03:48:36 localhost kernel: [55435.553157] [<ffffffff81101a83>]
> ? vfs_rmdir+0xa8/0xc3
> Sep 14 03:48:36 localhost kernel: [55435.553187] [<ffffffff81104248>]
> ? do_rmdir+0xb2/0xf9
> Sep 14 03:48:36 localhost kernel: [55435.553217] [<ffffffff81334c83>]
> ? ia32_do_call+0x13/0x13
> Sep 14 03:48:36 localhost kernel: [55435.553245] Code: 48 83 c4 50 5b
> 5d 41 5c c3 53 48 89 fb 48 83 ec 08 f0 ff 0f 79 05 e8 e1 01 00 00 65 48
> 8b 04 25 c0 b5 00 00 48 89 43 18 58 5b c3
> Sep 14 03:48:36 localhost kernel: [55435.553491] RIP
> [<ffffffff81331f32>] mutex_lock+0x22/0x22
> Sep 14 03:48:36 localhost kernel: [55435.553522] RSP <ffff88010586be80>
> Sep 14 03:48:36 localhost kernel: [55435.553545] CR2: 0000000000000050
> Sep 14 03:48:36 localhost kernel: [55435.553960] ---[ end trace
> 57513ff63fad6714 ]---

> As always, there is a remaining httpd process that i cannot kill and
> that makes impossible to correctly stop the VServer (and to restart it,
> as there is still a running process) :

> root@cl3-www5:~# vps aux|grep mutu0115
> 11704 15973 20115 mutu0115 0.0 0.7 78464 57984 ?
> D 06:11 0:01 /home/apache/bin/httpd -k start
> root@cl3-www5:~# vkill -s 9 15973
> root@cl3-www5:~# vps aux|grep mutu0115
> 11704 15973 20115 mutu0115 0.0 0.7 78464 57984 ?
> D 06:11 0:01 /home/apache/bin/httpd -k start

> Regards.
Received on Thu Sep 15 02:21:42 2011

[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Thu 15 Sep 2011 - 02:21:43 BST by hypermail 2.1.8