On 20/10/2010 18:39, Ed W wrote:
> On 20/10/2010 15:02, Gordan Bobic wrote:
>> I have just tested this with a new kernel. And the issue doesn't arise
>> with 22.214.171.124-vs126.96.36.199.33! So it is definitely a kernel bug, most
>> likely in ext4.
> Cripes. That's rather scary...
Tell me about it! I'm guessing it will affect anything that works with
hard-links on ext4 in that kernel. I'd rather like to know how that got
through testing without being picked up. At the very least, that kernel
should have been tagged with a big fat warning message.
> Well done for tracing it down - hope those filesystems are replaceable..?
I was just about to put the system into production so there was no data
on it yet. The only vaguely valuable thing was the configs (a few hours
work at most, I probably spent more time debugging the issue when it all
went wrong), and those survived since they didn't get hashified. :)
So not much harm done, but it could have been much worse. Since it
doesn't become apparent until after a reboot due to the healthy versions
sticking in caches as unflushable, that data could have been getting
corrupted for days before the machine ran out of RAM and died. Only then
would the corruption have become apparent.
Received on Wed Oct 20 19:41:20 2010