I have just has something very, very strange happen with one of my
vserver hosts. I have been prototyping things on it for a while and was
about to move it into production. I powered it off and back on, and
that's where things started to get inexplicably weird.
Long story short - as far as I can tell _every_ single binary file I
could find in /bin, /sbin, /lib, /usr/lib, etc is corrputed. They don't
even check out as executables/libraries using the file command.
My first thought was disk corrpution, but fsck found no errors on the
disks. The hosts files are all fine, too. Even more strangely, all the
configuration files in /etc in the guests are fine! I have so far not
found any files there corrupted!
All the files have the correct size and timestamp (compared with the
host's non-corrupted files).
I am at a complete loss to explain this at the moment. Everything worked
fine before shutting down the host.
The only thing I can think of at the moment is that hashify could
somehow have been responsible for this, but that still doesn't explain
why everything worked fine right up until shutdown.
Can anybody hazard a guess as to what happened here? I'm prepared to
consider any theory at the moment, no matter how far fetched.
I'm running 188.8.131.52-vs184.108.40.206.14-pre8. The file system is ext4
without journal and in data=writeback mode.
Has there ever been a report of anything like this happening before?
I am not too concerned about the damage in this particular instance -
the RPM database lives outside the guests and is thus OK (I checked), so
re-building the guests will be fairly trivial and all the configuration
files in /etc seem to be fine. What DOES concern me, however, is the
possibility of this happening again, so I am rather keen to hear any
testable hypotheses that might help get to the bottom of this.
Received on Sun Oct 17 00:59:36 2010