Re: [vserver] Poll: High (ish) availability - how are you doing it?

From: Gordan Bobic <gordan_at_bobich.net>
Date: Thu 29 Jul 2010 - 01:08:18 BST
Message-ID: <4C50C672.1090009@bobich.net>

Ed W wrote:
> On 28/07/2010 15:16, Edward Capriolo wrote:
>> I also successfully ran a vserver on top of a Mirrored GlusterFS
>> volume with linux-ha failover.( This is essentially User space DRBD.)
>
> Thanks to everyone who has replied - very interesting.
>
> Glusterf is the one which most interests me (active-active clustering),
> but it seems that the public forums are full of people with problems,
> but then a handful of folks who are very happy. Obviously you always
> hear most about the outliers, but it does suggest that there are quite a
> few subtleties with gluster? Anyone got anything to share?

It depends on what you do with it and in what environment.

Pros:
- It stores files on the underlying, conventional file system (e.g.
ext3, xfs, etc.), so data recovery is quite straightforward if things go
wrong.
- It doesn't require fencing devices.

Cons:
- Without fencing devices, splitbrain is a common occurrence if packets
start getting dropped.
- Slow (especially access latencies, even worse than conventional
cluster file systems like GFS or OCFS2).
- There are a few obscure things that are still subtly broken (e.g.
nvidia driver installer does something weird with sockets that works
fine on a normal FS but produces huge and corrupted files on GlusterFS).
- Some performance translators break things in unpredictable ways.

> Seems like there is nothing really on the horizon that will do cross
> datacenter clustering yet... Sounds like XtreemFS might turn out to
> work, but it seems a long away from being stable and well tested...

I wouldn't hold my breath. The problem is two-fold:
1) Performance
Latencies between data centers make achieving decent performance
essentially impossible on a file system that guarantees consistency.

2) Fencing
You need reliable fencing between the nodes to ensure everything works
as it should. This isn't a cluster/fs issue, it is a WAN networking
issue. If your WAN is redundant and fault tolerant, it isn't really a
problem.

One file system that hasn't been mentioned is SeznamFS. Think of it as
GlusterFS (both SeznamFS and GlusterFS are fuse based, both store files
on the underlying conventional file system) but applying file
replication in the same way that MySQL replication works (binary
logging, master-slave or ring master-master). I doesn't have any POSIX
locking, and suffers from all the racing conditions that MySQL suffers
from in a master-master setup (it is conceptually inherent in the
approach). It may, however, be good enough for what you want.

IIRC the SeznamFS project only provides debian packages. If you need
spec files for rpm building, I wrote some a while back and will try to
find them.

Gordan
Received on Thu Jul 29 01:08:32 2010

[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Thu 29 Jul 2010 - 01:08:33 BST by hypermail 2.1.8