Re: [vserver] Ocf resource agent

From: Laurent Spagnol <laurent.spagnol_at_univ-reims.fr>
Date: Mon 12 Apr 2010 - 09:31:20 BST
Message-ID: <4BC2DA58.6080908@univ-reims.fr>

Olivier BATARD a écrit :
> Hi,
>
>
> I've set up a Debian Lenny pacemaker / corosync / drbd cluster with vservers. Each vserver is on a drbd device with lvm and meta internal.
>
> I've set up drbd resources and filesystem resource, everything woeks fine, failover is nice but now I want to have a resource to start and stop vserver ? Does anyone know how to do that ? I've found several script with google for heartbeat but doesn't work so much ... especially with the meta part.
>
> Any experiences ?
>
> Thanks,
>
>
> Olivier
>
>
Hi,

I also tried Heartbeat, and I also encountered difficulties.
So i wrote my own shell script, running as a daemon.

It **try** to manage 2 difficulties:
- split-brain ( ... running accidentaly the same database on two nodes
is one of the worst thing i can imagine ... )
- problems with ARP caches (especialy with some L3 switches)

I use 2 nics on my servers (one for Vservers, and "service" interface
for drbd).
The script will enable / disable the Vservers's interface with the same
MAC address on the two machines, but i also use "arping" to force
refresh of ARP tables.

/dev/drbd is mounted under /DRBD, and I use Debian package "drbdlinks"
to handle symlinks on the right place.
/dev/sdXX -> /dev/vgX/lvX -> /dev/drbdX

Each node monitor it's own connectivity, and the state of the remote node.
This solution is not perfect, but it is running for 2 years without big
problems
But sometimes i has to manage manualy drbd ...

vsh-04:~# tree -L 2 /DRBD/etc/
/DRBD/etc/
`-- vservers
    |-- cas
    |-- casper
    |-- ebureau
    |-- idp
    |-- ldap
    |-- ldap-auth
    |-- ldap-auth.old
    |-- ldap.old
    |-- service-hs
    |-- supext
    |-- webmail
    |-- wme
    `-- wmp

vsh-04:~# tree -L 2 /DRBD/var/lib/
/DRBD/var/lib/
`-- vservers
    |-- cas
    |-- casper
    |-- ebureau
    |-- idp
    |-- ldap
    |-- ldap-auth
    |-- ldap-auth.old
    |-- ldap.old
    |-- service-hs
    |-- supext
    |-- webmail
    |-- wme
    `-- wmp

This may give you some ideas:

*** Main script:

#!/bin/bash
. /usr/lib/vs-tools/functions.sh

[ -z $HA_CONF ] && exit
[ ! -f $HA_CONF ] && exit
. $HA_CONF

while [ true ] ; do

    if [ "$(drdb_remote_status)" == "unknown" ] ; then
        /etc/init.d/drbd reload 2>/dev/nul >/dev/null
    else
        unset STOPPED
    fi

    if [ ! "$(get_running_list)" ] &&\
       [ "$(drdb_local_status)" == "secondary" ] &&\
       [ ! "$STOPPED" ] ; then
        if [ "$(test_ping $HA_PING)" ] &&\
           [ "$(drdb_remote_status)" != "primary" ] ; then
            drbd_storage_on $HA_DRDB_DEV $HA_MOUNT_POINT $HA_FS_TYPE
$HA_MOUNT_OPTS
            if [ $? -eq 0 ] ; then
                while [ $? -eq 0 ] ; do
                    for ip in $(cat /etc/vservers/*/interfaces/0/ip) ; do
                        ping -c 1 -w 1 $ip 2>/dev/null >/dev/null
                    done
                done
                ha_if_on $HA_NET_DEV $HA_MAC_ADDR
                /etc/init.d/util-vserver start 2>/dev/null >/dev/null
            fi
        fi
    fi

    if [ "$(get_running_list)" ] &&\
       [ "$(drdb_local_status)" == "primary" ] ; then
           if [ ! "$(test_ping $HA_PING)" ] ||\
              [ ! "$(test_net_link $HA_NET_DEV)" ] ; then
            /etc/init.d/util-vserver stop 2>/dev/null >/dev/null
            ha_if_off $HA_NET_DEV $HA_MAC_ADDR
            drbd_storage_off $HA_DRDB_DEV $HA_MOUNT_POINT
            STOPPED=1
        fi
    fi

    sleep $HA_SLEEP

done

*** Functions:

function drbd_storage_on {
[ "$(test_mount $1 $2)" ] && return
drbdsetup $1 primary 2>/dev/null >/dev/null || return 1
mount -o $4 $1 $2 2>/dev/null >/dev/null
drbdlinks start 2>/dev/null >/dev/null
return 0
}

function drbd_storage_off {
[ "$(test_mount $1 $2)" ] || return
while [ "$(lsof |grep ^sleep |grep $VDIR/)" ] ; do
    sleep 1
done
drbdlinks stop 2>/dev/null >/dev/null
umount $2 2>/dev/null >/dev/null
drbdsetup $1 secondary 2>/dev/null >/dev/null
return 0
}

function drdb_local_status {
[ -f /proc/drbd ] || return
grep -v "^SVN " /proc/drbd |grep "0:" |awk -F : '{print $4}' | awk -F /
'{print tolower($1)}'
}

function drdb_remote_status {
[ -f /proc/drbd ] || return
grep -v "^SVN " /proc/drbd |grep "0:" |awk -F : '{print $4}' | awk -F /
'{print $2}' |awk '{print tolower($1)}'
}

The full script is here:
http://dokuwicri.univ-reims.fr/wiki/doku.php?id=public:projets:vs-tools:start

-- 
Laurent Spagnol
Administrateur Linux
Université de Reims
Centre de Ressources Informatiques
Campus du Moulin de la Housse
BP 1039 - 51687 Reims cedex 2
Tel: 03.26.91.88.32
Fax: 03.26.91.31.87
Received on Mon Apr 12 09:31:35 2010
[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Mon 12 Apr 2010 - 09:31:37 BST by hypermail 2.1.8