On Thu, 2009-04-02 at 23:47 +0200, Herbert Poetzl wrote:
> On Thu, Apr 02, 2009 at 08:36:40AM -0400, John A. Sullivan III wrote:
> >
> >
> > On Thu, 2009-04-02 at 13:04 +0200, Herbert Poetzl wrote:
> > > On Tue, Mar 31, 2009 at 09:41:06AM -0400, John A. Sullivan III wrote:
> > > > On Tue, 2009-03-31 at 08:40 +0000, Christoph Lukas wrote:
> > >
> > > [stuff zapped]
> > >
> > > > Yes, the numbers match and they also match the size of the space
> > > > consumed on the thinly provisioned ZFS zvol on which they reside
> > > > (28.1 GB):
> > >
> > > there is no attribute support for ZFS, and thus
> > > there is no barrier and no CoW-link breaking
> > > available either ... your options are:
> > >
> > > - sponsor development for ZFS/xattrs
> > > - have immutable hardlinks without CoW link breaking
> > > - move to a supported filesystem
> > >
> > > best,
> > > Herbert
> > <snip>
>
> > Thanks, Herbert, but the files aren't sitting on ZFS.
> > I wasn't as clear as I should be.
>
> > ZFS is the underlying file store on the SAN but it exporting
> > block devices to VServer which are formatted as ext3 - sort
> > of like a block image file in KVM.
>
> okay, so you are using ZFS (on the SAN) to store a
> file, which gets exported (via ??) as block device
> to the host, which in turn, uses an ext3 filesystem
> to contain guest data ...
>
> > All the vservers are sitting in the same pseudo block device
> > and all as ext3.
>
> they are created in the same filesystem, which
> is linked from the hash store?
>
> > I wouldn't think that would ZFS would come into play but,
> > as I said, I know little of the internals
>
> no, in this case the ZFS and its features are not
> really used and thus not relevant ... the way the
> space arrives at the host might be relevant, but
> doesn't matter for the unification case
>
> double check that files which _should_ definitely
> be shared and identical (stat /path/to/guest/bin/bash)
> are indeed identical in all aspects, and see if they
> are shared (link count ~ #guests) after hashification
<snip>
Strange. I'm starting to feel like I must be sending folks on a wild
goose chase. I looked at a number of random files in different parts of
the directory system and they all have between 8 and 14 links.
I assume I should be doing something like:
[root@vd01 vservers]# stat jasiii/usr/lib/firefox-3.0.8/firefox.sh
File: `jasiii/usr/lib/firefox-3.0.8/firefox.sh'
Size: 2602 Blocks: 8 IO Block: 4096 regular file
Device: fd08h/64776d Inode: 19794834 Links: 12
Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2009-04-02 22:11:49.000000000 -0400
Modify: 2009-03-27 07:51:26.000000000 -0400
Change: 2009-03-30 03:29:27.000000000 -0400
Yet, a df shows:
[root@vd01 vservers]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/VG0-rootlv
183975852 24203544 150276144 14% /
/dev/mapper/VG0-locallv
19838052 178996 18635056 1% /usr/local
/dev/md0 988024 36696 900328 4% /boot
tmpfs 16510400 0 16510400 0% /dev/shm
none 262144 0 262144 0% /tmp
/dev/mapper/VG1-vserverlv
231195772 22052628 197399096 11% /vservers
/dev/mapper/VG1-datalv
3996374136 22179988 3771190040 1% /data
I do have external file systems mounted inside the vserver directories
but I would not think this should interfere:
[root@vd01 vservers]# mount
/dev/mapper/VG0-rootlv on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/mapper/VG0-locallv on /usr/local type ext3 (rw)
/dev/md0 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /tmp type tmpfs (rw,noexec,nosuid,nodev,size=256m,mode=1777)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
/dev/mapper/VG1-vserverlv on /vservers type ext3 (rw)
/vservers/vetc on /etc/vservers type none (rw,bind)
/dev/mapper/VG1-datalv on /data type ext3 (rw)
/data/clients/a0000-0010 on /vservers/smcc/data type none (rw,bind)
/data/clients/a0000-0010 on /vservers/tvan/data type none (rw,bind)
/data/clients/a0000-0010 on /vservers/cler/data type none (rw,bind)
/data/clients/a0000-0010 on /vservers/mlap/data type none (rw,bind)
/data/clients/a0000-0010 on /vservers/vdem/data type none (rw,bind)
/data/clients/a0000-0010 on /vservers/jint/data type none (rw,bind)
/data/clients/a0000-0010 on /vservers/tkee/data type none (rw,bind)
/data/clients/a0000-0010 on /vservers/mint/data type none (rw,bind)
/data/clients/a0000-0100 on /vservers/gssdata type none (rw,bind)
/data/clients/a0000-0100 on /vservers/jas/data type none (rw,bind)
/data/clients/a0000-0002 on /vservers/simple1/data type none (rw,bind)
I'm particularly concerned about memory. Each of these guests is
running the same KDE desktop. I would suspect several hundred MB of RAM
to be consumed by the first user but then far less by each subsequent
user. Instead, I'm seeing many hundreds of MB each.
Perhaps it is normal and my expectations of what hashify does are
unrealistic but it seems we got much better mileage in our previous test
installations. Thanks - John
-- John A. Sullivan III Open Source Development Corporation +1 207-985-7880 jsullivan@opensourcedevel.com http://www.spiritualoutreach.com Making Christianity intelligible to secular societyReceived on Fri Apr 3 04:35:06 2009