About this list Date view Thread view Subject view Author view Attachment view

From: Chuck (chuck_at_sbbsnet.net)
Date: Wed 14 Sep 2005 - 12:17:39 BST


On Wednesday 14 September 2005 01:14 am, Herbert Poetzl wrote:
> On Wed, Sep 14, 2005 at 12:59:53AM -0400, Chuck wrote:
> > On Wednesday 14 September 2005 12:29 am, Herbert Poetzl wrote:
> >
> > > > I added it to the /vservers mount statement in fstab and it worked
> > > > perfectly!
> > >
> > > good to hear!
> > >
> > > > > > also the kernel has some extended reiserfs options which are
> > > > > > unchecked at this time as I have never known a need for them.
> > > > > > Should they be enabled as well?
> > > > >
> > > > > you probably want CONFIG_REISERFS_FS_XATTR=y but
> > > > > I'm not sure if this is required ...
> > > >
> > > > Guess it cannot hurt to enable it anyway for safety.
> > > >
> > > > Thank you!
> > >
> > > you're welcome!
> > >
> > > > now i just hope vservers live up to my expectations.. I have been
> > > > given the task of finding a way to consolidate which means looking
> > > > at virtualizing/partitioning to cut the number of servers down.
> > > > What I am hoping for is enough performance to be able to reduce 39
> > > > servers to just 8 or 9 hosts.. we are a fairly busy isp so some of
> > > > the servers are hit pretty hard..
> > >
> > > well, up to 200 guest for a dual CPU Xeon system were
> > > reported working, although I would not suggest to go
> > > that hight, especially if you expect higher load ...
> > >
> > > > the only other problem I fear I will run into is in one host there
> > > > will hve to be at least 4 nics since the servers are on various
> > > > vlans and each subnet has its assigned switch port.
> > >
> > > well, 4 nics with GB can be rarely satisfied with PC
> > > (and especially 32bit) arch ... if you think about
> > > several 100Mbit interfaces, consider using vlans instead
> >
> > even if all the vservers on a single host were in the same network
> > segment, there would still have to be 2 nics due to eth1 being the
> > private admin network and also the one that the servers use to
> > communicate to each other privately for nfs or remote logging.
>
> well, this could also be done via vlans, but 2 NICs
> are fine, no?
>

after getting my head more around virtual server concepts, the private network
only needs to be addressed in each host. the necessary communications can be
done host to host easily enough since it has physical access to all vservers
for backup. then i sat down and listed every server we run by network segment
and discovered that a majority of them can use a single nic.. for example
there are 10 severs now on the same segment. those could be put into one or
two hosts since most of them are low usage machines.

> > > > to complicate matters even more, several of the vservers will have
> > > > to support a few hundred ip addresses each (email and web for
> > > > domains with dedicated ip addresses).
> > >
> > > the current limit is at 16 IPs for each guest, but it
> > > can be raised (easily) but this comes with a drawback
> > > to the overall performance ...
> > >
> >
> > > you might want to check if it isn't possible to break
> > > the hosted IPs down into smaller chunks, and take e.g.
> > > 10 guests for 16 IPs each (or maybe 5 with 32)
> >
> > hmmm... wonder how much of a hit in performance. it is impossible to
> > separate them as the large chunks reside on machines with commercially
> > licensed and paid for software which means we cannot duplicate these
> > hosts to spread the load. it is all or nothing.
>
> the performance hit is basically a linear O(N) search
> for each connection/request which is not terribly
> expensive with 4 or 8 IPs but might have some impact
> with 128 or 256 IPs ...
>

we want to get rid of the webserver management software we use as it is hard
to impossible to set up without their support people doing it, is very
expensive, and has some ludicrous requirements that have driven me insane
trying to maintain security (plesk).. and of course we had no clue about
these things until we bought into it. however we have not found a suitable
substitute yet. if we can get around the licensing issue if there is one we
would more than like to separate the quantity of hosted sites per box into
smaller chunks :)

the requirement for multiple nics comes in in only a few instances where we
have name servers on the same vlan.. one is on 32 net and one is on 33 net
which is one /23 vlan. to get around the multiple nics in this situatio all
of those machines would have to be in one host to be able to use a single
nic. having 2 public name servers one one physical machine can lead to
trouble if there is a hardware failure

i personally am for killing the vlan concept completely as it has caused
nothing but headaches for me due to restrictions and I don't believe an isp
has any need for such things except one or two network segments to isolate
customer colo machines from the rest of our network. but... the boss paid
some contracted etwork engineer mucho $$$ to set this up and he would not
want us ripping it out. he did it in a typical vlan construct between the
routers and switches so that from the server side they would be physically
separated networks. and since we are dealing with a /19 issue from ARIN and
even some of those /24 segments broken into /28 or /29 segments you can see
the sheer quantity of the issues i have to deal with. this guy created a
monster that we now have to deal with and none of us can figure out the odd
stuff he did within the switches.. our cisco cert tech said it is so strange
she does not want to touch it for fear of breaking things. would be nice to
change the vlan segments into /23 each instead of smaller chunks to make life
easier virtualizing things. but.. at the time our mindset was individual
servers.

> > I was anticipating future growth with those numbers. In real numbers
> > today, the email machine has i think around 97 ip addresses and the
> > web machine has in the neighborhood of 124 addresses with the rest of
> > the sites using name space off the main machine ip.
>
> most services can be satisfied with a single IP
> (e.g. mail, ftp, web) only unfortunate ones require
> a separate ip (ssl)

and fully half or more of our installations are ssl driven:(

thankfully we do not issue shell accounts so I am the only one with ssh access
into any machine except customer colo machines, and even then they contract
us to do most of the security maintenance on them and only I have root access
to them. in fact we only have 4 machines owned by the customer rather than
leased from us and we have no access at all into those 4. they are on their
own vlan for security reasons.

all those web ip addresses are required by eitther the software they run or
the fact they need a discreet reverse lookup accurate to their hostname. I
have always tried to use web namespace as it is easier for me and saves ip
space and have been known to deny issuing an ip based on 'the customer just
wants one'. :) they have to prove they need it before i give them one.

>
> > all the mail ips are on the same network segment and all the web are
> > also on their own segment.. each network segment is its own vlan
> > within the switching system and each machine has its own assigned
> > switch port.
>
> > since the vlans are physical separations, it would not be possible to
> > combine them onto one nic..
>
> interesting, I always considered virtual lans (vlans)
> virtual ones not physical ones :)

and they are virtual between the routers and switches, then converted to
physical via assigned ports.

if we can find suitable admin software for the web servers then we can easily
split sites up which I want to do anyway... its the email that has me most
worried. the web will be sitting as its own dedicated machine until we figure
this out then we can move things over... the email still will have to be all
on one virtual server.. i am going to contact the author support about
changes to each domain to set up a single ip instead and use name space. I
have no idea of that impact within the mail server so I will have to go by
the author's guidance on this. if it cannot be 100% transparent to the
customer then I cannot do it.

I think I can break things down to 3 nics per machine.. 1 private net for
hosts only, and the other 2 in most cases can share the ip space to stay
within the 16 limitation.

this is going to take much more planning than I thought... I had better buy an
extra 50 lbs of coffee, I'm going to need it along with a fresh bottle of 500
count aspirin :)

>
> > this is why there would have to be a single nic for each vserver
> > guest. we have several quad-nic cards that we could use. we had 2 of
> > those for 8 nics on our news server once.
>
> IMHO a dual GB nic and a proper switch (which is vlan aware)
> would be a much better solution ... but YMMV
>

I believe the switches are more than good enough ( i dont know the gb model,
bu tthey are the gb versions of the cisco catylist 2924 and xl series) for
that its just the way they are set up. we don't dare touch them and, in fact
we only have 'user' access into them. The boss allowd this contract engineer
to maintain admin rights to them only.

> > not only that but the load would be horrendous using a single nic
> > for multiple vservers.. we are already beyond the limits of 100mbit
> > bandwidth on the web server and the email server is pushing about
> > 50mbit continuous at this time. our entire network is gigabit even
> > into the border routers.
>
> and this is exactly where vlans on a GB interface
> would improve throughput while reducing the load
>

i think i'm going to have to have a very long heart-to -heart talk with the
boss about some of his previous decisions. I don't see how they can properly
fit in with our virtualization concept we have just come up with. they are
causing me much strife in planning this vserver conversion. i think i have
gone through an entire ream of paper just trying to chart this thing out. :)

> > if i can't do this, then my only other choice is to leave those 2 as
> > dedicated servers which i really don't want to do. the rest of the
> > machines have less than 10 ip addresses in each of their nics. many
> > of these smaller servers are sharing the same net segment so packing
> > those into the same host would allow them to share a single nic or two
> > without trouble.
>
> everything is possible ... I'm just suggesting better
> solutions, so you get a picture ...
>

and you are doing exactly that. I thank you for this help. It has been more
valuable than any documentation I have been able to find. Since there is no
centralized 'proper' documentation yet, it is quite dificult to come up with
capabilities and proper ways to do things.

i am still coming to grips with the hard link concept of saving space and
sharing utilities that way. some say to link to the host utilities others say
to create a complete template server and link to that, it can be quite
confusing. building this test bed is going to take a bit of time with all the
study i have to do. unless i really have to i don't want to experiment with
all the ways.. i want to pick what appears to be useful to us and fly with
it. the biggest problem in this method is getting my head around all the
various concepts :)

I originally wanted to set up a /vservers mount to an nfs share on our
multi-TB SAN/backup machine and run them from there. By doing that I should
be able to change a virtual from one host to another with a simple
configuration change as all the vservers in all host machines would share
that one vserver directory. i hope i will still be able to do this or
something similar.

All this would'nt be so bad if I were able to create a single host machine and
play a while, but i don't have that luxury. the boss wants to be virtualized
completely by jan 1 (with the possible exception of the web servers if we
don't find suitable software) and "i'm the man to do it for him". right :)

> > its just the 2 big servers that has my ulcers churning. the boss wants
> > them all consolidated.
>
> well, and if the boss wants it, then it has to be
> done no matter what the costs are ... right?
> (I've no problem with that either :)
>

unfortunately, yes. and to make matters worse for me, the web servers running
plesk are all redhat9 since plesk is operating sytem specific. :(

> best,
> Herbert
>
> > > > I only hope this can do it as i have found nothing else suitable other
> > > > than the ibm partitioned minis or blades.....($$$).. I believe UML has
> > > > entirely too much overhead for our needs.
> > >
> > > I'm pretty sure it will ... but keep us updated
> > >
> > > best,
> > > Herbert
> > >
> > > >
> > > > > best,
> > > > > Herbert
> > > > >
> > > > > > Chuck
> > > > > >
> > > > > > > best,
> > > > > > > Herbert
> > > > > > >
> > > > > > > > > > the kernel version is 2.6.13-vs2.1.0-pre5-gentoo
> > > > > > > > > > the util-vserver version is util-vserver-0.30.208-r2
> > > > > > > > > > the host ip is 64.113.38.83 on eth0. when i saw the above
> > error I
> > > > > > > > > > the tried adding .84 ip to the host as eth0:1 but it
didn't
> > make a
> > > > > > > > > > the difference
> > > > > > > > >
> > > > > > > > > TIA,
> > > > > > > > > Herbert
> > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > >
> > > > > > > > > > Chuck
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > _______________________________________________
> > > > > > > > > > Vserver mailing list
> > > > > > > > > > Vserver_at_list.linux-vserver.org
> > > > > > > > > > http://list.linux-vserver.org/mailman/listinfo/vserver
> > > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > Chuck
> > > > > > > >
> > > > > > > > "...and the hordes of M$*ft users descended upon me in their
> > anger,
> > > > > > > > and asked 'Why do you not get the viruses or the
> > BlueScreensOfDeath
> > > > > > > > or insecure system troubles and slowness or pay through the
nose
> > > > > > > > for an OS as *we* do?!!', and I answered...'I use Linux'. "
> > > > > > > > The Book of John, chapter 1, page 1, and end of book
> > > > > > > >
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > Vserver mailing list
> > > > > > > > Vserver_at_list.linux-vserver.org
> > > > > > > > http://list.linux-vserver.org/mailman/listinfo/vserver
> > > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Chuck
> > > > > >
> > > > > > "...and the hordes of M$*ft users descended upon me in their
anger,
> > > > > > and asked 'Why do you not get the viruses or the
BlueScreensOfDeath
> > > > > > or insecure system troubles and slowness or pay through the nose
> > > > > > for an OS as *we* do?!!', and I answered...'I use Linux'. "
> > > > > > The Book of John, chapter 1, page 1, and end of book
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > Vserver mailing list
> > > > > > Vserver_at_list.linux-vserver.org
> > > > > > http://list.linux-vserver.org/mailman/listinfo/vserver
> > > > >
> > > >
> > > > --
> > > >
> > > > Chuck
> > > >
> > > > "...and the hordes of M$*ft users descended upon me in their anger,
> > > > and asked 'Why do you not get the viruses or the BlueScreensOfDeath
> > > > or insecure system troubles and slowness or pay through the nose
> > > > for an OS as *we* do?!!', and I answered...'I use Linux'. "
> > > > The Book of John, chapter 1, page 1, and end of book
> > > >
> > > >
> > > > _______________________________________________
> > > > Vserver mailing list
> > > > Vserver_at_list.linux-vserver.org
> > > > http://list.linux-vserver.org/mailman/listinfo/vserver
> > >
> >
> > --
> >
> > Chuck
> >
> > "...and the hordes of M$*ft users descended upon me in their anger,
> > and asked 'Why do you not get the viruses or the BlueScreensOfDeath
> > or insecure system troubles and slowness or pay through the nose
> > for an OS as *we* do?!!', and I answered...'I use Linux'. "
> > The Book of John, chapter 1, page 1, and end of book
> >
> >
> > _______________________________________________
> > Vserver mailing list
> > Vserver_at_list.linux-vserver.org
> > http://list.linux-vserver.org/mailman/listinfo/vserver
>

-- 

Chuck

"...and the hordes of M$*ft users descended upon me in their anger, and asked 'Why do you not get the viruses or the BlueScreensOfDeath or insecure system troubles and slowness or pay through the nose for an OS as *we* do?!!', and I answered...'I use Linux'. " The Book of John, chapter 1, page 1, and end of book

_______________________________________________ Vserver mailing list Vserver_at_list.linux-vserver.org http://list.linux-vserver.org/mailman/listinfo/vserver


About this list Date view Thread view Subject view Author view Attachment view
[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Wed 14 Sep 2005 - 12:18:09 BST by hypermail 2.1.3