Re: [vserver] vserver scalability

On Tue June 15 2010, Eugen Leitl wrote:
> On Tue, Jun 15, 2010 at 07:59:48AM -0500, Michael S. Zick wrote:
>
> > > Something which can be adressed with SSDs.
> > >
> >
> > You wouldn't by chance be using one of these things:
> > http://www.seamicro.com/
>
> No, but it's an interesting machine.
>
> > I.E: Are you dealing with Atoms one at a time or in bunches of 512?
>
> I'm so far dealing with two servers (four cores, 8 GByte RAM)/
> 1 U. This would mean some 80 servers/rack, 320 Atom cores/rack,
> 640 GByte RAM and some 80 TByte disk in a single rack.
>
> I agree that currently I/O contention is the biggest problem
> in a multi-guest server, assuming you can keep out the CPU and
> memory hogs.
>

Let see now, a quick common sense check on that sea micro blurb -
It says: supports 30 million concurrent connections -

Eight processors per card, even with a disk per processor rather
than a disk per card, that is potentially 60,000 concurrent read/writes
per processor/memory cache/disk.

Unless they have (they don't) 60,000 i/o channels, somewhere along
the way that gets serialized into sequential reads/writes.

Me thinks that front page "30,000,000" was written before the Q.A. dept.
completed their load testing of the box and signed off on the brag. ;-)
Or their definition of "concurrent" got stretched into "device lifetime".

The point(s) where electronics has to interface with the physical world
will always be a point that needs care and attention.

So solving (or at least optimizing) those problems in your smaller example
and then knowing what its limits are is a good step to knowing what the
bottom line limitations a huge collection of them would have to deal with.

---
With all that in mind, back to the post topic. . .
You need to refine the question by "what sort of operations" in the collection
of vserver tasks that you want to consider the limiting case (or typical
application) -
For illustration, two extreme examples:
1) We are interfacing with a human/dumb terminal/keyboard per vserver instance.
Humans can't type fast enough to generate much I/O load, and if they read the
command prompt before typing. . .
In this case you will probably find a limitation in the amount of memory available.
2) We are interfacing with other machines that are making electronic requests of,
say a database server with very de-normalized tables, per vserver instance.
For: select users in world, do ...
Yeah, buddy, that will suck up the hardware resources. ;-)
---
Keep in mind that somewhere along the way, all those activities have to be serialized;
in an eight core machine, divided by 8 and then serialized into 8 streams;
but serialized somewhere.
Without any practical means to "load test" even a "small" system like you are
studying, things are not entirely hopeless. . .
Multiple events, serialized, means queuing of some sort -
and now we enter the realm of queuing in the field of communications theory.
__That__ has been dealt with in the field for hundreds of years;
To the point where you don't even need any math, just some fancy look-up tables.
For failed attempts lost, you want Erlang-B
For failed attempts held (queued), you want Erlang-C
If you ignore the references to the telephone communications industry (my prior lifetime) -
The theory and math applies to everything from the waiting time in the grocery store
line at the check-out counter to waiting time of request packets to the MySQL server
or how many and how long the wait of the I/O requests in the disk read/write queue.
Start here:
http://en.wikipedia.org/wiki/Erlang_(unit)
Also consider:
http://www.erlang.com/calculator/
And then start burning up your "Google Credits".
---
Bottom line:
You will still need to bound and characterize the sort of operations you expect each
vserver instance to handle so that the input parameters to the theory can at least
be approximated.
And if Google can't find you on-line copies of the look-up tables, mail me off-list;
I printed a set that is widely used in a number of industries 30 years ago, I should
be able to find you a copy somewhere on-line.
Mike
> > Mike