Re: [vserver] Guest hourly/daily/... cron job parallel execution

From: Fiedler Roman <Roman.Fiedler_at_ait.ac.at>
Date: Fri 18 Jan 2013 - 08:41:09 GMT
Message-ID: <2ECE9D9EEF1F524185270138AE2326598370B1@S0MSMAIL112.arc.local>

> -----Ursprüngliche Nachricht-----
> Von: Corey Wright [mailto:undefined@pobox.com]
> On Thu, 17 Jan 2013 08:43:21 +0000
> Fiedler Roman <Roman.Fiedler@ait.ac.at> wrote:
>
> > Hello List,
> >
> > What would be the most suitable solution to avoid high load/overlong cron
> job execution for hourly/daily/... jobs? When using similar vservers, all of
> them would start those jobs at the same time.
>
> depends (imho):
>
> providing/using a hosting service and want guarantees? cgroup blkio
> controller.
> cooperative administration of vservers? nice and/or ionice.
> "poor man's" approach? ${RANDOM}.

So many ways to do it ....

Perhaps we should add a link to this discussion in the FAQs page.

> i haven't tested/used the cgroup's blkio controller, but in theory that's
> what it's suppose to do (cgroup's other controllers work for me).

OK, I will look on this one too.

> > Currently I'm using a script on the host to run no more than N guest
> hourly/daily/... jobs in parallel.
>
> i presume you are administering them on your own and/or somebody else's
> behalf and can schedule them cooperatively.

Right

> i wouldn't expect this level of
> cooperation from/among users of a hosting service (ie system administration
> by committee).

I was thinking about that problem also to see if there could be a more "generic" solution. The basic reasoning for un-cooperative usecase was:

* Since no "nice" cooperation, provider and consumer are linked via contract and SLAs only
* Contract guarantees number of CPU-cycles/IO-ops per timeslice, e.g. >0.1 CPU-core-equivalent
* Consumer can run jobs at any time, no cooperation but he can rely to get at least guaranteed execution speed
* Provider must at least provide n(guests) * minGuarantee resources or do optimization gambling to go with less
 
Hence in this case, all that the host implementation could do is to ensure that host-resources are available and per-guest quotas are not exceeded. I would expect, that on such a setup much of the resources would be idle due to large resource reserve, that has to be kept.

> i do cooperative scheduling of some jobs and do it sequentially on each
> vserver from the host, but that's because i'm the only administrator/user of
> my vservers and host them myself (and it's simple/stupid).
>
> if i was using a vserver provided by someone else (eg hosting service), then
> i would either expect them to provide guaranteed io throughput (just like for
> cpu and memory) or i'd randomize it myself (based on how much i was paying;
> poor man's solution).

Also my opinion, except that it might be hard to handle "randomization" in SLAs accordingly.

> i've already seen randomizing as a solution to not overloading resources
> among controlled but uncoordinated systems: system/virus updates are
> randomly
> staggered on corporate desktops so as to not overload the server and/or
> congest the network.

Seems the best way for jobs with quite some resource consumption but run seldomly and where exact execution time is not so much an issue.
Received on Fri Jan 18 08:41:17 2013

[Next/Previous Months] [Main vserver Project Homepage] [Howto Subscribe/Unsubscribe] [Paul Sladen's vserver stuff]
Generated on Fri 18 Jan 2013 - 08:41:17 GMT by hypermail 2.1.8