On Thu, Jan 17, 2013 at 10:35:43AM +0000, Bendtsen, Jon wrote:
> On 17/01/2013, at 11.05, Fiedler Roman <Roman.Fiedler@ait.ac.at> wrote:
>>> -----Ursprüngliche Nachricht-----
>>> Von: Bendtsen, Jon [mailto:Jon.Bendtsen@laerdal.dk]
>>> On 17/01/2013, at 10.07, Fiedler Roman <Roman.Fiedler@ait.ac.at>
>>> wrote:
>>>>> -----Ursprüngliche Nachricht-----
>>>>> Von: Bendtsen, Jon [mailto:Jon.Bendtsen@laerdal.dk]
>>>>> On 17/01/2013, at 09.43, Fiedler Roman <Roman.Fiedler@ait.ac.at>
>>> wrote:
> [cuuuuuut]
>>> Or maybe just pick some random time, sort of what old
>>> ethernet did when there is a collision because 2 or more
>>> computers tried to transmit data at the same time.
>> Yes. Or use the context-ID modulo something.
> good idea
> [cuuuuuut]
>>> I see your point. If so maybe we should split up
>>> daily/weekly/hourly cronjobs even further, such that all the
>>> same programs which are hard linked together can use the same
>>> cache and ram for instructions. But running locate to update
>>> the database might not be a good choice to all run at the
>>> same time.
>> I would expect, that caching of the application code itself
>> is not the main performance boost. It is more about getting
>> rid of the bottle-necks. As you noted, too many locate-updates
>> in parallel will kill disk performance. But also loosing the
>> usual disk-cache benefit might be problematic. If e.g. just a
>> few databases run a job, the relevant db-content from disk is
>> likely the end up completely in OS RAM cache. All disk-reads
>> from db will return immediately. When too many DBs execute in
>> parallel, disk content of one process will be put in cache,
>> eliminating pages soon need again by another job again.
> This is not really any different from running many other kinds
> of virtual machines in parallel. Your storage system needs to
> perform.
if the guest is 'friendly' and basically controlled by the
host admin, it should suffice to put crond in the idle I/O
class and renice it a little.
>>> Maybe we need kind of scheduler in the kernel that notices
>>> which processes in the different guests are hard linked and
>>> then prioritizing running those?
>>> Maybe we need some new kind of scheduler system that is made
>>> with virtualization in mind, such that it adjusts to when
>>> there is a low load and then tries to run the maintenance
>>> scripts, meaning that sometimes scripts are run with only 20
>>> hours between them, other times it might be 36 hours.
>> In my opinion, such an intelligent/learning scheduler would
>> allow significance increase in execution performance, but
>> looking of the simplicity of current cron, I think, that such
>> a program is years away, if even written ever.
> probably. But I can imagine other ways to do it. I already use
> the at system for some tasks. Like my backup scripts. If the
> load is too high or the system is other wise not ready, I have
> my backup scripts call 'at now + 1 hour' to run the script
> itself 1 hour later. If it is still not ready, I call +2 hours,
> +4 and finally +8 hours.
> But it all hinges on the guest knowing the over all system load
> of the virtualization host, just like a process in a normal
> stand alone system can see the system load. Maybe we need a new
> field in the load data? Or a new syscall to give those data?
you can simply provide that information to a guest from
a cron job on the host (e.g. via a bind mounted file or
simply by echoing the necessary information into the
(root) filesystem of each guest.
> [cuuuuut]
>>> I am not starting cron daemon with nice. I put nice in side
>>> the crontab file, like these examples:
>>> 37 0 * * * root nice -n 15
>>> /usr/local/sbin/AD_integration/find_disabled_users_from_AD_in_groups.sh
>>> 0,15,30,45 * * * * root nice -n 5
>>> /usr/local/sbin/AD_integration/merge_AD_groups_with_unix.sh
>> Ah I see.
>> My current solution is:
>> * Check for each guest, if cron scheduler is installed inside
>> * Check if guest cron would run hourly/daily by himself, if
>> yes let him do it. A misconfigured/malicious guest can always
>> run any process at any time, this has to be addressed via
>> other means
>> * If guest cron is installed, cron.daily/hourly ...
>> directories exist but guest scheduler does NOT run those jobs
>> from etc/crontab (cooperative guest) by itself, then start
>> those via vserver-exec
>> * Make sure to run only a given number of those guest
>> processes in parallel
>> So all you need to do is to install cron in guest but remove
>> run-scripts directives for hourly/daily from /etc/crontab to
>> opt in.
> Yes, that is a method too, but I would not really like that
> outsiders, like hosting providers ran tasks inside my guest, at
> least not without my beforehand knowledge. I guess it works if
> one administrates both the host and the guests.
I think it might be just fine if the procedure is well
documented and most importantly well tested. i.e. 95%
of all customers will except whatever the provider offers
as environment without changing anything at all, the
remaining 5% might just setup their cron jobs differently
without causing a big problem for the system.
as usual, it's all about communication :)
best,
Herbert
> JonB
Received on Fri Jan 18 05:57:27 2013