TooBasic: Shell Tools and Crons

What's a Shell Tools?

Let's say your site is so complex it requires some kind of external process to maintain it, some kind of script you can run directly in the server and not through an URL. A shell tool is one of those processes written in a way TooBasic can handle.

Once again, let's use a simple example to understand how it works. Suppose we have a strict site where inactive users get removed, so, if you haven't logged in the last 3 months you loose your account. For this, you created a model saved at ROOTDIR/site/models/Users.php that allows you to remove inactive users and you've tried to use it in your log-in service. Using it there was a good idea at first, but your site now have thousands and thousands of users and that operation is taking a long time. Let's port that idea into a shell tool so it could be executed in the server whenever your site admin sees it fit.

Creating a shell tool

Following the example we're going to write the next code and save it in ROOTDIR/site/shell/tools/users.php:

<?php
class UsersTool extends \TooBasic\Shell\ShellTool {
    protected function mainTask($spacer = "") {
        echo "{$spacer}Removing invalids: ";
        $this->model->Users->removeInvalids();
        echo "Done\n";
    }
    protected function setOptions() {}
}

Now we can execute this tool directly in our server with a command like this one:

$ sudo -u www-data php /var/www/mysite/shell.php tool users

Assumptions:

  • You are using a *nix operation system.
  • php is inside a directory named in $PATH.
  • Your document root is /var/www.
  • You installed TooBasic in /var/www/mysite.
  • Your web user is www-data

Why sudo? well, we don't want to have access problems later.

Let's make things interesting

Let's suppose your model allows you to know which users are going to be removed and you want to list them before anything. For this we're going to add the option -l and give it some meaning.

<?php
class UsersTool extends \TooBasic\Shell\ShellTool {
    protected function mainTask($spacer = "") {
        echo "{$spacer}Removing invalids: ";
        $this->model->Users->removeInvalids();
        echo "Done\n";
    }
    protected function setOptions() {
        $this->_options->setHelpText("This tool allows to perform certain tasks related with users.");
        $opt = new \TooBasic\Shell\Option("ListInvalids");
        $opt->setHelpText("Prompts a list of users that have to be removed due to inactivity.");
        $opt->addTrigger("-l");
        $opt->addTrigger("--list");
        $this->_options->addOption($opt);
    }
    protected function taskListInvalids($spacer = "") {
        echo "{$spacer}Invalids users:\n";
        foreach($this->model->Users->invalids() as $user) {
            echo "{$spacer}\t- {$user->name} ({$user->id})\n";
        }
    }
}

Now we can execute this command:

$ sudo -u www-data php /var/www/mysite/shell.php tool users -l

What's going on here?:

  • We added a help text to explain our tool.
  • We created a new option with its help text and triggers, and added it to our options.
  • And we created a methods taskListInvalids() for that option.

Now we can do this:

$ sudo -u www-data php /var/www/mysite/shell.php tool users -l
Invalids users:
    - john42 (6532)
    - daemonraco (666)
$ sudo -u www-data php /var/www/mysite/shell.php tool users -h
This tool allows to perform certain tasks related with users.

    --help, -h
        Shows this help text.

    --version, -V
        Shows this tool's version number.

    --info, -I
        Shows this tool's information.

    -l, --list
        Prompts a list of users that have to be removed due to inactivity.

But, How? Yes, you may not see the connection here. A shell tool assumes that every option has a method that responds to it, if not mainTask() is called. In our example we created an option called ListInvalids, that means there may be a method called taskListInvalids() to attend it.

And -h? By default, every shell tool has some general options to show:

  • Its help text (--help or -h).
  • Its version number (--version or -V).
  • Some extra information about it (--info or -I).

Things we didn't explain

When you are creating a shell tool you must first pick a name in lower-case without special characters (geeky info: /([a-z0-9_]+)/), in our example it was users. Then you must create a class using that name (in camel-case) and add the suffix Tool, something like UsersTool. It must inherit from \TooBasic\Shell\ShellTool. Finally you must store it in ROOTDIR/site/shell/tools/users.php and that's it.

Recommendation

We recommend you to create a tool like the one in our example in this way:

<?php
use \TooBasic\Shell\Option as TBS_Option;
class UsersTool extends \TooBasic\Shell\ShellTool {
    protected function setOptions() {
        $this->_options->setHelpText("This tool allows to perform certain tasks related with users.");
        $this->_options->addOption(TBS_Option::EasyFactory("ListInvalids", ["-l","--list"], TBS_Option::TYPE_NO_VALUE, "Prompts a list of users that have to be removed due to inactivity."));
        $this->_options->addOption(TBS_Option::EasyFactory("RemoveInvalids", ["-rm","--remove-invalids"], TBS_Option::TYPE_NO_VALUE, "Removes users that have become invalid due to inactivity."));
    }
    protected function taskListInvalids($spacer = "") {
        echo "{$spacer}Invalids users:\n";
        foreach($this->model->Users->invalids() as $user) {
            echo "{$spacer}\t- {$user->name} ({$user->id})\n";
        }
    }
    protected function taskRemoveInvalids($spacer = "") {
        echo "{$spacer}Removing invalids: ";
        $this->model->Users->removeInvalids();
        echo "Done\n";
    }
}

In this way you force the user to give a parameter specifying what they want to do. Also, you are using a simpler way to add options.

Cron tools

What is a cron tool? Well, basically it is a shell tool but it controls that only one instance of a task is running at the same time.

In our example, if more than one user or an automatic process is using the option to remove invalid users, it may cause collisions when removing historic files, cleaning database tables, etc. This is an excellent example of a task that must run one at a time.

Create a cron tool

At this point we've realized that our shell tool must be a cron tool and we're going to make some changes:

  • Move it to ROOTDIR/site/shell/crons/users.php.
  • Change its class name and inheritance:
    <?php
class UsersCron extends \TooBasic\Shell\ShellCron {
  . . .

Now you can call it this way:

$ sudo -u www-data php /var/www/mysite/shell.php cron users -rm

How does it work?

Every time you call a cron tool task, it:

  • creates a flag file in ROOTDIR/cache/shellflags to refer the running task (for example UsersCron_taskRemoveInvalids.flag),
  • execute the task and
  • removes the flag file.

If a cron tool task is called and there's already a flag file, it avoids executing the task and prompts an error.

Dead flags

Sometimes when a cron tool fails due to unexpected errors or development bugs, the cron tool may not remove the flag file and return and error of other instance running when there's none. For those cases you can add the flag -CF and it will clear the flag without executing the actual task. Something like this:

$ sudo -u www-data php /var/www/mysite/shell.php cron users -rm -CF

Profiles

If your site gets bigger, you'll probably end up with many cron tools and a lot of entries in your crontab, and this may be a little messy unless your are a really organized administrator. To avoid this, TooBasic provides a mechanism to group a list of cron executions under a name and then run them all together with one command line.

Let's make an example to make this easier. Suppose you have four cron tools you run one after another with a specific set of parameters, something like this (crontab like):

#
# m h  dom mon dow   command
00 00 * * * /usr/bin/php /www/mysite/shell.php cron users --clean-inactives
15 00 * * * /usr/bin/php /www/mysite/shell.php cron users --remove-banned
30 00 * * * /usr/bin/php /www/mysite/shell.php cron posts --remove-spam
45 00 * * * /usr/bin/php /www/mysite/shell.php cron comments --kick-impolite severe

In this case you only see four crontab line, but in real life there may be even more. Also, each command executes 15 minutes after the previous one, but there's no warranties for overlapping processes.

Now let's say we create a profile called mysite_cron in which we collect all these crontab line into one. To accomplish this we are going to add a configuration like this into, let's say, ROOTDIR/site/configs/config_shell.php

<?php
$CronProfiles["mysite_cron"] = [
    [
        GC_CRONPROFILES_TOOL => "users",
        GC_CRONPROFILES_PARAMS => ["--clean-inactives"]
    ], [
        GC_CRONPROFILES_TOOL => "users",
        GC_CRONPROFILES_PARAMS => ["--remove-banned"]
    ], [
        GC_CRONPROFILES_TOOL => "posts",
        GC_CRONPROFILES_PARAMS => ["--remove-spam"]
    ], [
        GC_CRONPROFILES_TOOL => "comments",
        GC_CRONPROFILES_PARAMS => ["--kick-impolite", "severe"]
    ]
];

Now, let's change our crontab configuration into something like this:

#
# m h  dom mon dow   command
00 00 * * * /usr/bin/php /www/mysite/shell.php profile mysite_cron

This way, your crontab is cleaner and you know each task will start when the previous one ends.

Another interesting feature this mechanism provides is to let modules to insert their own cron tools in a profile.

Aliases

Something that can be a little tedious is to write a log list of parameters over and over again, so there's a shortcut mechanism for that kind of situations.

Let's go back to our examples and say that you run this line quite a lot:

$ sudo -u www-data php /var/www/mysite/shell.php tool users -l

Now imagine that you could run something like this:

$ sudo -u www-data php /var/www/mysite/shell.php ulist

Still a bit long, but if you are in the right place with the right permissions, you may end up running this:

$ php shell.php ulist

But how? Of course, somewhere in your site there's a configuration telling TooBasic to understand ulist as tool users -l and that's what we're going to explain in the next section.

Configuration

Following the examples we can add a code like the ext one to our sites configuration:

$Defaults[GC_DEFAULTS_SHELLTOOLS_ALIASES]['ulist'] = [
    'sys',
    'shell',
    'remove'
];

Using this configuration, TooBasic will expand command line parameters before executing the proper command line. In other words, you write something shorter, but it executes the long way.

Rules

This alias mechanism works only with the first parameter and not with the rest. If you run something like the next line it won't work:

$ sudo -u www-data php /var/www/mysite/shell.php -someparam ulist

Also, if the alias is a core parameter like cron, profile, etc., it will be ignored as if it's not defined.

Suggestions

If you want or need, you may visit these documentation pages:

results matching ""

    No results matching ""