Skip to content

Commit

Permalink
Add option --interval; improve validation; refs #83
Browse files Browse the repository at this point in the history
  • Loading branch information
@jigarius
jigarius committed Dec 28, 2024
1 parent acbc117 commit 874fdfa
Show file tree
Hide file tree
Showing 3 changed files with 183 additions and 65 deletions.
39 changes: 23 additions & 16 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ interrupt signal is received, then Drall stops immediately.

In this method, the `--uri` option is sent to `drush`.

drall exec drush --uri=@@dir core:status
drall exec -- drush --uri=@@dir core:status

If it is a Drush command and no valid `@@placeholder` are present, then
`--uri=@@dir` is automatically added after each occurrence of `drush`.
Expand All @@ -117,7 +117,7 @@ If it is a Drush command and no valid `@@placeholder` are present, then
# Raw drush command (no placeholders)
drall exec drush core:status
# Command that is executed (placeholders injected)
drall exec drush --uri=@@dir core:status
drall exec -- drush --uri=@@dir core:status
```

##### Example
Expand Down Expand Up @@ -174,17 +174,16 @@ cat web/sites/ralph/settings.local.php
#### Options

For the `drall exec` command, all Drall options must be set right after
`drall exec`. For example,
`drall exec`. Additionally, `--` must be used before the command to be
executed. Following are some examples of running `drush` with options.

```shell
# Correct: Drall gets --verbose.
drall exec --verbose drush core:status
# Incorrect: --verbose is ignored.
drall exec drush core:status --verbose
# Correct: Drall and Drush, both --verbose.
# Drall is --verbose
drall exec --verbose -- drush core:status
# Drush is verbose
drall exec -- drush --verbose core:status
# Both Drall and Drush are --verbose
drall exec --verbose -- drush --verbose core:status
# Incorrect: Only Drall gets --verbose.
drall exec --verbose drush core:status
```

In summary, the syntax is as follows:
Expand All @@ -195,6 +194,14 @@ drall exec [DRALL-OPTIONS] -- drush [DRUSH-OPTIONS]

Besides the global options, the `exec` command supports the following options.

#### --interval

This option makes Drall wait for `n` seconds after processing each item.

drall exec --interval=3 -- drush core:rebuild

Such an interval cannot be used when using a multiple workers.

#### --workers

Say you have 100 sites in a Drupal installation. By default, Drall runs
Expand All @@ -213,7 +220,7 @@ conflict between the Drall workers.

The command below launches 3 instances of Drall to run `core:rebuild` command.

drall exec drush core:rebuild --workers=3
drall exec --workers=3 -- drush core:rebuild

When a worker runs out of work, it terminates automatically.

Expand All @@ -229,7 +236,7 @@ bar can be disabled using the `--no-progress` option.

##### Example: Hide progress bar

drall exec --no-progress drush core:rebuild
drall exec --no-progress -- drush core:rebuild

#### --dry-run

Expand All @@ -239,7 +246,7 @@ executing them.
##### Example: Dry run

```shell
$ drall exec --dry-run --group=bluish core:status
$ drall exec --dry-run --group=bluish -- drush core:status
drush --uri=donnie core:status
drush --uri=leo core:status
```
Expand Down Expand Up @@ -333,7 +340,7 @@ This section covers some options that are supported by all `drall` commands.
Specify the target site group. See the section *site groups* for more
information on site groups.

drall exec --group=GROUP core:status --field=site
drall exec --group=GROUP -- drush core:status --field=site

If `--group` is not set, then the Drall uses the environment variable
`DRALL_GROUP`, if it is set.
Expand All @@ -345,9 +352,9 @@ commands on specific sites.

```shell
# Run only on the "leo" site.
drall exec --filter=leo core:status
drall exec --filter=leo -- drush core:status
# Run only on "leo" and "ralph" sites.
drall exec --filter="leo||ralph" core:status
drall exec --filter="leo||ralph" -- drush core:status
```

For more on using filter expressions, refer to the documentation on
Expand Down
141 changes: 99 additions & 42 deletions src/Command/ExecCommand.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,14 @@ protected function configure() {
1,
);

$this->addOption(
'interval',
NULL,
InputOption::VALUE_OPTIONAL,
'Number of seconds to wait between commands.',
0,
);

$this->addOption(
'dry-run',
'X',
Expand All @@ -92,26 +100,22 @@ protected function configure() {
}

protected function initialize(InputInterface $input, OutputInterface $output): void {
if (!method_exists($input, 'getRawTokens')) {
parent::initialize($input, $output);
return;
}
$this->checkObsoleteOptions($input, $output);
$this->checkOptionsSeparator($input, $output);
$this->checkIntervalOption($input, $output);
$this->checkWorkersOption($input, $output);
$this->checkInterOptionCompatibility($input, $output);

// Parts of the command after "exec".
$rawTokens = $input->getRawTokens(TRUE);
parent::initialize($input, $output);
}

// If obsolete --drall-* options are present, then abort.
foreach ($rawTokens as $token) {
if (str_starts_with($token, '--drall-')) {
$output->writeln(<<<EOT
In Drall 4.x, all --drall-* options have been renamed.
See https://github.com/jigarius/drall/issues/99
EOT);
throw new \RuntimeException('Obsolete options detected');
}
private function checkOptionsSeparator(InputInterface $input, OutputInterface $output): void {
if (!method_exists($input, 'getRawTokens')) {
return;
}

// If options are present, an options separator (--) is required.
$rawTokens = $input->getRawTokens(TRUE);
if (!in_array('--', $rawTokens)) {
foreach ($rawTokens as $token) {
if (str_starts_with($token, '-')) {
Expand All @@ -127,8 +131,71 @@ protected function initialize(InputInterface $input, OutputInterface $output): v
}
}
}
}

parent::initialize($input, $output);
private function checkObsoleteOptions(InputInterface $input, OutputInterface $output): void {
if (!method_exists($input, 'getRawTokens')) {
return;
}

// If obsolete --drall-* options are present, then abort.
foreach ($input->getRawTokens(TRUE) as $token) {
if (str_starts_with($token, '--drall-')) {
$output->writeln(<<<EOT
In Drall 4.x, all <comment>--drall-*</comment> options have been renamed.
See https://github.com/jigarius/drall/issues/99
EOT);
throw new \RuntimeException('Obsolete options detected');
}
}
}

private function checkIntervalOption(InputInterface $input, OutputInterface $output): void {
$interval = $input->getOption('interval');

if ($interval < 0) {
$output->writeln(<<<EOT
The value for <comment>--interval</comment> must be a positive integer.
EOT);
throw new \RuntimeException('Invalid options detected');
}
}

private function checkWorkersOption(InputInterface $input, OutputInterface $output): void {
$workers = $input->getOption('workers');

if ($workers > self::WORKER_LIMIT) {
$limit = self::WORKER_LIMIT;
$output->writeln(<<<EOT
The value for <comment>--workers</comment> must be less than or equal to $limit.
EOT);
throw new \RuntimeException('Invalid options detected');
}
}

private function checkInterOptionCompatibility(InputInterface $input, OutputInterface $output): void {
if (
$input->getOption('workers') > 1 &&
$input->getOption('interval') > 0
) {
$output->writeln(<<<EOT
The options <comment>--interval</comment> and <comment>--workers</comment> cannot be used together.
EOT);
throw new \RuntimeException('Incompatible options detected');
}
}

protected function preExecute(InputInterface $input, OutputInterface $output): void {
parent::preExecute($input, $output);

$workers = $input->getOption('workers');
if ($workers > 1) {
$this->logger->notice("Using {count} workers.", ['count' => $workers]);
}

if ($interval = $input->getOption('interval')) {
$this->logger->notice("Using a $interval-second interval between commands.", ['interval' => $interval]);
}
}

protected function execute(InputInterface $input, OutputInterface $output): int {
Expand Down Expand Up @@ -159,7 +226,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
return 0;
}

$workers = $this->getWorkerCount($input);
$workers = $workers = $input->getOption('workers');

// Display commands without executing them.
if ($input->getOption('dry-run')) {
Expand Down Expand Up @@ -198,6 +265,7 @@ protected function execute(InputInterface $input, OutputInterface $output): int
$values,
$command,
$placeholder,
$input,
$output,
$progressBar,
$workers,
Expand All @@ -207,7 +275,15 @@ protected function execute(InputInterface $input, OutputInterface $output): int
yield ConcurrentIterator\each(
Iterator\fromIterable($values),
new LocalSemaphore($workers),
function ($value) use ($command, $placeholder, $output, $progressBar, &$exitCode, &$isStopping) {
function ($value) use (
$command,
$placeholder,
$input,
$output,
$progressBar,
&$exitCode,
&$isStopping,
) {
if ($isStopping) {
return;
}
Expand Down Expand Up @@ -237,6 +313,11 @@ function ($value) use ($command, $placeholder, $output, $progressBar, &$exitCode

$progressBar->advance();
$progressBar->display();

// Wait between commands if --interval is specified.
if ($interval = $input->getOption('interval')) {
sleep($interval);
}
}
);
});
Expand Down Expand Up @@ -291,30 +372,6 @@ private function getCommand(InputInterface $input, OutputInterface $output): ?st
return $command;
}

/**
* Gets the number of workers that should be used.
*
* @param \Symfony\Component\Console\Input\InputInterface $input
* The input.
*
* @return int
* Number of workers to be used.
*/
protected function getWorkerCount(InputInterface $input): int {
$result = $input->getOption('workers');

if ($result > self::WORKER_LIMIT) {
$this->logger->warning('Limiting workers to {count}, which is the maximum.', ['count' => self::WORKER_LIMIT]);
$result = self::WORKER_LIMIT;
}

if ($result > 1) {
$this->logger->notice("Using {count} workers.", ['count' => $result]);
}

return $result;
}

/**
* Get unique placeholder from a command.
*/
Expand Down
68 changes: 61 additions & 7 deletions test/Integration/Command/ExecCommandTest.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -668,6 +668,43 @@ public function testWithDryRunQuiet(): void {
EOF, $process->getOutput());
}

/**
* @testdox Shows error when --interval is negative.
*/
public function testNegativeInterval(): void {
$process = Process::fromShellCommandline(
'drall ex --interval=-3 -- drush st --fields=site',
static::PATH_DRUPAL,
);
$process->run();
$this->assertOutputEquals(<<<EOT
The value for --interval must be a positive integer.
EOT, $process->getOutput());
$this->assertOutputContainsString('Invalid options detected', $process->getErrorOutput());
$this->assertEquals(1, $process->getExitCode());
}

/**
* @testdox With --interval.
*/
public function testWithInterval(): void {
$process = Process::fromShellCommandline(
'drall ex --interval=2 --verbose -- ./vendor/bin/drush st --fields=site',
static::PATH_DRUPAL,
);
$process->run();
$this->assertOutputContainsString(
'[notice] Using a 2-second interval between commands.' . PHP_EOL,
$process->getOutput(),
);

// The command must take 2 * count($sites) seconds.
// This confirms that the sleep(2) command is actually executed.
$timeTaken = microtime(TRUE) - $process->getStartTime();
$this->assertGreaterThan(10, $timeTaken);
}

/**
* @testdox With --workers=2.
*/
Expand Down Expand Up @@ -697,20 +734,37 @@ public function testWithWorkers(): void {
}

/**
* @testdox With --workers=17.
*
* Drall caps the maximum workers to a pre-determined limit.
* @testdox Shows error when --worker limit exceeds the maximum.
*/
public function testWorkerLimit(): void {
$process = Process::fromShellCommandline(
'drall ex --workers=17 --verbose -- drush --uri=@@dir st --fields=site',
'drall ex --workers=17 -- drush st --fields=site',
static::PATH_DRUPAL,
);
$process->run();
$this->assertStringStartsWith(
'[warning] Limiting workers to 16, which is the maximum.' . PHP_EOL,
$process->getOutput(),
$this->assertOutputEquals(<<<EOT
The value for --workers must be less than or equal to 16.
EOT, $process->getOutput());
$this->assertOutputContainsString('Invalid options detected', $process->getErrorOutput());
$this->assertEquals(1, $process->getExitCode());
}

/**
* @testdox Shows error when --workers and --interval are used together.
*/
public function testIntervalWithWorkers(): void {
$process = Process::fromShellCommandline(
'drall ex --workers=2 --interval=2 -- drush st --fields=site',
static::PATH_DRUPAL,
);
$process->run();
$this->assertOutputEquals(<<<EOT
The options --interval and --workers cannot be used together.
EOT, $process->getOutput());
$this->assertOutputContainsString('Incompatible options detected', $process->getErrorOutput());
$this->assertEquals(1, $process->getExitCode());
}

/**
Expand Down

0 comments on commit 874fdfa

Please sign in to comment.