[9.x] Introduce artisan docs command

[timacdonald]

(sorry for the length of this description. Just wanted to get everything out of my head)

This PR introduces a new artisan command focused on giving developers quick access to documentation pages of the Laravel documentation site.

$ php artisan docs

The introduction of this feature was driven by my constant want to be able to access specific pages in the documentation quickly, such as the collection methods or validation rules, and is also inspired by the npm docs {package} command that opens the repository (and generally documentation) for different packages:

$ npm docs picomatch

For developers that spend a lot of time with artisan and a lot of time in the terminal, often you find yourself aliasing php artisan to a. With this in mind, I can now access the unique validation rule documentation with the following command:

$ a docs va un

> Opening the docs to: https://laravel.com/docs/9.x/validation#rule-unique

The new command also provides ways to perform an Algolia powered general search across the documentation site, as well as a few other handy features.

artisan docs

The bare artisan docs command takes the approach of asking the user what page they would like to visit. This is done via the choice mechanism, which is useful for asking for input from a known list of options.

The choice list is populated with the top level pages of the Laravel documentation site.

artisan-docs.mov

Default choice

The default choice for the interaction is installation. This allows developers to simply hit <Enter> and they will be taken to the entry point for the documentation (i.e. the same page you land on when you visit https://laravel.com/docs. See the following example where <Enter> represents you hitting the Enter / Return key on your keyboard.

$ php artisan docs<Enter><Enter>

> Opening the docs to: https://laravel.com/docs/9.x/installation

Alternatively, the command also respect the global --no-interaction flag, so the following will also take you to the entry point of the documentation without asking you for input:

$ php artisan docs -n

> Opening the docs to: https://laravel.com/docs/9.x

Note: Casing

In order to provide a good user experience when using the choice mechanism, the optional key passed to the choice command uses a lower-cased version of the page title. This is why you will see the following in the output:

[laravel mix    ] Laravel Mix
[laravel octane ] Laravel Octane
 
 ^ lower case     ^ normal case

This means that users do not have to perfectly match the casing of the title, and can enter, for example: validation or Validation and will get the correct matches. Without this, only Validation would match correctly. This is a limitation of the choice mechanism, but I think this solution works well enough.

artisan docs {page}

After using the command a few times, or when knowing where it is you want to go, it becomes nice to no longer be asked where you want to open, but instead to tell the command where you want to go.

The first argument of the command allows developers to specify a page to open.

$ php artisan docs validation

> Opening the docs to: https://laravel.com/docs/9.x/validation

Note I've made it somewhat of a rule that the command always results in opening the docs, so there isn't a blocking step even on some kind of failure.

The command attempts to be smart when trying to determine the page the user is attempting to visit. First the command finds any page titles that begin with the argument provided, thus I don't need to enter validation completely, instead I can just enter va:

$ php artisan docs va

> Opening the docs to: https://laravel.com/docs/9.x/validation

If multiple pages begin with the argument, then the first match is selected, which results in the first matching page from the Laravel documentation sidebar being selected. To demo this, take the following command:

$ php artisan docs v

This command could match either "Validation" or "Views". However because "Views" comes first in the documentation sidebar, it is matched.

$ php artisan docs v

> Opening the docs to: https://laravel.com/docs/9.x/views

If no pages start with the given argument, then the command attempts to "guess" what page the user was requesting. This is done with the similar_text function and also by giving priority to matches that contain the given argument exactly.

Given the misspelling of "views"

$ php artisan docs veiws

> Opening the docs to: https://laravel.com/docs/9.x/views

or given "ora" which is contained in "File stORAge". (remember that the matches are based on the page title, not the URL slug)

$ php artisan docs ora

> Opening the docs to: https://laravel.com/docs/9.x/filesystem

I have found the current algorithm to be rather forgiving and seems to work pretty well. No doubt when the community take a look there could be potential improvements. Perhaps a finely tuned levenshtein call would improve results, but I've personally been really happy with how it currently works and I'm not concerned about cost factor as it is all in-memory and not a production concern at all.

The algorithm does reject low quality matches (< 3) unless the argument passed in less than 3 characters long. There is probably room for improvement here as well where "low quality" always determined by the length of the argument.

Finally, the argument and page title are slugified (is that a word?!) before matching. This allows developers to not have to escape spaces in their arguments (although you still can) and instead utilise - characters instead.

This means that a docs art-con is equivilent to calling a docs art\ con

$ php artisan docs art-con

> Opening the docs to: https://laravel.com/docs/9.x/artisan

That being said, php artisan docs a does the same thing due to the previously discussed matching criteria.

artisan docs {page} {section}

Now that you can quickly and easily get to the validation page, you’re going to be soon asking yourself:

🤔 Why the heck can't I just go straight to the available rules?!?

- You (probably)

Well have I got good new for you!

$ php artisan docs va rules

> Opening the docs to: https://laravel.com/docs/9.x/validation#available-validation-rules

And again, based on the above rules the shorter php artisan docs va ru also opens this link.

The section argument has the same guessing logic that the page argument has, however it only looks at sections of the already determined page.

There is future scope to improve this further still be providing extra weighting to higher level headings, I.e. h2 headings might get priority over h3 headings. Everything has been designed with this kind of enhancement in mind.

php artisan docs -- query string here

In addition to being able to target specific pages, the command also allows you to open a global Algolia powered search on the Laravel documentation.

$ php artisan docs -- service container

> Opening the docs to: https://laravel.com/docs/9.x?q=service%20container

which will result in the following on page load...

I've documented this affordance in the commands help:

Documentation version

The application version is read from the Illuminate\Foundation\Application::version() to generate the documentation links. This means that when Laravel X is released (🕵️‍♂️), projects using it will be directed to the appropriate documentation version.

$ composer create-project laravel/laravel:^10.0

$ php artisan docs -n

> Opening the docs to: https://laravel.com/docs/10.x

Opening the URL

Out of the box, the command comes with some common ways of opening URLs across operating systems.

Window: start {url}
Mac: open {url}
Linux: xdg-open {url}

However, this doesn't cover every possible system. The command allows developer to customise how URLs are opened. But what's more is that the way this is customised is not on a project by project basis, but instead on a system level basis.

This means that I can customise the way the artisan docs command opens URLs once on my system, and it will be used across multiple projects without customisation within the project itself.

To customise the way the command opens URLs, a user may create an environment variable in their shell, that points to a PHP script.

For example, with ZSH on Mac I could do the following...

# file: /Users/tim/.zshrc

export ARTISAN_DOCS_OPEN_STRATEGY="/Users/tim/artisan-open.php"

then at the path specified above, you can create a PHP script that returns a function. The function receives the URL to open as the first parameter.

<?php

// file: /Users/tim/artisan-open.php

return fn ($url) => dump("my custom handler to open: {$url}");

This is nice as it allows each user to globally customise how their personal system opens URLs via the artisan docs command, however it is also interesting as it allows developers to come up with interesting things and share them as well.

It would be plausible that someone could, instead of opening the URL in the browser, instead pipe the documentation as some kind of terminal friendly output, so you never leave the terminal. This could be done with something like tty-markdown or maybe termwind, however for this to be a first party solution, I believe we would need to put some work across the docs to make it happen. I would absolutely think this is something we should investigate further and potentially introduce behind a flag in 9.x and then, if it works well, we could make it the default in 10.x.

The shape of the docs json representation generated by the Laravel website has been specifically designed with this and other future enhancements in mind.

The command also tries to recover from bad openers. This is again so that the user always lands on the docs even when an error occurs.

Caching

When the command is first used, the data required to power the command is downloaded from the Laravel website.

This data is then cached for 2 months. During this month, whenever the command is used again, the cache is used to determine the page to visit, however the cached data is also lazily refreshed in the background after the command has opened the user to the documentation site.

This means that the user doesn't have to wait for the data to be downloaded every time the command runs, but also gets the benefit of having fresh data in the cache if used often.

Custom ask strategies

Although the Symfony choice mechanism does work, it isn't as user friendly as I had hoped. This drove me to see what else I could come up with for the command to improve the experience of being asked a question, which might drive others to come up with interesting approaches as well.

I'm not 100% sure this feature should be included, but it is novel, so I've left it in for now.

Just like with opening the URL, a developer may globally change how they are asked for pages from the command.

On my system I have installed the cross-platform fzf fuzzy finder. I'm going to create a strategy to allow me to use FZF in place of the choice mechanism.

# file: /Users/tim/.zshrc

export ARTISAN_DOCS_ASK_STRATEGY="/Users/tim/artisan-ask.php"

Then in my ask strategy I return a function. The function accepts the command as it's first argument. Note I have full access to Laravel and other installed dependencies - however you should keep your requirements to a minimum as they need to be cross-project.

<?php

// file: /Users/tim/artisan-ask.php

use Illuminate\Support\Str;
use Symfony\Component\Process\Exception\ProcessFailedException;
use Symfony\Component\Process\ExecutableFinder;
use Symfony\Component\Process\Process;

return function ($command) {
    $path = tempnam(sys_get_temp_dir(), 'artisan-docs-fzf');

    $options = $command->pages()->flatMap(fn ($page, $pageSlug) => [
        $pageSlug . ': ' . $page['title'],
        //
        // Un-comment this to include page sections as well...
        //
        // ...$command->sectionsFor($pageSlug)->map(fn ($section, $sectionSlug) => $pageSlug.'#'.$sectionSlug.': '.$page['title'].' / '.$section['title'])->all()
        //
    ]);

    $process = tap(Process::fromShellCommandline(sprintf(
        "echo %s | %s > %s",
        escapeshellarg($options->implode(PHP_EOL)),
        escapeshellarg((new ExecutableFinder())->find('fzf')),
        escapeshellarg($path)
    ))->setTty(true))->run();

    if (! $process->isSuccessful()) {
        throw new ProcessFailedException($process);
    }

    return Str::before(file_get_contents($path), ':');
};

Now I am using fzf as my ask strategy for ultimate goodness...

artisan-docs-fzf.mov

Taking it for a spin

This PR is dependent on some other PRs (list below) to "just work". In the meantime however, you may run the following script on a Mac to get it working locally. You may need to make adjustments for other systems.

composer create-project laravel/laravel artisan-docs-command \
  && cd artisan-docs-command \
  && wget -O pages.json https://github.com/laravel/laravel.com/files/9178424/pages.json.txt \
  && mkdir app/Console/Commands \
  && wget -O app/Console/Commands/DocsCommand.php https://raw.githubusercontent.com/timacdonald/framework/artisan-docs/src/Illuminate/Foundation/Console/DocsCommand.php \
  && sed -i "s/namespace Illuminate\\\\Foundation\\\\Console/namespace App\\\\Console\\\\Commands/" app/Console/Commands/DocsCommand.php \
  && echo "<?php namespace App\Providers;use Illuminate\Support\Facades\Http;use Illuminate\Support\ServiceProvider;class AppServiceProvider extends ServiceProvider {public function boot() {Http::preventStrayRequests()->fake(['https://laravel.com/docs/9.x/index.json' => Http::response(file_get_contents(base_path('pages.json')))]);}}" > app/Providers/AppServiceProvider.php

Future scope

• Showing docs in the terminal as previously mentioned
• Passing a class / namespace and being taken to the most relevant docs for that FQCN
• Give weighting to different heading levels for sections.
• Improved first-party console based fuzzy finding / matching.

Dependent PRs

• Add JSON representation of the docs for the artisan docs command laravel.com#242
• Allow search to be prefilled for artisan docs command laravel.com#243

[timacdonald]

I'll address the failing tests shortly.

[nunomaduro]

@timacdonald Can you replace calls to $this->info and $this->warn to $this->components->info and $this->components->warn.

Same goes for choice, yet i am not sure if my components supports yet everything you are using.

[timacdonald]

Will do Nuno. Thanks for the reminder!

[michaelnabil230]

Nice idea 👍🏻

[aneesdev]

@timacdonald isn't there a way to display docs content (markdown/html) directly inside cli instead of opening docs URL 😅 ?

[timacdonald]

@aneesdev as mentioned in description, that is absolutely a possibility for future enhancement and also something that this PR would actually enable via a custom opener. I believe we would need to put in a bit of work across the docs before there was a truly clean 1st party way of doing this however. But this is absolutely something I am keen to pursue further.

[timacdonald]

Updated screenshots using out internal components...

[timacdonald]

Bumped this as there is a bump in lower versions parsing escaped spaces in arguments, which was breaking my test for an escaped value.

[taylorotwell]

@timacdonald can you go ahead and register the command in ArtisanServiceProvider in this PR so it's easier to try?

[timacdonald]

Done.