BLT Automated Test Suite will not run locally

[aweingarten]

The instructions here seem broken :(
https://blt.readthedocs.io/en/latest/CONTRIBUTING/#developing-blt-locally

Steps to reproduce:

Clone the blt repo.  Checkout the 10.x branch
cd blt
composer install
./vendor/bin/robo create:from-symlink
cd ../blted8
vagrant up

Fails with the following:

TASK [Run configured post-provision shell scripts.] ****************************
failed: [blted8] (item=../../../acquia/blt/scripts/drupal-vm/configure-solr.sh) => {"changed": false, "item": "../../../acquia/blt/scripts/drupal-vm/configure-solr.sh", "msg": "Could not find or access '../../../acquia/blt/scripts/drupal-vm/configure-solr.sh'\nSearched in:\n\t/vagrant/vendor/geerlingguy/drupal-vm/provisioning/files/../../../acquia/blt/scripts/drupal-vm/configure-solr.sh\n\t/vagrant/vendor/geerlingguy/drupal-vm/provisioning/../../../acquia/blt/scripts/drupal-vm/configure-solr.sh\n\t/vagrant/vendor/geerlingguy/drupal-vm/provisioning/files/../../../acquia/blt/scripts/drupal-vm/configure-solr.sh\n\t/vagrant/vendor/geerlingguy/drupal-vm/provisioning/../../../acquia/blt/scripts/drupal-vm/configure-solr.sh on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}
failed: [blted8] (item=../../../acquia/blt/scripts/drupal-vm/post-provision.php) => {"changed": false, "item": "../../../acquia/blt/scripts/drupal-vm/post-provision.php", "msg": "Could not find or access '../../../acquia/blt/scripts/drupal-vm/post-provision.php'\nSearched in:\n\t/vagrant/vendor/geerlingguy/drupal-vm/provisioning/files/../../../acquia/blt/scripts/drupal-vm/post-provision.php\n\t/vagrant/vendor/geerlingguy/drupal-vm/provisioning/../../../acquia/blt/scripts/drupal-vm/post-provision.php\n\t/vagrant/vendor/geerlingguy/drupal-vm/provisioning/files/../../../acquia/blt/scripts/drupal-vm/post-provision.php\n\t/vagrant/vendor/geerlingguy/drupal-vm/provisioning/../../../acquia/blt/scripts/drupal-vm/post-provision.php on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}

When I then work around by doing

vagrant ssh
cd vendor/acquia/blt
./vendor/bin/robo release:test

I then see the following

----------------------------------------------------------Acquia\Blt\Tests\Blt\AcCloudHooksTest::testSetupCloudHooks----------------------------------------------------------
Copying sandbox master to sandbox instance...

Executing /tmp/blt-sandbox-instance/vendor/bin/drush sql-drop...
Command failure is permitted.
------Begin command output-------
sh: 1: /tmp/blt-sandbox-instance/vendor/bin/drush: not found
------End command output---------

E

Time: 1.18 minutes, Memory: 6.00MB

There was 1 error:

1) Acquia\Blt\Tests\Blt\AcCloudHooksTest::testSetupCloudHooks
file_get_contents(/tmp/blt-sandbox-instance/blt/ci.blt.yml): failed to open stream: No such file or directory

/var/www/blt/vendor/symfony/phpunit-bridge/DeprecationErrorHandler.php:111
/var/www/blt/src/Robo/Common/YamlMunge.php:65
/var/www/blt/src/Robo/Common/YamlMunge.php:25
/var/www/blt/tests/phpunit/src/BltProjectTestBase.php:85

ERRORS!
Tests: 1, Assertions: 0, Errors: 1.
 [ExecStack]  Exit code 2  Time 01:16

[aweingarten]

BTW it looks like the file_get_contents(/tmp/blt-sandbox-instance/blt/ci.blt.yml): failed to open stream: No such file or directory can be solved by boosting PHP memory

[danepowell]

You cannot run BLT commands within the VM while also debugging BLT. This is a fundamental limitation of Vagrant / NFS and I don't think there's any workaround for it. I think we should just update the documentation to clarify that you cannot test BLT in a VM in this way. Any better ideas?

Specifically, there's a symlink from blted8/vendor/acquia/blt to ../../../blt. The problem is that this links to a location outside of the NFS mount, so it won't exist when you're in the VM.

[alexxed]

BTW it looks like the file_get_contents(/tmp/blt-sandbox-instance/blt/ci.blt.yml): failed to open stream: No such file or directory can be solved by boosting PHP memory

I have 4G and still get it

[alexxed]

I see no mention of vagrant in https://blt.readthedocs.io/en/latest/CONTRIBUTING/#developing-blt-locally.

./vendor/bin/robo release:test mentioned there works for me

[aweingarten]

@danepowell until the documentation is updated closing this seems premature? At this point few people have a local lamp stack anymore. I haven't set one up in about 4 years. Right now it's pretty much impossible for folks to contribute to BLT.

We could probably work around the symlink issue by getting the DVM for blted8 to point vendor/acquia/blt inside the vm to the /blt directory on the host.

[danepowell]

I think calling it impossible is a little strong. I don't use a LAMP stack very often either, but it took me all of 5 minutes to create the necessary local DBs for testing and install chromedriver, and I have no problems running the release tests locally dozens of times per day. Sounds like @alexxed was able to set it up too.

The only way I'm aware of to test inside the VM is to use NFS to mount BLT from the custom location, overriding what composer provides. But that makes debugging a lot harder. Additionally, running things inside the VM is an order of magnitude slower than not. I'm usually not even able to run composer update inside the VM without it completely hanging.

There's also no viable hybrid solution of running composer outside the VM and BLT tests inside, since vagrant will not mount the BLT directory via NFS over a BLT symlink on the host.

These are fundamental blockers to even considering using a VM for testing. If you know of any workarounds or can point to examples of other projects that have managed to make this work, I'd love a PR. But I've thought about it a lot and never came to a better solution than what we have now.

[aweingarten]

Both me and @ba66e77 were impacted by this. I'm not saying we are rocket scientists but we know things. Since we were blocked its likely going to be a blocker to others.

If doing this inside a VM is impossible (I am skeptical) then provide instructions for getting this working in an alternate fashion. Right now the current instructions do not meet the level of rigor set by other aspects of BLT.

[danepowell]

The only error I saw in this issue is related to the VM. If you are having problems with the normal test instructions (which don't involve the VM), can you open a new issue with more details? And if you know of a way of testing inside the VM, PRs would be welcome for that as well.