Skip to content
This repository has been archived by the owner on Feb 26, 2024. It is now read-only.

Avoid compilation when doing tests #469

Closed
JustinDrake opened this issue Jun 29, 2017 · 29 comments
Closed

Avoid compilation when doing tests #469

JustinDrake opened this issue Jun 29, 2017 · 29 comments

Comments

@JustinDrake
Copy link

I am writing tests (leaving the contract code intact) but truffle will recompile every time I run truffle test. Ideally truffle test would be smart about recompiling only files that have changed. As a quick fix, a flag to skip compilation would do for me 👍

@JustinDrake
Copy link
Author

I see there's the --compile-all option. I wonder why the default contracts generated by truffle init get recompiled every time.

--compile-all: Compile all contracts instead of intelligently choosing.

@jellegerbrandy
Copy link

I have the same problem:

jelle@:~/projects ★ truffle version
Truffle v3.3.0, bundle version: 3.3.1
Solidity v0.4.11 (solc-js)
jelle@:~/projects/daostack/play ★ mkdir play
jelle@:~/projects/daostack/play ★ cd play
jelle@:~/projects/daostack/play ★ truffle init
Downloading project...
Project initialized.

  Documentation: http://truffleframework.com/docs

Commands:

  Compile: truffle compile
  Migrate: truffle migrate
  Test:    truffle test

jelle@:~/projects/play ★ truffle compile
Compiling ./contracts/ConvertLib.sol...
Compiling ./contracts/MetaCoin.sol...
Compiling ./contracts/Migrations.sol...
Writing artifacts to ./build/contracts

jelle@:~/projects/play ★ truffle test
Using network 'development'.

Compiling ./contracts/ConvertLib.sol...
Compiling ./contracts/MetaCoin.sol...
Compiling ./test/TestMetacoin.sol...
Compiling truffle/Assert.sol...
Compiling truffle/DeployedAddresses.sol...


  TestMetacoin
    ✓ testInitialBalanceUsingDeployedContract (44ms)
    ✓ testInitialBalanceWithNewMetaCoin (38ms)

  Contract: MetaCoin
    ✓ should put 10000 MetaCoin in the first account
    ✓ should call a function that depends on a linked library (64ms)
    ✓ should send coin correctly (120ms)


  5 passing (594ms)

jelle@:~/projects/play ★ truffle test
Using network 'development'.

Compiling ./contracts/ConvertLib.sol...
Compiling ./contracts/MetaCoin.sol...
Compiling ./test/TestMetacoin.sol...
Compiling truffle/Assert.sol...
Compiling truffle/DeployedAddresses.sol...


  TestMetacoin
    ✓ testInitialBalanceUsingDeployedContract
    ✓ testInitialBalanceWithNewMetaCoin

  Contract: MetaCoin
    ✓ should put 10000 MetaCoin in the first account
    ✓ should call a function that depends on a linked library (59ms)
    ✓ should send coin correctly (120ms)


  5 passing (572ms)

@dstreppa
Copy link

Not the solution for you, but maybe the following command can be helpful:

$ truffle test test/TestScrooge.sol 

Only TestScrooge.sol and its imported contracts will be compiled by Truffle, and only the tests in TestScrooge.sol will be executed.

@jellegerbrandy
Copy link

jellegerbrandy commented Jul 16, 2017

i resolved this for my case: apparently, the problem was due to outdated solc or solc-cli packages.

@travs
Copy link

travs commented Aug 1, 2017

@tcoulter
Is this a duplicate of the second part of #343 ?

@dkent600
Copy link

dkent600 commented Jan 3, 2018

You could also try running truffle compile separately before running truffle test. Has worked for me.

@PaulLaux
Copy link

Had the same problem, fixed by compiling in advance for the same network that is used for the tests:

first:
truffle.cmd compile --network testrpc
then:
truffle.cmd test --network testrpc test\bigTest.test.js

@zulhfreelancer
Copy link

zulhfreelancer commented Jun 29, 2018

Running truffle compile followed by truffle test [path] did not work for me. I'm using Truffle v4.1.13.

Everytime I ran truffle test [path], all other contracts get compiled and deployed.

I'm expecting only the contracts related in my test will get compiled and deployed. Not everything.

Update

I found a workaround for this. I create an empty folder in my Truffle project called migrations_null and I put nothing inside it.

Next, I ran the test with --migrations_directory flag and it worked. Other contracts didn't get compiled and migrated.

$ truffle tests ./test/MyTest.js --migrations_directory migrations_null

@eduardonunesp
Copy link

@gnidan any fix for this issue? beyond the workaround? for version 4.1.15

@0xferit
Copy link

0xferit commented Oct 19, 2018

There should be an option to disable compiling also.

I needed to debug a flaky test situation so I wanted to run a test a hundred times to be sure. But compiling a hundred times is totally unnecessary and a waste of time.

@robertmagier
Copy link
Contributor

I have exactly same problem. Would be nice if one can avoid compilation every time test is executed.

@gnidan gnidan added the needs requirements feature request is unclear label Jan 9, 2019
@gnidan
Copy link
Contributor

gnidan commented Jan 9, 2019

We will look into this. Thanks!

@AC0DEM0NK3Y
Copy link

AC0DEM0NK3Y commented Jan 15, 2019

Adding to this, with v4 I was able to work around this with these steps:

truffle compile --network test --all
truffle test --network test

With v5 if you compile first as above, the test runs/hangs indefinitely with no output. I have to wipe the build folder to get back into a working state and then have truffle test do the compilation every time.

@CruzMolina
Copy link
Contributor

Hmm, so I don't believe this is a problem any more on the latest truffle (v5.0.10).

If anyone has a repo they could share to reproduce, I'll get right to debugging. 😎

@rogerioyuuki
Copy link

rogerioyuuki commented Apr 18, 2019

I also have this problem on truffle v5.0.13. I used a modified version of the updated function to debug this issue.

On line 42, it finds all contracts and creates an empty array using the path as a key.

On line 83, it should look at the artifacts json files and pushes the json in the empty array created before, using data.sourcePath as a key. However, in my case, line 42 was getting relative paths and data.sourcePath contained absolute paths, so instead of using the previosly created arrays, it was creating another key in the object.

So, for the rest of the function, truffle was believing that some contracts didn't have artifacts json files, and the result of that is the recompilation of these contracts.

I don't know why they were different since it seems to work for some people and they were only different for some contracts. I'm on Windows, but the same thing happened inside a Docker container (node:8 image). Maybe some path normalization can fix this issue?

@CruzMolina
Copy link
Contributor

CruzMolina commented Apr 19, 2019

Thanks for that info @rogerioyuuki , looking into this. Hopefully there's a quick fix.

@CruzMolina CruzMolina self-assigned this Apr 19, 2019
@CruzMolina
Copy link
Contributor

CruzMolina commented Apr 19, 2019

@rogerioyuuki , I'm on macOS and when using v5.0.13, if I truffle compile and then truffle test, the only contracts compiled/recompiled are any solidity unit tests (i.e. TestContract.sol).

Can you verify what the behavior is on your end when doing the above?

@rogerioyuuki
Copy link

rogerioyuuki commented Apr 22, 2019

@CruzMolina I think my problem might be Windows related.

Instead of running a single script similar of that updated function, I added a few console.log directly in my installed truffle cli.bundled.js to get more accurate information.

What I found is similar to what I described, with one further addition. I also found another type of path mismatch, but instead of absolute-relative mismatch, there is a mix of paths with inverted slash and forward slash and it causes the same problem. It happens even on files that I didn't create (from openzeppelin dependency, installed via npm). I don't understand why the slashes are getting mixed up. My guess is that contracts that are found via import works differently than not imported contracts.

> Compiling openzeppelin-solidity/contracts/math/SafeMath.sol
> Compiling openzeppelin-solidity/contracts/ownership/Ownable.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/ERC20.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/ERC20Burnable.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/ERC20Detailed.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/ERC20Mintable.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/ERC20Pausable.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/IERC20.sol
> Compiling openzeppelin-solidity/contracts/token/ERC20/SafeERC20.sol
> Compiling openzeppelin-solidity\contracts\access\Roles.sol
> Compiling openzeppelin-solidity\contracts\access\roles\MinterRole.sol
> Compiling openzeppelin-solidity\contracts\access\roles\PauserRole.sol
> Compiling openzeppelin-solidity\contracts\lifecycle\Pausable.sol
> Compiling openzeppelin-solidity\contracts\math\SafeMath.sol
> Compiling openzeppelin-solidity\contracts\token\ERC20\ERC20.sol
> Compiling openzeppelin-solidity\contracts\token\ERC20\IERC20.sol

Reproduction
I was able to create a brand new truffle project and break this behaviour.

On a new project ( created with truffle init), I added these contracts:
contracts/HelloWorld.sol (copied from https://www.ethereum.org/greeter)

pragma solidity >=0.4.22 <0.6.0;

contract Mortal {
    /* Define variable owner of the type address */
    address owner;

    /* This constructor is executed at initialization and sets the owner of the contract */
    constructor() public { owner = msg.sender; }

    /* Function to recover the funds on the contract */
    function kill() public { if (msg.sender == owner) selfdestruct(msg.sender); }
}

contract Greeter is Mortal {
    /* Define variable greeting of the type string */
    string greeting;

    /* This runs when the contract is executed */
    constructor(string memory _greeting) public {
        greeting = _greeting;
    }

    /* Main function */
    function greet() public view returns (string memory) {
        return greeting;
    }
}

contracts/Test.sol

pragma solidity >=0.4.22 <0.6.0;

import "./HelloWorld.sol";

contract Test is Mortal {
    function greet() public view returns (uint) {
        return 1;
    }
}

When you first run truffle compile, I can see it compiled HelloWorld.sol twice:

> Compiling .\contracts\HelloWorld.sol
> Compiling .\contracts\Migrations.sol
> Compiling .\contracts\Test.sol
> Compiling .\contracts\HelloWorld.sol

And when running truffle test it recompiles HelloWorld.sol even without modifications. I logged the paths on that updated function on lines 43 and 90 and the return of that function. Here is what I got:

43: C:/Projetos/truffle-debug2/contracts/HelloWorld.sol
43: C:/Projetos/truffle-debug2/contracts/Migrations.sol
43: C:/Projetos/truffle-debug2/contracts/Test.sol
90: C:\Projetos\truffle-debug2\contracts\HelloWorld.sol
90: C:/Projetos/truffle-debug2/contracts/Migrations.sol
90: C:\Projetos\truffle-debug2\contracts\HelloWorld.sol
90: C:/Projetos/truffle-debug2/contracts/Test.sol
[ 'C:/Projetos/truffle-debug2/contracts/HelloWorld.sol' ]

And looking at the artifacts json files, both contracts Greeter and Mortal have the "sourcePath" with \, and I think it's related with that twice compilation of HelloWorld.sol.

I guessed that it had to do with imports because you can change this project to have the absolute and relative path mismatch I mentioned earlier. Change the import to import "contracts/HelloWorld.sol"; and the result now is:
truffle compile

> Compiling .\contracts\HelloWorld.sol
> Compiling .\contracts\Migrations.sol
> Compiling .\contracts\Test.sol
> Compiling contracts/HelloWorld.sol

updated function:

43: C:/Projetos/truffle-debug2/contracts/HelloWorld.sol
43: C:/Projetos/truffle-debug2/contracts/Migrations.sol
43: C:/Projetos/truffle-debug2/contracts/Test.sol
90: contracts/HelloWorld.sol
90: C:/Projetos/truffle-debug2/contracts/Migrations.sol
90: contracts/HelloWorld.sol
90: C:/Projetos/truffle-debug2/contracts/Test.sol
[ 'C:/Projetos/truffle-debug2/contracts/HelloWorld.sol' ]

Obs: I'm on v5.0.13

$ truffle version
Truffle v5.0.13 (core: 5.0.13)
Solidity v0.5.0 (solc-js)
Node v8.12.0
Web3.js v1.0.0-beta.37

PS: To add more context why this matters, it's not only an optimization issue. I was trying to generate coverage reports to my tests, but some tools rely on intermediate steps between the compilation and the tests that gets overridden if truffle recompiles during truffle test. My current workaround is to generate coverage reports inside a special Docker container with a patched truffle that never recompiles the test. I think this may be a valid use case for a flag that forces no recompilation.

@CruzMolina
Copy link
Contributor

Hey @rogerioyuuki , thanks again for looking into this.

Can you go into this section of the code locally https://github.com/trufflesuite/truffle/blob/develop/packages/truffle-compile/index.js#L58-L79

and replace it with:

  Object.keys(sources).forEach(function(source) {
    var replacement = path.resolve(source);

    // Save the result
    operatingSystemIndependentSources[replacement] = sources[source];

    // Just substitute replacement for original in target case. It's
    // a disposable subset of `sources`
    if (hasTargets && options.compilationTargets.includes(source)) {
      operatingSystemIndependentTargets[replacement] = sources[source];
    }

    // Map the replacement back to the original source path.
    originalPathMappings[replacement] = source;
  });

Hoping this might help. Unfortunately I don't have access to Windows to debug this locally.

@CruzMolina
Copy link
Contributor

@AC0DEM0NK3Y
Copy link

Still an issue with this setup (using windows):

Truffle v5.0.20 (core: 5.0.20)
Solidity - 0.5.9 (solc-js)
Node v10.15.0
Web3.js v1.0.0-beta.37

Seems to be possibly unrelated to tests as I get the same recompile if I just run "truffle compile" one after the other after wiping the build folder.

@flockonus
Copy link

Still an significant nice to have in big projects where there is a lot of source code to compile.

If I run truffle build just before testing, still some files are compiled and it takes roughly the same time as compiling all the files.

The motivation is that often I have my Solidity source but don't actually alter it that at all, just the javascript test files, many times, as I re-run the tests.

@pizzarob
Copy link

Just an update - still happens on 5.0.30

@wjmelements
Copy link

Because we are working around another issue we are redoing truffle test once per test file. Not having this feature means that we recompile the contracts for each test file, which almost doubles test time.

@eggplantzzz
Copy link
Contributor

Truffle no longer compiles your contracts if they are up-to-date with your built artifacts during testing. It is mandatory, however, that Truffle compile any Solidity tests that you have as it does not store those as artifact files. If you want to ensure that none of your non-test artifacts are compiled you can also use the --compile-none flag to skip the check to see if the built files are up-to-date.

@cbruguera
Copy link

This is absolutely still happening in Truffle 5.1.40... @eggplantzzz Which version is this supposed to be fixed?

@ghost
Copy link

ghost commented Nov 11, 2020

I have All Issues described by @rogerioyuuki in Truffle v5.1.49

  • Mixed slashes
  • Most of the contracts are double compiled
  • truffle compile re-compiles unchanged contracts
Compiling @openzeppelin/contracts-ethereum-package/contracts/GSN/Context.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/Initializable.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/access/AccessControl.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/access/Ownable.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/math/SafeMath.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/token/ERC20/ERC20.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/token/ERC20/ERC20Burnable.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/token/ERC20/SafeERC20.sol
Compiling @openzeppelin/contracts-ethereum-package/contracts/utils/ReentrancyGuard.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\GSN\Context.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\Initializable.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\math\SafeMath.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\token\ERC20\ERC20.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\token\ERC20\IERC20.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\utils\Address.sol
Compiling @openzeppelin\contracts-ethereum-package\contracts\utils\EnumerableSet.sol
Compiling .\contracts\Migrations.sol
Compiling .\contracts\Modules\AddressProvider.sol
Compiling .\contracts\Modules\SchoolModule.sol
Compiling .\contracts\Modules\OracleModule.sol
Compiling .\contracts\Modules\ClassModule.sol
Compiling .\contracts\City\City.sol
Compiling .\contracts\inc\Base.sol
Compiling .\contracts\inc\CoreLibrary.sol
Compiling .\contracts\Modules\AddressProvider.sol
Compiling .\contracts\Modules\SchoolModule.sol
Compiling .\contracts\Modules\OracleModule.sol
Compiling .\contracts\Modules\ClassModule.sol
Compiling .\contracts\City\City.sol
Compiling .\contracts\inc\Base.sol
Compiling .\contracts\inc\CoreLibrary.sol

@ghost
Copy link

ghost commented Nov 11, 2020

@eggplantzzz please reopen this issue

@cbruguera
Copy link

Screenshot from 2020-12-05 19-45-26

Am I missing something? (Truffle v5.1.40)

Note: this is probably way more important than currently regarded. Development times are severely hindered during test implementation due to unnecessary frequent compiling.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests