WIP: Migrations

[tcoulter]

Migrations: TIP 1

This pull request adds migration support as per TIP #1.

Installation

$ npm uninstall -g truffle
$ git clone https://github.com/ConsenSys/truffle.git
$ cd trufffle
$ git checkout develop
$ npm install -g .

New Features

Including the migrations feature, this branch also includes:

• High modularity between all pieces of code.
• Update to ether-pudding 3.0. Note: In ether-pudding 3.0, Contract.deployed_address no longer exists. Use Contract.address, or Contract.deployed().address.
• Removal of environments directory, in favor of "network" configuration (see below). Built contracts now exist in a build/contracts directory, one per contract file (as opposed to one per contract file per environment) and the build artifacts from each network are stored inside that built file.
• deploy configuration completely removed, as well as after_deploy. Both featuresets are superseded by this PR.
• Files meant to be run with truffle exec must be written in a different structure, exporting a function that accepts a callback.
• Significant updates to the truffle console. You can now type commands inside it as if you were running truffle on the command line. For instance, migrate --reset within the console will rerun your migrations and immediately make your contract abstractions available for use within the same console.

What are Migrations?

The migrations feature is a "script management" tool that allows you to perform scripted contract deployment. It's built under the assumption that your application's contract structure will evolve over time, and that you'll need a solution to manage this evolution.

Deployment scripts -- otherwise called "migrations" -- exist within the migrations directory. They are Javascript files that are given a specific naming scheme, prefixed by a number, that allows for their execution history to be recorded. Over time, your projects evolution will require you to write new migration scripts, and recording their execution allows you to run only the scripts necessary for any given network.

Migration history is recorded on-chain through a special Migrations contract. This contract is required by Truffle, and must exist within your contracts directory in order to use the migrations feature. You can edit the Migrations contract at will so long as it maintains the same interface for recording and reading the migration state. View the default migrations contract.

Example/Howto

Let's create the first migration together. This portion assumes you're migrating an older Truffle project to this version. You can also run truffle init to get a new project with migrations included.

Create a migrations directory:

$ cd path/to/my/project
$ mkdir migrations

Then create the initial migration. For most projects, this initial migration will be the same. Note that this migration deploys the migrations contract:

File: migrations/1_initial_migration.js

module.exports = function(deployer) {
  deployer.deploy(Migrations);
};

Now, add the migrations contract to your project and place it within the contracts directory.

After adding it to your project, run truffle migrate. You should see the following output:

$ truffle migrate
Running migration: 1_initial_migration.js
  Deploying Migrations...
  Migrations: 0x1e0158ede7c96f57d5d8cd9186c378a2b1aad078
Saving successful migration to network...
Saving artifacts...

You can create further migrations by adding new Javascript files to your migrations directory with prefixes in ascending order, having them export a function similar to the one above. Since we've already executed the first migration on chain, truffle migrate will only execute any migrations with a higher prefix (i.e., higher than 1) the next time it is run. If no new migration files exist, it won't do anything at all.

The Deployer

You'll notice we deployed the Migrations contract using the deployer object. This is a special object provided by Truffle that will help you perform the mundane aspects of deployment, such as saving deployed contract addresses for future use. Note that like Truffle tests, your contract abstractions (e.g., the Migrations object, in the example above) are provided to your migration files for you.

Your migration file will use the deployer to queue deployment tasks. As such, you can write deployment tasks synchronously and they'll be executed in the correct order:

// Deploy A before B
deployer.deploy(A); 
deployer.deploy(B);

Alternatively, each function on the deployer can be used as a Promise, to queue up deployment tasks that depend on the execution of the previous task:

// Deploy A, then deploy B, passing in A's newly deployed address
deployer.deploy(A).then(function() {
  return deployer.deploy(B, A.address);
});

It is possible to write your deployment as a single promise chain if you find that to be more clear.

Deployer API

The deployer has many functions available to simplify your deployment.

deploy(contract, [arg1, [arg2, ...]])

Deploy a specific contract, specified by the contract object, with optional constructor arguments. This is useful for singleton contracts, such that only one instance of this contract exists for your dapp. This will set the address of the contract after deployment (i.e., Contract.address will equal the newly deployed address), and it will override any previous address stored.

You can optionally pass an array of contracts, or an array of arrays, to speed up deployment of multiple contracts.

Note that deploy will automatically link any required libraries to the contracts that are being deployed, if addresses for those libraries are available. You must deploy your libraries first before deploying a contract that depends on one of those libraries.

Examples:

// Deploy a single contract without arguments
deployer.deploy(A);

// Deploy a single contract with arguments
deployer.deploy(A, arg1, arg2, ...);

// Deploy multiple contracts, some with arguments and some without.
// This is quicker than writing three `deployer.deploy()` statements as the deployer
// can perform the deployment as a batched request.
deployer.deploy([
  [A, arg1, arg2, ...],
  B, 
  [C, arg1]
]);

deployer.link(library, destinations)

Link an already-deployed library to a contract or multiple contracts. destinations can be a single contract or an array of multiple contracts. If any contract within the destination doesn't rely on the library being linked, the deployer will ignore that contract.

This is useful for contracts that you don't intend to deploy (i.e. are not singletons) yet will need linkage before being used in your dapp.

Example:

// Deploy library LibA, then link LibA to contract B
deployer.deploy(LibA);
deployer.link(LibA, B);

// Link to many contracts
deployer.link(LibA, [B, C, D]);

autolink(contract)

Link all libraries that contract depends on to that contract. This requires that all libraries contract depends on have already been deployed or were queued for deployment in a previous step.

Example:

// Assume A depends on a LibB and LibC
deployer.deploy([LibB, LibC]);
deployer.autolink(A);

then(function() {...})

Just like a promise, run an arbitrary deployment step.

Example:

deployer.then(function() {
  // Create a new version of A
  return A.new();
}).then(function(instance) {
  // Set the new instance of A's address on B. 
  var b = B.deployed();
  return b.setA(instance.address);
});

new(contract, [arg1, arg2, ...])

Exactly like deploy() except that this function does not save the contract address. This is purely syntactic sugar, but will be recorded as a deployment step in the logs. Might be removed.

An alternative is to use Contract.new() within a then block. See example above.

exec(pathToFile)

Execute a file meant to be run with truffle exec as part of the deployment. This takes the role of the after_deploy configuration available in the previous version of Truffle and can be used conditionally depending on the network -- adding demo data when deploying to the staging or test environment, for instance.

Example:

// Run the script, relative to the migrations file.
// See networks discussion below for more advanced usage.
deployer.exec("../path/to/file/demo_data.js");

Networks

This version of Truffle introduced the idea of networks, to replace environments. Networks are now configured within your main truffle.js file, and this configuration is not required. Network artifacts are saved within a single .sol.js file per contract, and you can require this .sol.js file in your frontend deploying your frontend only once. Using the features of ether-pudding 3.0, this allows you to only deploy your frontend once, and your contract abstractions will detect the currently running network and use the correct artifacts accordingly. TL;DR: These new changes allow a single frontend deployment to facilitate multiple networks, including the live network, the morden network, etc.

Configuration

truffle.js

{
  networks: {
    "live": {
      network_id: 1,
      // optional config values
      // host - defaults to "localhost" 
      // port - defaults to 8545
      // gas 
      // gasPrice
      // from - default address to use for any transaction Truffle makes during migrations
    }
  }
}

The above configuration tells Truffle that you have a network called "live" that will be used when the network id is 1. You can also specify optional parameters, detailed above. When the contract abstractions detect a connected client with the network id of 1, the saved contract artifacts for that network, like the deployed address, will be used.

By default, Truffle uses a network with the title "default" if none are specified. You can successfully deploy a Truffle application using the "default" network and not configuring any other networks, but in practice this will rarely be the case (think, "default" is used for development while "live" is used when the contract is ready for deployment).

Usage

To deploy to a network, specify it by name:

$ truffle migrate --network live

This will run your migrations against the network specified. Note that if you've never ran truffle migrate on this network before, it will run your migrations from the beginning.

Conditionally migrating depending on network

You can run separate deployment steps conditionally based on the network chosen. To do so, add a second parameter to your migration:

3_some_migration.js

module.exports = function(deployer, network) {
  deployer.deploy(MyContract);  

  // Add demo data
  // Don't run this on the live network
  if (network != "live") {
    deployer.exec("./path/to/demo_data.js");
  }
};

Conclusion

The migrations feature is a work in progress. For instance, it was heavily inspired by Active Record Migrations from Rails, but you won't find any mention of "up" and "down" directions within our implementation. This is because Ethereum dapps exist within a different environment, and require different deployment needs. These needs we'll continue to explore, and like your dapp's deployment structure, will consistently evolve. Your feedback is most welcome.

[karlfloersch]

The deployer API makes a lot of sense. Feature-wise it seems similar to DappleScript but written in pure Javascript (which seems like a win). Would you say that's accurate?

The networks feature looks great. Totally support embedding network data inside of .sol.js.

I don't know if I fully understand how the migrations feature works. It might be useful to add a diagram which can describe how the migrations stuff works, like what information is stored in Javascirpt, and what's on chain.

[PeterBorah]

I don't think this works the way you want it to. The default authentication scheme only allows setCompleted to be called by the owner, but the old Migrations is not going to be the owner. Granted your new code might be different, but seems weird to have to write a new Migrations contract with the assumption that the old one can call it.

I don't think this is necessary anyway, since it's a single uint that will need to be ported to the new Migrations contract, and the new contract can get it itself by calling last_completed_migration.

More important might be to let the owner set a new owner, so that people aren't stuck with the first key they use to deploy it.

[tcoulter]

Good points. The idea is that you can deploy a different Migrations contract as long as it contains the same signature as the old one for two specific functions, setCompleted() and last_completed_migration(), the latter provided by solidity. If this is true, the implementation of the new one does not matter: during an upgrade, we'll just treat it like its the old one and call its setCompleted() function, which must exist. The data field that gets generated for that transaction should be the same.

You have very good points though. This is actually something I added initially, but have not tested. Perhaps I can write a test for it.

[tcoulter]

@karlfloersch

The deployer API makes a lot of sense. Feature-wise it seems similar to DappleScript but written in pure Javascript (which seems like a win). Would you say that's accurate?

I'm not intimately familiar with DappleScript's deployment features, so I don't have enough information to answer this. However, I would say that both Dapple and Truffle are aiming to solve the same issues, albeit in different ways. Having your deployment language the same language as your tests and the same language as your frontend, in my opinion, is a big win. This means you can use the same code constructs throughout.

I don't know if I fully understand how the migrations feature works. It might be useful to add a diagram which can describe how the migrations stuff works,

I'll see if I can cook something up.

like what information is stored in Javascirpt, and what's on chain.

Only contracts that you deploy are on chain, one of those contracts being the default Migrations contract. From there a single uint tracks which migrations have been run and which haven't, based on the prefix of migration file.

[tcoulter]

Note: Docs for the develop branch (which includes the migrations branch) are available here: http://truffle.readthedocs.io/en/develop/. Edit: They're actively being worked on.

[tcoulter]

Closing this ticket. I've hand merged it into the develop branch.