Upgrade Moddable and Native XSnap Worker

[kriskowal]

What is the Problem Being Solved?

Our chain is currently mired on an old version of Moddable’s XS JavaScript engine and our native XSnap Worker implementation. In order to realize the benefits of new XS features, performance improvements, and eventually security patches, we need to be able to upgrade it at the chain’s discretion. Challenges in upgrading this subsystem include:

1. Every vat on the chain is currently obliged to use the same version of XS
2. Every vat is obliged to replay its message transcript—since its last upgrade—without any observable difference. XS changes risk effecting observable differences.
3. At least one vat, the bootstrap vat, is not upgradable and never will be. We may be able to replace that vat by passing all of its authority to another with a “core eval” proposal and then forgetting about it, but we can’t upgrade it in place.
4. Vats may be upgraded, but currently, the worker type is fixed. An XS upgrade would hold the worker type fixed but use new software.

Priors:

• how to change XS on a deployed system #6361
• define worker-v1 package #6596
• ability to restart all production vats #8104
• xsnap upgrade by restart-time forced upgrade of all vats by kernel #8405
• suspended vats: no worker, no snapshot, no transcript, held in-between incarnations #8955
• add kernel API to restart all vats at this/next startup #8954

Description of the Design

We present these to the Agoric kernel with the @agoric/xsnap package, which subsumes the compiled worker binary and provides a JavaScript API for execution, messaging, metering, and snapshots. In order to address the above challenges, we propose to add an option to the XSnap API that allows the caller to specify the desired version. The implementation will at any time support only two versions: a legacy version for the non-upgradable vats, and a “current” version. When the chain adopts a new “current” version of XS, every vat will need to upgrade. Those upgrades may fail, which must put the vat in a “suspended” state, where messages are queued until the vat receives a software upgrade and attempts to “resume”. The queue flushes after the first successful upgrade.

The “version” of a worker type will not just entrain the version of the Moddable, XS, and XSnap software but also the in-vat supervisor bundle.

• Add versioning to the worker interface
• Add binary/executable version multiplexing to @agoric/xsnap
• Create replay validation framework for versioned vats
• Support vat suspension for failed upgrades
• Distribute vat worker version upgrade across blocks, resume on successful upgrade

Security Considerations

The above system must preserve the invariants of all extant contracts. A vat, once upgraded, must not be observably different that its behavior up to the moment of its upgrade. So, this change entrains replay validation tooling.

Scaling Considerations

We expect upgrades to take time. This change may entail further work to ensure that upgrades are adopted gradually over multiple blocks. That might mean suspending all vats and resuming them serially, queuing messages until they successfully upgrade.

Test Plan

This change will require considerable replay testing in integration.

Upgrade Considerations

Numerous.