Program-Runtime v2 - Road Map

[Lichtso]

The performance optimizations we have been working on inside the runtime under the ABIv2 project the past year will be deployed separately as "account data direct mapping". The redesign of the ABI between runtime and on-chain program is pushed out further as we changed the goals again.

Presentations

• Slides from 2023/01/20
• Slides from 2023/08/15
• Breakpoint 2023/11/01

Tasks

• LoadedPrograms #29803
  • Refactor - Delay visibility of program un-/re-/deployment #29654
  • API to match account owner against a set of owners #30154
  • LoadedPrograms cache implementation and tests #30139
  • Code to load a program from its account #30282
  • Return loaded program entry from the cache after insert #30336
  • Return index of matching owner for a program account #30337
  • Remove executors trait and replace with LoadedProgram #30348
  • API to enumerate executable program accounts in tx batch #30371
  • Find and load missing programs in LoadedPrograms cache #30275
  • Replenish executor cache eagerly during transaction processing #30425
  • Adds test for programs which read from a program account #30803
  • Increment usage count for loaded programs and call eviction #30900
  • Fix deployment slot for upgraded programs #30902
  • Remove redundant check during loading upgradeable program for writing #30561
  • Stop loading program accounts if program exists in cache #30703
  • Increment usage count for loaded programs and call eviction #30900
  • Fix deployment slot for upgraded programs #30902
  • Return error early if program is a tombstone #30940
  • SBF test that runs with stable bank and genesis to produce deterministic hash #30945
  • Use feature gate to set effective slot of loaded programs #30950
  • Cleanup - load_and_get_programs_from_cache() #30959
  • Add expiration slot to loaded program cache entry #31034
  • Add provisions to unload a program from the cache #31036
  • Fix unused import error in LoadedPrograms #31116
  • Fix finding ancestor during program cache pruning #31118
  • Retain usage counter on program upgrade #31142
  • Do not evict tombstones #31311
  • Check program modification slots during cold start #31331
  • Remove obsolete code for lazily loading programs #31395
  • Add a second level cache to contain tx batch specific programs #31413
  • Do not evict unloaded programs #31465
  • Return implicit delay visibility tombstone #31493
  • Allow replacing tombstone with a non-tombstone #31494
  • Update SBF bench to account for delay visibility #31501
  • Programs deployed before the latest root available on all current forks #31530
  • Refactor implicit handling of delay visibility tombstones #31537
  • Generalize matching criteria for tombstone at boot up #31547
  • Use current root slot to disambiguate pruning of old programs #31548
  • Remove executor cache from the codebase #31566
  • Add stats for LoadedPrograms #31577
  • LoadedPrograms::extract() code cleanup and unit tests #31604
  • Refactor - Merges BuiltinPrograms into LoadedPrograms #31654
  • Remove Rc and RefCell to carry program cache in invoke_context #31684
  • Do not overwrite builtin program in the same slot #31718
  • Refactor - Share RBPF config and syscall registry globally #31727
  • More metrics for LoadedPrograms cache #31735
  • Set deployment slot for non-upgradeable programs #31792
  • Reverify programs that are extended using ExtendProgram #31886
  • Fix - LoadedProgramType::Closed #31922
  • Evict program cache using transaction usage counter #31927
  • Reset program cache stats after submit #31929
  • Feature - Prune on feature set transition #31945
  • Retain runtime environment config for unloaded programs #31953
  • Add runtime environment to FailedVerification tombstones #31991
  • Cleanup load_program() in bank.rs #32146
  • Do not unnecessarily re-verify unloaded program #32722
  • Prune programs deployed in duplicate unconfirmed slot #32999
  • Handle closing of the program in the same slot as deployment #33025
  • Feature - Epoch boundary recompilation phase #33477
  • Refactor - LoadedPrograms #33482
  • Refactor - LoadedPrograms part 2 #33694
  • Feature - Cooperative Program Loading #33715
  • Set a global fork graph in program cache #33776
  • Custom debug impl for LoadedPrograms cache #33808
  • Initialize fork graph in program cache during bank_forks creation #33810
  • Do not prune cache entry if the runtime environment is different #34100
• Solana BPF v2 #20323
  • Verifier - Stricter control flow rbpf#454
  • Feature - Adds MemoryMapping::Identity rbpf#460
  • Feature - Restrict ro data layout in SBPFv2 rbpf#481
  • Feature - lduw (load upper word immediate) rbpf#486
  • Refactor - Replaces sub r11, imm with add r11, -imm rbpf#488
  • Feature - Swaps parameters of sub reg, imm and removes neg rbpf#489
  • Fix - callx encoding rbpf#491
  • Refactor - Disables little-endian byte swap instructions in SBPFv2 rbpf#493
  • Refactor - lduw to hor64 rbpf#496
  • Feature - Product/Quotient/Remainder (PQR) Instruction Class rbpf#498
  • Sign Extension in SBPFv2 #32924
  • Dynamic Dispatch in SBPFv2 #33971
• Solana BPF v3 #29863
• Loader v4 Built-In Program #29864
  • Refactor - Simplify program accounts in transaction loading #29728
  • Refactor - Remove process_instruction parameter first_instruction_account #30579
  • Refactor - Remove first_instruction_account in bpf_loader #30614
  • Feature - Loader built-in program v4 #30464
  • Refactor - Cleanup error handling in program runtime #30693
  • Refactor - mock_process_instruction() #30893
  • Cleanup - mock InvokeContext #31007
  • Cleanup - mock_process_instruction() #31088
  • Refactor - Moves the system program into its own crate #31244
  • Refactor - Use builtin function interface from rbpf #31221
  • Refactor - Moves the system program into its own crate #31244
  • Refactor - BuiltinPrograms #31324
  • Cleanup - Removes the CLI option "no-bpf-jit" #31329
  • Refactor - Moves bpf_loader registration into the runtime #31345
  • Refactoring - Remove redundant definition of BuiltinProgram #31429
  • Feature Gate Update - Requires the owner accounts of programs to be built-ins. #31488
  • Refactor - Renames loader-v3 to loader-v4. #31570
  • Bump solana_rbpf to v0.5.0 #31943
  • Bump solana_rbpf to v0.4.0 #31594
  • Initialize and store PRv2 environment in cache #32812
  • Integrate program loader-v4 with bank #32832
  • Add loader-v4 instruction constructors #33151
  • Check loader-v4::id() as a valid program owner #33167
  • Processors to compile and send LoaderV4 CLI commands #33228
  • Set loader-v4 program deployment slot at actual deployment #33278
  • Fix lamport calculation and transfer for loader-v4 program upgrade #33279
  • Implement CLI front-end for loader-v4 commands #33289
  • Feature - Enable Program-Runtime-v2 and Loader-v4 #33294
  • Decouple program-v4 command processors from CliConfig #33554
  • Cargo registry service for deploying programs #33570
  • Fix deployment of program-v4 in freshly started test validator #33583
  • Fix failure in updating GIT index of cargo registry server #33628
  • Enable running remote cargo registry server #33629
  • Replace cargo registry server's GIT index with sparse index #33666
  • Handle cargo registry index lookup requests #33681
  • Add CLI command to show/dump v4 programs #33693
  • Code cleanup in cargo-registry #33711
  • Add support for fetching program in cargo registry #33759
  • Cargo registry server updates to download crates #33805
  • Append crate to ELF file while deploying program #33849
  • Retrieve crate version from cargo-registry package #33867
  • Cleanup cargo registry code #33879
  • Fix transitive cargo dependencies
• Program Runtime v2 - ABI #32154
• [Not MVP] Adjust other built-in programs and the testing framework

[austbot]

This is amazing!

A few questions:
1 is this the issue to discuss?
2. "Replace VM nesting by dynamic linking using two levels of indirection" does this make CPI less expensive since the entire account context doesn't need to be cloned for the CPI execution?
3. With multiple entry points can those entrypoints return data to the client on an outer instruction?

[leoluk]

Share the host address space for all programs in a transaction (similar to Native Client)

How will isolation work? Will the verifier enforce it?

[alessandrod]

This is amazing!

A few questions: 1 is this the issue to discuss?

This is a meta-tracking issue we created after talks at breakpoint. Input is of course welcome, here and on discord in the #virtual-machine channel. Going forward we should probably create separate issues for sub features tho or this is likely going to get out of hand.

2. "Replace VM nesting by dynamic linking using two levels of indirection" does this make CPI less expensive since the entire account context doesn't need to be cloned for the CPI execution?

The account clone issue is already going away Real Soon Now (tm) with the existing runtime: #28755. The other large issue of creating nested VMs (what the current runtime does during CPI) will go away with v2. Both will reduce the cost of doing CPI yes.

3. With multiple entry points can those entrypoints return data to the client on an outer instruction?

If you mean if callees will be able to return data to their callers the answer is yes - one of the goals of this work is to improve the sdk ergonomics, so that you'll be able to "just call some functions" without the runtime getting in the way.

[Lichtso]

3. With multiple entry points can those entrypoints return data to the client on an outer instruction?

I understand the question being about the client, the one off chain which triggers the transaction via RPC. If that is the question then, no or at least that is outside of the scope of the program runtime and has to do with networking. It is tricky because the process is asynchronous in the sense that submitting a transaction and polling for its completion (including its possible results) are independent flows. In other words, in the program runtime we could return the return value of the last instruction in a transaction (or something along these lines), but it would not be propagated anywhere. That requires additional infrastructure outside of the program-runtime.

[Lichtso]

How will isolation work? Will the verifier enforce it?

@leoluk Yes, that is the plan. On deployment the verifier would do type inference for the entire instruction stream and could then find any misbehaving memory accesses before execution. For dynamic offsets / indices we would check that the correct (canonical) bounds checks are in the instruction stream as well.

[mschneider]

Looks like a good change. Can you include the planned security process for this redesign in the roadmap?

[chen-robert]

On deployment the verifier would do type inference for the entire instruction stream and could then find any misbehaving memory accesses before execution. For dynamic offsets / indices we would check that the correct (canonical) bounds checks are in the instruction stream as well.

Sounds like fun