Add NoFork External Connectors & Clients #294
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Switch to Resource.RefreshWithoutUpgrade & Resource.Apply Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Cem Mergenci <[email protected]> Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
…seconds metrics Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Cem Mergenci <[email protected]> Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
…WS v5.x tags Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
…ource schema elements - Add support for explicitly configuring fields to be added to the Observation types. Signed-off-by: Alper Rifat Ulucinar <[email protected]>
… Terraform configuration parameters Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
…ization option by default - May result in unnecessary drift if some fields are not late-initialized. Signed-off-by: Alper Rifat Ulucinar <[email protected]>
…ion back - Late-initialization of nil values from corresponding zero-values results in a late-initialization loop and prevents the resource from becoming ready. Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
…ceDiffs Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
- Terraformed resources which are reconciled using the synchronous no-fork external client also use the in-memory Terraform state cache. This was needed because there are resources who put certain attributes into the TF state only during the Create call. Signed-off-by: Alper Rifat Ulucinar <[email protected]>
…al clients Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
ulucinar
force-pushed
the
no-fork
branch
2 times, most recently
from
e241dfc
to
d4fd8af
Compare
… schema and config.Resource.OperationTimeouts - The timeouts specified in upjet resource configuration (config.Resource.OperationTimeouts) prevails any defaults configured in the Terraform schema. Signed-off-by: Alper Rifat Ulucinar <[email protected]>
* Remove all length keys, which are suffixed with % or #, from initProvider-exclusive diffs. Map and set diffs are successfully applied without these length keys. List diffs require length keys. To be addressed later. Signed-off-by: Cem Mergenci <[email protected]>
Signed-off-by: Cem Mergenci <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Sergen Yalçın <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Erhan Cagirici <[email protected]>
sergenyalcin
force-pushed
the
no-fork
branch
from
9fd3224
to
bc6e655
Compare
Signed-off-by: Alper Rifat Ulucinar <[email protected]>
Signed-off-by: Sergen Yalçın <[email protected]>
Signed-off-by: Sergen Yalçın <[email protected]>
Signed-off-by: Sergen Yalçın <[email protected]>
- Add unit test for InitProvider Signed-off-by: Sergen Yalçın <[email protected]>
1 task
erhancagirici
approved these changes
Nov 16, 2023
|
Question 🙏 We've up-jetted some providers internally and would like to adopt these wonderful performace improvements. How do we migrate to this? We used If there's a doc or quip somewhere and I missed it feel free to link it, thank you! |
|
@cnuss we're going to fast-follow the 1.0.0 release with some documentation updates that will address this. |
This was referenced Nov 17, 2023
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of your changes
This PR adds two new
managed.Connecters and two newmanaged.ExternalClients for reconcilingTerraformedresources without forking any Terraform CLI or Terraform provider processes. The new external client implementation consumes the Terraform provider's Go provider schema and invokes the CRUD functions registered in that schema via the terraform-plugin-sdk utility functions.The two new external clients are the
controller.noForkExternaland thecontroller.noForkAsyncExternaland are implemented inpkg/controller/external_nofork.goandpkg/controller/external_async_nofork.go, respectively. The async client is composed using the sync client and all the interaction with the underlying Terraform provider via the Terraform plugin SDK is implemented inpkg/controller/external_nofork.go.Both clients employ an in-memory Terraform state cache as these clients no longer construct Terraform workspaces while reconciling their resources. We have an initial evaluation of the memory cost of this cache with 10k MRs and it's in the order of a few hundred megabytes (~200 MB), much smaller than the footprint of the process fork-based clients.
I have:
make reviewableto ensure this PR is ready for review.backport release-x.ylabels to auto-backport this PR if necessary.How has this code been tested
Tested in crossplane-contrib/provider-upjet-aws#938 & crossplane-contrib/provider-upjet-aws#910.