Support to Google Cloud Platform

[pethron]

Update 29/05/23

This feature has been delayed for quite some time. Right now we can't work on this because we are focusing on bootstrapping (we need to make the project sustainable for everyone). We would love to see people using GCP to start using Leapp.

We had already implemented this feature in the previous Golang project, but we don't have time to port it. If anyone is willingly to work on this, we can open the previous repository and let you work with support from us.

We would greatly appreciate community support on this, so if anyone is interested, please reach out on our Slack channel or comment on this issue.

• To integrate with Client

• https://github.com/Noovolari/leapp-daemon/issues/8

• https://github.com/Noovolari/leapp-daemon/issues/13

[Sytten]

Any progress?

[pethron]

A quick update on this. This was already implemented in the migration to the daemon-service. Since we've decided to drop this migration we need to be port all the logic in this project. Our goal is to support GCP by the end of the first quarter of 2022.

[Sytten]

Just checking in since we are almost there :P

[pethron]

Running late on this @Sytten, but we're getting there! I think we will probably manage by the first half of Q2

[varunrayen]

Any updates?

[tonidy]

@pethron any update on this? I want to use GCP 😄

[pethron]

And we very much would like to let you @tonidy ! Our goal is to provide essential support before the end of this year, but we can't commit to a date. We're in the middle of bootstrapping, and our top priority is ensuring the project's continuity (also to support GCP properly). We have a repo and project in Golang that is implemented, but we need to port it to this one.

I'll flag this issue as "help wanted"; if anyone is willing, we can open the repo and let you work on it. We would greatly appreciate community support on this, so if anyone is interested, please reach out on our Slack channel.

[tonidy]

@pethron so, you actually have GCP implementation but in golang and you needs to port it to Leapp? I'm interesting to port it. Would be great if you can open the repo

FYI, I've joined to the slack channel

[ericvilla]

Hi @tonidy @pethron!

Since the last GCP implementation in Go, the usage flow has changed. We have reviewed the Go implementation internally and, as a result, porting from a Go version to a Typescript one will be a waste of effort, because a lot of things have changed in the meantime. Instead, we can support you in developing a new solution to support GCP.

We've studied the GCP usage flow from a gcloud CLI point of view. Let us provide you with some insights that could be useful for integrating it into Leapp.

---

We firstly studied the gcloud main usage patterns (if you know other critical patterns, please add them to the list).

gcloud config configurations list
    NAME     IS_ACTIVE  ACCOUNT  PROJECT  COMPUTE_DEFAULT_ZONE  COMPUTE_DEFAULT_REGION
    default  True

gcloud auth login
    updates the access_tokens.db (access token and id token) and credentials.db (refresh token, client id, and client secret)

gcloud config configurations list
    NAME     IS_ACTIVE  ACCOUNT                     PROJECT  COMPUTE_DEFAULT_ZONE  COMPUTE_DEFAULT_REGION
    default   True             [email protected]

gcloud projects list
    PROJECT_ID                NAME             PROJECT_NUMBER
    project-1-id                 project-1        012345678910
    project-2-id                 project-2       012345678910
    project-3-id                 project-3       012345678910

gcloud config set project project-1-id

gcloud config configurations list
    NAME     IS_ACTIVE  ACCOUNT                           PROJECT     COMPUTE_DEFAULT_ZONE  COMPUTE_DEFAULT_REGION
    default    True             [email protected]        project-1

In this example, we used the default named configuration but we already thought about supporting multiple gcloud named configurations.

We found a correlation between the gcloud usage patterns and a Leapp Integration. In particular, there are three main phases:

• create
• log in
• sync
• start/stop/rotate an integration specific session
• logout

Create

Firstly, we have to create a GCP integration inside Leapp, so that we can log into it. The only information we have to provide is a name.

Log in

Once we've created the integration, we can log into it. In this case, we'll use the same Oauth 2.0 login flow as the gcloud CLI. During the login, you have to provide the credentials associated with the Account used to log into the integration. The output of this operation is a payload containing all the tokens needed in the next steps: client_id, client_secret, refresh_token, access_token, and id_token. The idea is to persist this sensitive information inside the system keychain (NOT in the gcloud default configuration files). REMEMBER that the tokens are associated with the Account used to log into the integration.

Sync

Using the tokens saved in the system keychain, we can get the list of projects that are accessible by the Account used to log into the integration. This projects list will be persisted in Leapp's workspace file (the one that contains all Leapp configurations).

Start/stop/rotate

When it comes to start a Leapp GCP session, we have to use the refresh_token (saved in the keychain) to generate a new access token; the access token (together with the id token) will be saved in the access_tokens.db file (sqlite db), while the other tokens will be saved in the credentials.db file. In addition, it is necessary to overwrite (if present) the ~/.config/gcloud/config_default named configuration file with the account and project fields.

To stop the Leapp GCP session, we could simply remove the tokens from the gcloud configuration files.

To rotate the credentials associated with the session, we have to generate a new access token via the refresh token.

Logout

Stop all the GCP sessions associated with the GCP integration. We still have to understand how to revoke the previously generated tokens (including the refresh_token).

---

Let us share with you the first two activities proposal. This proposal is the result of an R&D activity; please, share with us your thoughts and let us know if there is something that is not clear to you or if you want to provide some additions!

---

Proposal

LEAP: create GCP integration

WHY

As a Leapp User, I want to create a GCP integration from the sidebar, as for AWS SSO and Azure integrations.

AT

• Run Leapp Desktop App
• Add a new integration from the sidebar (+ button)
• Select GCP integration type
• Fill in the required fields (name)

LEAP: GCP integration login

WHY

As a Leapp User, I want to log into a GCP integration.

AT

• Run Leapp Desktop App
• Double-click the GCP integration from the left-side panel
• Click the link generated by the Google CloudSDK; from the Browser, provide your GCP account credentials.
• Submit the login form
• In the system keychain, check if a “gcp-tokens“ key is present.

Dev Notes

• Setup an Express server to catch the oauth 2 login flow callback
• Once catched, save the login response in the keychain as “gcp-tokens“.

[tonidy]

@ericvilla thanks for your explanation. Let me learn it first.

Regarding this,

In particular, there are three main phases:

create
log in
sync
start/stop/rotate an integration specific session
logout

Should be 5 phases instead 3? Or I missed something?

[tonidy]

@ericvilla any update on this? I haven't touched it yet. But I want to do some PoC first before implement the feature.