Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Cloud SQL Address Range Picker for Clones and Read replicas #5664

Merged
merged 6 commits into from
Feb 8, 2022
Merged

Support Cloud SQL Address Range Picker for Clones and Read replicas #5664

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7499e7a
added address range picker for clone
shuyama1 Jan 28, 2022
fa66f4c
rename test, clean up and remove forceNew
shuyama1 Jan 31, 2022
79f4257
added update test
shuyama1 Feb 1, 2022
df65e2d
fix typo
shuyama1 Feb 1, 2022
35c3f06
using separate private networks
shuyama1 Feb 2, 2022
2260e35
rebase and update location
shuyama1 Feb 7, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion mmv1/third_party/terraform/go.mod.erb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ require (
golang.org/x/mod v0.5.0 // indirect
golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420
golang.org/x/oauth2 v0.0.0-20211104180415-d3ed0bb246c8
google.golang.org/api v0.65.0
google.golang.org/api v0.66.0
google.golang.org/grpc v1.40.1
)

Expand Down
8 changes: 7 additions & 1 deletion mmv1/third_party/terraform/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1153,6 +1153,8 @@ golang.org/x/sys v0.0.0-20211124211545-fe61309f8881/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e h1:fLOSk5Q00efkSvAm+4xcoXD+RRmLmmulPn5I3Y9F2EM=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220114195835-da31bd327af9 h1:XfKQ4OlFl8okEOr5UvAqFRVj8pY/4yfcXrddB8qAbU0=
golang.org/x/sys v0.0.0-20220114195835-da31bd327af9/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
Expand Down Expand Up @@ -1311,6 +1313,8 @@ google.golang.org/api v0.63.0/go.mod h1:gs4ij2ffTRXwuzzgJl/56BdwJaA194ijkfn++9tD
google.golang.org/api v0.64.0/go.mod h1:931CdxA8Rm4t6zqTFGSsgwbAEZ2+GMYurbndwSimebM=
google.golang.org/api v0.65.0 h1:MTW9c+LIBAbwoS1Gb+YV7NjFBt2f7GtAS5hIzh2NjgQ=
google.golang.org/api v0.65.0/go.mod h1:ArYhxgGadlWmqO1IqVujw6Cs8IdD33bTmzKo2Sh+cbg=
google.golang.org/api v0.66.0 h1:CbGy4LEiXCVCiNEDFgGpWOVwsDT7E2Qej1ZvN1P7KPg=
google.golang.org/api v0.66.0/go.mod h1:I1dmXYpX7HGwz/ejRxwQp2qj5bFAz93HiCU1C1oYd9M=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
Expand Down Expand Up @@ -1394,6 +1398,8 @@ google.golang.org/genproto v0.0.0-20211223182754-3ac035c7e7cb/go.mod h1:5CzLGKJ6
google.golang.org/genproto v0.0.0-20220107163113-42d7afdf6368/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc=
google.golang.org/genproto v0.0.0-20220111164026-67b88f271998 h1:g/x+MYjJYDEP3OBCYYmwIbt4x6k3gryb+ohyOR7PXfI=
google.golang.org/genproto v0.0.0-20220111164026-67b88f271998/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc=
google.golang.org/genproto v0.0.0-20220114231437-d2e6a121cae0 h1:aCsSLXylHWFno0r4S3joLpiaWayvqd2Mn4iSvx4WZZc=
google.golang.org/genproto v0.0.0-20220114231437-d2e6a121cae0/go.mod h1:5CzLGKJ67TSI2B9POpiiyGha0AjJvZIUgRMt1dSmuhc=
google.golang.org/grpc v1.8.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
Expand Down Expand Up @@ -1495,4 +1501,4 @@ rsc.io/binaryregexp v0.2.0 h1:HfqmD5MEmC0zvwBuF187nq9mdnXjXsSivRiXN7SmRkE=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o=
sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o=
9 changes: 7 additions & 2 deletions mmv1/third_party/terraform/resources/resource_sql_database_instance.go.erb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,6 @@ settings.backup_configuration.binary_log_enabled are both set to true.`,
"allocated_ip_range": {
Type: schema.TypeString,
Optional: true,
ForceNew: true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the update case for this field something we can add a test for, or is it too hard to orchestrate?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can create a test on updating this field with a new private network, as I mentioned in the PR description, but just not within the same VPC. I will add one on this.

AtLeastOneOf: ipConfigurationKeys,
Description: `The name of the allocated ip range for the private ip CloudSQL instance. For example: "google-managed-services-default". If set, the instance ip will be created in the allocated range. The range name must comply with RFC 1035. Specifically, the name must be 1-63 characters long and match the regular expression [a-z]([-a-z0-9]*[a-z0-9])?.`,
},
Expand Down Expand Up @@ -694,6 +693,11 @@ settings.backup_configuration.binary_log_enabled are both set to true.`,
DiffSuppressFunc: timestampDiffSuppress(time.RFC3339Nano),
Description: `The timestamp of the point in time that should be restored.`,
},
"allocated_ip_range": {
Type: schema.TypeString,
Optional: true,
Description: `The name of the allocated ip range for the private ip CloudSQL instance. For example: "google-managed-services-default". If set, the cloned instance ip will be created in the allocated range. The range name must comply with [RFC 1035](https://tools.ietf.org/html/rfc1035). Specifically, the name must be 1-63 characters long and match the regular expression [a-z]([-a-z0-9]*[a-z0-9])?.`,
},
},
},
},
Expand Down Expand Up @@ -1005,7 +1009,8 @@ func expandCloneContext(configured []interface{}) (*sqladmin.CloneContext, strin
_cloneConfiguration := configured[0].(map[string]interface{})

return &sqladmin.CloneContext{
PointInTime: _cloneConfiguration["point_in_time"].(string),
PointInTime: _cloneConfiguration["point_in_time"].(string),
AllocatedIpRange: _cloneConfiguration["allocated_ip_range"].(string),
}, _cloneConfiguration["source_instance_name"].(string)
}

Expand Down
198 changes: 197 additions & 1 deletion mmv1/third_party/terraform/tests/resource_sql_database_instance_test.go.erb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -697,6 +697,8 @@ func TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange(t *testi
databaseName := "tf-test-" + randString(t, 10)
addressName := "tf-test-" + randString(t, 10)
networkName := BootstrapSharedTestNetwork(t, "sql-instance-private-allocated-ip-range")
addressName_update := "tf-test-" + randString(t, 10) + "update"
networkName_update := BootstrapSharedTestNetwork(t, "sql-instance-private-allocated-ip-range-update")

vcrTest(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
Expand All @@ -712,10 +714,80 @@ func TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange(t *testi
ImportStateVerify: true,
ImportStateVerifyIgnore: []string{"deletion_protection"},
},
{
Config: testAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRange(databaseName, networkName_update, addressName_update),
},
{
ResourceName: "google_sql_database_instance.instance",
ImportState: true,
ImportStateVerify: true,
ImportStateVerifyIgnore: []string{"deletion_protection"},
},
},
})
}

func TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeReplica(t *testing.T) {
t.Parallel()

databaseName := "tf-test-" + randString(t, 10)
addressName := "tf-test-" + randString(t, 10)
networkName := BootstrapSharedTestNetwork(t, "sql-instance-private-replica")

vcrTest(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
Providers: testAccProviders,
CheckDestroy: testAccSqlDatabaseInstanceDestroyProducer(t),
Steps: []resource.TestStep{
{
Config: testAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeReplica(databaseName, networkName, addressName),
},
{
ResourceName: "google_sql_database_instance.instance",
ImportState: true,
ImportStateVerify: true,
ImportStateVerifyIgnore: []string{"deletion_protection"},
},
{
ResourceName: "google_sql_database_instance.replica1",
ImportState: true,
ImportStateVerify: true,
ImportStateVerifyIgnore: ignoredReplicaConfigurationFields,
},
},
})
}

func TestAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeClone(t *testing.T) {
t.Parallel()

databaseName := "tf-test-" + randString(t, 10)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noting here that we cannot use the bootstrap db instance since this uses a private network. If more than one private network clone/backup test is needed, we should create a new bootstrap instance.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree! Looks like this is the only one now. We will probably need to create a bootstrap instance with private networks if more tests need it.

addressName := "tf-test-" + randString(t, 10)
networkName := BootstrapSharedTestNetwork(t, "sql-instance-private-clone")

vcrTest(t, resource.TestCase{
PreCheck: func() { testAccPreCheck(t) },
Providers: testAccProviders,
CheckDestroy: testAccSqlDatabaseInstanceDestroyProducer(t),
Steps: []resource.TestStep{
{
Config: testAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeClone(databaseName, networkName, addressName),
},
{
ResourceName: "google_sql_database_instance.instance",
ImportState: true,
ImportStateVerify: true,
ImportStateVerifyIgnore: []string{"deletion_protection"},
},
{
ResourceName: "google_sql_database_instance.clone1",
ImportState: true,
ImportStateVerify: true,
ImportStateVerifyIgnore: []string{"deletion_protection", "clone"},
},
},
})
}

func TestAccSqlDatabaseInstance_createFromBackup(t *testing.T) {
// Sqladmin client
Expand Down Expand Up @@ -1097,7 +1169,7 @@ resource "google_compute_global_address" "foobar" {
name = "%s"
purpose = "VPC_PEERING"
address_type = "INTERNAL"
prefix_length = 24
prefix_length = 16
network = data.google_compute_network.servicenet.self_link
}

Expand Down Expand Up @@ -1125,6 +1197,130 @@ resource "google_sql_database_instance" "instance" {
`, networkName, addressRangeName, databaseName)
}


func testAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeReplica(databaseName, networkName, addressRangeName string) string {
return fmt.Sprintf(`
data "google_compute_network" "servicenet" {
name = "%s"
}

resource "google_compute_global_address" "foobar" {
name = "%s"
purpose = "VPC_PEERING"
address_type = "INTERNAL"
prefix_length = 16
network = data.google_compute_network.servicenet.self_link
}

resource "google_service_networking_connection" "foobar" {
network = data.google_compute_network.servicenet.self_link
service = "servicenetworking.googleapis.com"
reserved_peering_ranges = [google_compute_global_address.foobar.name]
}

resource "google_sql_database_instance" "instance" {
depends_on = [google_service_networking_connection.foobar]
name = "%s"
region = "us-central1"
database_version = "MYSQL_5_7"
deletion_protection = false
settings {
tier = "db-f1-micro"
ip_configuration {
ipv4_enabled = "false"
private_network = data.google_compute_network.servicenet.self_link
}
backup_configuration {
enabled = true
start_time = "00:00"
binary_log_enabled = true
}
}
}
resource "google_sql_database_instance" "replica1" {
depends_on = [google_service_networking_connection.foobar]
name = "%s-replica1"
region = "us-central1"
database_version = "MYSQL_5_7"
deletion_protection = false
settings {
tier = "db-f1-micro"
ip_configuration {
ipv4_enabled = "false"
private_network = data.google_compute_network.servicenet.self_link
allocated_ip_range = google_compute_global_address.foobar.name
}
}

master_instance_name = google_sql_database_instance.instance.name

replica_configuration {
connect_retry_interval = 100
master_heartbeat_period = 10000
password = "password"
username = "username"
ssl_cipher = "ALL"
verify_server_certificate = false
}
}
`, networkName, addressRangeName, databaseName, databaseName)
}

func testAccSqlDatabaseInstance_withPrivateNetwork_withAllocatedIpRangeClone(databaseName, networkName, addressRangeName string) string {
return fmt.Sprintf(`
data "google_compute_network" "servicenet" {
name = "%s"
}

resource "google_compute_global_address" "foobar" {
name = "%s"
purpose = "VPC_PEERING"
address_type = "INTERNAL"
prefix_length = 16
network = data.google_compute_network.servicenet.self_link
}

resource "google_service_networking_connection" "foobar" {
network = data.google_compute_network.servicenet.self_link
service = "servicenetworking.googleapis.com"
reserved_peering_ranges = [google_compute_global_address.foobar.name]
}

resource "google_sql_database_instance" "instance" {
depends_on = [google_service_networking_connection.foobar]
name = "%s"
region = "us-central1"
database_version = "MYSQL_5_7"
deletion_protection = false
settings {
tier = "db-f1-micro"
ip_configuration {
ipv4_enabled = "false"
private_network = data.google_compute_network.servicenet.self_link
}
backup_configuration {
enabled = true
start_time = "00:00"
binary_log_enabled = true
}
}
}

resource "google_sql_database_instance" "clone1" {
name = "%s-clone1"
region = "us-central1"
database_version = "MYSQL_5_7"
deletion_protection = false

clone {
source_instance_name = google_sql_database_instance.instance.name
allocated_ip_range = google_compute_global_address.foobar.name
}

}
`, networkName, addressRangeName, databaseName, databaseName)
}

var testGoogleSqlDatabaseInstance_settings = `
resource "google_sql_database_instance" "instance" {
name = "%s"
Expand Down
2 changes: 2 additions & 0 deletions mmv1/third_party/terraform/website/docs/r/sql_database_instance.html.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,8 @@ The optional `clone` block supports:

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

* `allocated_ip_range` - (Optional) The name of the allocated ip range for the private ip CloudSQL instance. For example: "google-managed-services-default". If set, the cloned instance ip will be created in the allocated range. The range name must comply with [RFC 1035](https://tools.ietf.org/html/rfc1035). Specifically, the name must be 1-63 characters long and match the regular expression [a-z]([-a-z0-9]*[a-z0-9])?.

The optional `restore_backup_context` block supports:
**NOTE:** Restoring from a backup is an imperative action and not recommended via Terraform. Adding or modifying this
block during resource creation/update will trigger the restore action after the resource is created/updated.
Expand Down