diff --git a/README.md b/README.md index 7cadfed..e62cee5 100644 --- a/README.md +++ b/README.md @@ -46,6 +46,10 @@ The Cloud Spanner schema that supports the backend services looks like this. ### Setup infrastructure +You can either set up the Spanner infrastructure using the gcloud command line or Terraform. Instructions for both are below. + +> **NOTE:** The Terraform scripts also create a GKE Autopilot cluster. + #### Gcloud command line To create the Spanner instance using gcloud, you must first [install and configure gcloud](https://cloud.google.com/sdk/docs/install-sdk). @@ -61,9 +65,9 @@ gcloud config set project Now, create the Spanner instance and database: ``` -gcloud spanner instances create game-instance --config=regional-us-central1 --description=gaming-instance --processing-units=500 +gcloud spanner instances create sample-instance --config=regional-us-central1 --description=gaming-instance --processing-units=500 -gcloud spanner databases create --instance game-instance sample-game +gcloud spanner databases create --instance sample-instance sample-game ``` > **NOTE:** The above command will create an instance using the us-central1 [regional configuration](https://cloud.google.com/spanner/docs/instance-configurations) with a compute capacity of 500 processing units. Be aware that creating an instance will start billing your account unless you are under Google Cloud's [free trial credits](https://cloud.google.com/free). @@ -73,7 +77,7 @@ A terraform file is provided that creates the appropriate resources for these sa Resources that are created: - Spanner instance and database based on user variables in main.tfvars -- (TODO) GKE cluster to run the load generators +- [GKE cluster](./docs/GKE.md) to run the services To set up the infrastructure, do the following: @@ -83,41 +87,60 @@ To set up the infrastructure, do the following: ``` cd infrastructure +terraform init cp terraform.tfvars.sample terraform.tfvars vi terraform.tfvars # modify variables +# Authenticate to gcloud services so Terraform can make changes +gcloud auth application-default login + terraform apply ``` ### Setup schema Schema is managed by [Wrench](https://github.com/cloudspannerecosystem/wrench). -After installing wrench, migrate the schema by running the `schema.bash` file (replace project/instance/database information with what was used in terraform file): +After installing wrench, migrate the schema by running the `./scripts/schema.sh` file (replace project/instance/database information with what was used in terraform file): ``` -export SPANNER_PROJECT_ID=PROJECTID -export SPANNER_INSTANCE_ID=INSTANCEID -export SPANNER_DATABASE_ID=DATABASEID -./schema.bash +export SPANNER_PROJECT_ID=YOUR_PROJECT_ID +export SPANNER_INSTANCE_ID=YOUR_INSTANCE_ID +export SPANNER_DATABASE_ID=YOUR_DATABASE_ID +./scripts/schema.sh ``` -### Player profile sample +> **NOTE:** The schema must be in place for the services to work. Do not skip this step! + +### Deploy services +You can deploy the services to the GKE cluster that was configured by Terraform, or you can deploy them locally. + +To deploy to GKE, follow the [instructions here](./docs/GKE.md). + +Otherwise, follow the local deployment instructions for player profile and tradepost. + +Once the services are deployed you can use the generators to [run workloads](./generators/README.md). + +Then follow the README to clean up based on whether you deployed with gcloud or Terraform. + +#### Local player profile deployment + +> **NOTE:** Skip this section if you deployed the services using [GKE](./docs/GKE.md) - Configure [`profile-service`](src/golang/profile-service) either by using environment variables or by copying the `profile-service/config.yml.template` file to `profile-service/config.yml`, and modify the Spanner connection details: ``` -# environment variables -export SPANNER_PROJECT_ID=PROJECTID -export SPANNER_INSTANCE_ID=INSTANCEID -export SPANNER_DATABASE_ID=DATABASEID +# environment variables. change the YOUR_* values to your information +export SPANNER_PROJECT_ID=YOUR_PROJECT_ID +export SPANNER_INSTANCE_ID=YOUR_INSTANCE_ID +export SPANNER_DATABASE_ID=YOUR_DATABASE_ID ``` ``` -# config.yml spanner connection details +# config.yml spanner connection details. change the YOUR_* values to your information spanner: - project_id: YOUR_GCP_PROJECT_ID - instance_id: YOUR_SPANNER_INSTANCE_ID - database_id: YOUR_SPANNER_DATABASE_ID + project_id: YOUR_PROJECT_ID + instance_id: YOUR_INSTANCE_ID + database_id: YOUR_DATABASE_ID ``` @@ -132,17 +155,17 @@ go run . ``` # environment variables -export SPANNER_PROJECT_ID=PROJECTID -export SPANNER_INSTANCE_ID=INSTANCEID -export SPANNER_DATABASE_ID=DATABASEID +export SPANNER_PROJECT_ID=YOUR_PROJECT_ID +export SPANNER_INSTANCE_ID=YOUR_INSTANCE_ID +export SPANNER_DATABASE_ID=YOUR_DATABASE_ID ``` ``` # config.yml spanner connection details spanner: - project_id: YOUR_GCP_PROJECT_ID - instance_id: YOUR_SPANNER_INSTANCE_ID - database_id: YOUR_SPANNER_DATABASE_ID + project_id: YOUR_PROJECT_ID + instance_id: YOUR_INSTANCE_ID + database_id: YOUR_DATABASE_ID ``` @@ -153,25 +176,25 @@ cd src/golang/matchmaking-service go run . ``` -- [Generate load](generators/README.md). +#### Local player trading deployment -### Player trading sample +> **NOTE:** Skip this section if you deployed the services using [GKE](./docs/GKE.md) - Configure [`item-service`](src/golang/item-service) either by using environment variables or by copying the `item-service/config.yml.template` file to `item-service/config.yml`, and modify the Spanner connection details: ``` # environment variables -export SPANNER_PROJECT_ID=PROJECTID -export SPANNER_INSTANCE_ID=INSTANCEID -export SPANNER_DATABASE_ID=DATABASEID +export SPANNER_PROJECT_ID=YOUR_PROJECT_ID +export SPANNER_INSTANCE_ID=YOUR_INSTANCE_ID +export SPANNER_DATABASE_ID=YOUR_DATABASE_ID ``` ``` # config.yml spanner connection details spanner: - project_id: YOUR_GCP_PROJECT_ID - instance_id: YOUR_SPANNER_INSTANCE_ID - database_id: YOUR_SPANNER_DATABASE_ID + project_id: YOUR_PROJECT_ID + instance_id: YOUR_INSTANCE_ID + database_id: YOUR_DATABASE_ID ``` @@ -186,17 +209,17 @@ go run . ``` # environment variables -export SPANNER_PROJECT_ID=PROJECTID -export SPANNER_INSTANCE_ID=INSTANCEID -export SPANNER_DATABASE_ID=DATABASEID +export SPANNER_PROJECT_ID=YOUR_PROJECT_ID +export SPANNER_INSTANCE_ID=YOUR_INSTANCE_ID +export SPANNER_DATABASE_ID=YOUR_DATABASE_ID ``` ``` # config.yml spanner connection details spanner: - project_id: YOUR_GCP_PROJECT_ID - instance_id: YOUR_SPANNER_INSTANCE_ID - database_id: YOUR_SPANNER_DATABASE_ID + project_id: YOUR_PROJECT_ID + instance_id: YOUR_INSTANCE_ID + database_id: YOUR_DATABASE_ID ``` @@ -207,31 +230,7 @@ cd src/golang/tradepost-service go run . ``` -- [Generate load](generators/README.md). - - -### Generator dependencies - -The generators are run by Locust.io, which is a Python framework for generating load. - -There are several dependencies required to get the generators to work: - -- Python 3.7+ -- Locust - -Assuming python3.X is installed, install dependencies via [pip](https://pypi.org/project/pip/): - -``` -# if pip3 is symlinked to pip -pip install -r requirements.txt - -# if pip3 is not symlinked to pip -pip3 install -r requirements.txt -``` - -> **NOTE:** To avoid modifying existing pip libraries on your machine, consider a solution like [virtualenv](https://pypi.org/project/virtualenv/). - -## How to build the services +## How to build the services locally A Makefile is provided to build the services. Example commands: @@ -272,7 +271,7 @@ make test-all If the Spanner instance was created using the gcloud command line, it can be delete using gcloud: ``` -gcloud spanner instances delete game-instance +gcloud spanner instances delete sample-instance ``` ### Terraform diff --git a/docs/GKE.md b/docs/GKE.md new file mode 100644 index 0000000..388ad54 --- /dev/null +++ b/docs/GKE.md @@ -0,0 +1,76 @@ +# GKE + +This repository provides support for running the backend applications on GKE. + +## Setup + +### Terraform +The provided Terraform [gke.tf](../infrastructure/gke.tf) will provision a [GKE Autopilot](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview) cluster. This method of provisioning GKE will automatically manage the nodes for the cluster as the backend applications are added. + +These terraform [services.tf](../infrastructure/services.tf) file enables the required cloud API services, such as `cloudbuild.googleapis.com` and `container.googleapis.com`. + +Additionally, there is a Cloud Spanner service account created for the backend services. + +### Kubectl +To interact with the GKE cluster, ensure kubectl is installed. + +Once that is done, authenticate to GKE with the following commands: + +``` +export USE_GKE_GCLOUD_AUTH_PLUGIN=True +export GKE_CLUSTER=sample-game-gke # change this based on the terraform configuration +gcloud container clusters get-credentials $GKE_CLUSTER --region us-central1 +kubectl get namespaces +``` + +If there are no issues with the kubectl commands, kubectl is properly authenticated. + +### Create Cloud Build images +Each service has a dockerfile that needs to be deployed to a container registry, such as [Google Container Registry](https://cloud.google.com/container-registry). The appropriate Cloud API services will have been enabled by the Terraform scripts found in the infrastructure folder. + +To deploy the services, run the [`scripts/build.sh`](../scripts/build.sh) script. This will submit all service images to Cloud Build. + +``` +export PROJECT_ID= +./scripts/build.sh +``` + +> **NOTE:** This will take some time time to complete all builds + +### Spanner configuration and secrets + +Once the images have been built, it is time to deploy the [kubernetes manifests](../kubernetes-manifests). Each backend application provides a LoadBalance service and a deployment. + +Create the secret for the service account that will connect to the Spanner instance: + +``` +export SERVICE_ACCOUNT=sample-game-backend +gcloud iam service-accounts keys create backend_sa_key.json \ + --iam-account=${SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com + +kubectl create secret generic sample-game-backend-sa-key \ + --from-file=backend_sa_key.json=./backend_sa_key.json +``` + +Create a config map for the Spanner instance + +``` +sed -e "s/PROJECT_ID/$PROJECT_ID/" \ + -e "s/INSTANCE_ID/$SPANNER_INSTANCE_ID/" \ + -e "s/DATABASE_ID/$SPANNER_DATABASE_ID/" \ + spanner_config.yaml.tmpl > ./kubernetes-manifests/spanner_config.yaml + +kubectl apply -f spanner_config.yaml +``` + +### Deploy the manifests +Once you have the kubernetes secret and config map established to connec to Cloud Spanner, the only thing left is to deploy the manifests. A [`scripts/deploy.sh`](../scripts/deploy.sh) file has been created to assist with this process. + +``` +export PROJECT_ID= +./scripts/deploy.sh +``` + +## What's next + +Once the services are deployed you can use the generators to [run workloads](../generators/README.md). diff --git a/generators/README.md b/generators/README.md index acd8f66..6e2948f 100644 --- a/generators/README.md +++ b/generators/README.md @@ -4,15 +4,34 @@ These generators leverage the [Locust](https://locust.io) python framework for g The generators can be run via the command line, or a web interface. -If using the web interface, when you run the `locust` command for each service, you can point your web browser to the exposed port to determine -concurrency of the load, in terms of "users". Then the load runs until the test is stopped in the browser. +If using the web interface, when you run the `locust` command for each service, you can point your web browser to the exposed port to determine concurrency of the load, in terms of "users". Then the load runs until the test is stopped in the browser. Various charts are provided by the web interface to indicate the performance of the load test. If you do not want to use the web interface, the command line options specify the user concurrency, as well as a run time. Statistics are printed on the command line for the test. +### Generator dependencies +There are several dependencies required to get the generators to work: + +- Python 3.7+ +- Locust + +Assuming python3.X is installed, install dependencies via [pip](https://pypi.org/project/pip/): + +``` +# if pip3 is symlinked to pip +pip install -r requirements.txt + +# if pip3 is not symlinked to pip +pip3 install -r requirements.txt +``` + +> **NOTE:** To avoid modifying existing pip libraries on your machine, consider a solution like [virtualenv](https://pypi.org/project/virtualenv/). + + +## Using the generators Provided generators do the following: - _authentication\_server.py_: mimics player signup and player retrieval by UUID. Login is not handled currently due to the necessity to track password creation. diff --git a/generators/match_server.py b/generators/match_server.py index 214279c..9cfb42b 100644 --- a/generators/match_server.py +++ b/generators/match_server.py @@ -24,7 +24,7 @@ class GameMatch(HttpUser): leveraging the matchmaking-service """ - @task(2) + @task(1) def create_game(self): """Task to create a new game""" diff --git a/infrastructure/gke.tf b/infrastructure/gke.tf new file mode 100644 index 0000000..1efc31f --- /dev/null +++ b/infrastructure/gke.tf @@ -0,0 +1,49 @@ +// Copyright 2022 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +variable "gke_config" { + type = object({ + cluster_name = string + location = string + resource_labels = map(string) + }) + + description = "The configuration specifications for a GKE Autopilot cluster" +} + +resource "google_compute_network" "vpc" { + name = "cymbal-game-staging-vpc" + auto_create_subnetworks = false +} + +resource "google_compute_subnetwork" "subnet" { + name = "cymbal-game-subnet" + ip_cidr_range = "10.1.0.0/16" + region = "us-central1" + network = google_compute_network.vpc.id +} + +resource "google_container_cluster" "sample-game-gke" { + name = var.gke_config.cluster_name + location = var.gke_config.location + + network = google_compute_network.vpc.name + subnetwork = google_compute_subnetwork.subnet.name + + # See issue: https://github.com/hashicorp/terraform-provider-google/issues/10782 + ip_allocation_policy {} + +# Enabling Autopilot for this cluster + enable_autopilot = true +} diff --git a/infrastructure/main.tf b/infrastructure/main.tf index bc84112..8e56aaf 100644 --- a/infrastructure/main.tf +++ b/infrastructure/main.tf @@ -16,73 +16,6 @@ variable "gcp_project" { type = string } -variable "spanner_config" { - type = object({ - instance_name = string - database_name = string - configuration = string - display_name = string - processing_units = number - environment = string - }) - description = "The configuration specifications for the Spanner instance" - - validation { - condition = length(var.spanner_config.display_name) >= 4 && length(var.spanner_config.display_name) <= "30" - error_message = "Display name must be between 4-30 characters long." - } - - validation { - condition = (var.spanner_config.processing_units <= 1000) && (var.spanner_config.processing_units%100) == 0 - error_message = "Processing units must be 1000 or less, and be a multiple of 100." - } -} - provider "google" { project = var.gcp_project } - -resource "google_spanner_instance" "instance" { - name = var.spanner_config.instance_name # << be careful changing this in production - config = var.spanner_config.configuration - display_name = var.spanner_config.display_name - processing_units = var.spanner_config.processing_units - labels = { "env" = var.spanner_config.environment } -} - -resource "google_spanner_database" "database" { - instance = google_spanner_instance.instance.name - name = var.spanner_config.database_name - deletion_protection = false -} - -# Create IAM service account for locked down cloud run container to access Spanner service -resource "google_service_account" "profile_backend_sa" { - account_id = "profile-backend" - display_name = "Player profile backend service" -} - -resource "google_spanner_database_iam_binding" "profile_iam_spanner" { - instance = google_spanner_instance.instance.name - database = google_spanner_database.database.name - role = "roles/spanner.databaseUser" - - members = [ - "serviceAccount:${google_service_account.profile_backend_sa.email}", - ] -} - -# Create user frontend IAM service account and appropriate service -# resource "google_service_account" "user_frontend_sa" { -# account_id = "user-frontend" -# display_name = "User Frontend Service" -# } - -# resource "google_project_iam_binding" "user_frontend_iam" { -# project = var.gcp_project -# role = "roles/run.invoker" - -# members = [ -# "serviceAccount:${google_service_account.user_frontend_sa.email}", -# ] -# } diff --git a/infrastructure/services.tf b/infrastructure/services.tf new file mode 100644 index 0000000..481cc58 --- /dev/null +++ b/infrastructure/services.tf @@ -0,0 +1,31 @@ +// Copyright 2022 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +variable "gcp_services" { + description = "Enable project services" + type = list(string) + default = ["compute.googleapis.com", "spanner.googleapis.com", "cloudbuild.googleapis.com", "container.googleapis.com", "artifactregistry.googleapis.com"] +} + +resource "google_project_service" "project_services" { + count = length(var.gcp_services) + service = var.gcp_services[count.index] + + timeouts { + create = "30m" + update = "40m" + } + + disable_dependent_services = true +} diff --git a/infrastructure/spanner.tf b/infrastructure/spanner.tf new file mode 100644 index 0000000..e524134 --- /dev/null +++ b/infrastructure/spanner.tf @@ -0,0 +1,73 @@ +// Copyright 2022 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +variable "spanner_config" { + type = object({ + instance_name = string + database_name = string + configuration = string + display_name = string + processing_units = number + environment = string + }) + description = "The configuration specifications for the Spanner instance" + + validation { + condition = length(var.spanner_config.display_name) >= 4 && length(var.spanner_config.display_name) <= "30" + error_message = "Display name must be between 4-30 characters long." + } + + validation { + condition = (var.spanner_config.processing_units <= 1000) && (var.spanner_config.processing_units%100) == 0 + error_message = "Processing units must be 1000 or less, and be a multiple of 100." + } +} + +variable backend_sa_config { + type = object({ + name = string + description = string + }) + description = "The configuration specifications for the backend service account" +} + +resource "google_spanner_instance" "instance" { + name = var.spanner_config.instance_name # << be careful changing this in production + config = var.spanner_config.configuration + display_name = var.spanner_config.display_name + processing_units = var.spanner_config.processing_units + labels = { "env" = var.spanner_config.environment } +} + +resource "google_spanner_database" "database" { + instance = google_spanner_instance.instance.name + name = var.spanner_config.database_name + deletion_protection = true +} + +# Create IAM service account for locked down cloud run container to access Spanner service +resource "google_service_account" "backend_sa" { + account_id = var.backend_sa_config.name + display_name = var.backend_sa_config.description +} + +resource "google_spanner_database_iam_binding" "backend_iam_spanner" { + instance = google_spanner_instance.instance.name + database = google_spanner_database.database.name + role = "roles/spanner.databaseUser" + + members = [ + "serviceAccount:${google_service_account.backend_sa.email}", + ] +} diff --git a/infrastructure/terraform.tfvars.sample b/infrastructure/terraform.tfvars.sample index 930480b..5a7e97d 100644 --- a/infrastructure/terraform.tfvars.sample +++ b/infrastructure/terraform.tfvars.sample @@ -1,11 +1,24 @@ gcp_project = "PROJECT" spanner_config = { - instance_name = "INSTANCE" - database_name = "DATABASE" + instance_name = "sample-instance" + database_name = "sample-game" configuration = "regional-us-central1" display_name = "gaming spanner instance" processing_units = 100 environment = "staging" } +backend_sa_config = { + name = "sample-game-backend" + description = "Sample Game backend service account" +} + +gke_config = { + cluster_name = "sample-game-gke" + location = "us-central1" + resource_labels = { + environment = "staging" + } +} + diff --git a/kubernetes-manifests/.gitignore b/kubernetes-manifests/.gitignore new file mode 100644 index 0000000..1e82fc7 --- /dev/null +++ b/kubernetes-manifests/.gitignore @@ -0,0 +1 @@ +*.yaml diff --git a/kubernetes-manifests/item-service.yaml.tmpl b/kubernetes-manifests/item-service.yaml.tmpl new file mode 100644 index 0000000..7d376e8 --- /dev/null +++ b/kubernetes-manifests/item-service.yaml.tmpl @@ -0,0 +1,83 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +kind: Service +metadata: + name: item +spec: + type: LoadBalancer + selector: + app: cymbal-games-item + ports: + - port: 80 + targetPort: 80 +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: cymbal-games-item +spec: + replicas: 1 # EDIT: Number of instances of deployment + selector: + matchLabels: + app: cymbal-games-item + template: + metadata: + labels: + app: cymbal-games-item + spec: + volumes: + - name: backend-sa-key + secret: + secretName: sample-game-backend-sa-key + containers: + - name: item-service + image: gcr.io/GCP_PROJECT_ID/item-service:latest # EDIT: Your GCP Project ID + ports: + - containerPort: 80 + volumeMounts: + - mountPath: /var/secrets + name: backend-sa-key + env: + - name: GOOGLE_APPLICATION_CREDENTIALS + value: /var/secrets/backend_sa_key.json + - name: SPANNER_PROJECT_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_project_id + - name: SPANNER_INSTANCE_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_instance_id + - name: SPANNER_DATABASE_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_database_id + - name: SERVICE_HOST + value: "0.0.0.0" + - name: SERVICE_PORT + value: "80" + resources: + requests: + cpu: "500m" + memory: "512Mi" + ephemeral-storage: "100Mi" + limits: + cpu: "500m" + memory: "512Mi" + ephemeral-storage: "100Mi" diff --git a/kubernetes-manifests/matchmaking-service.yaml.tmpl b/kubernetes-manifests/matchmaking-service.yaml.tmpl new file mode 100644 index 0000000..13b5b4a --- /dev/null +++ b/kubernetes-manifests/matchmaking-service.yaml.tmpl @@ -0,0 +1,83 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +kind: Service +metadata: + name: matchmaking +spec: + type: LoadBalancer + selector: + app: cymbal-games-matchmaking + ports: + - port: 80 + targetPort: 80 +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: cymbal-games-matchmaking +spec: + replicas: 1 # EDIT: Number of instances of deployment + selector: + matchLabels: + app: cymbal-games-matchmaking + template: + metadata: + labels: + app: cymbal-games-matchmaking + spec: + volumes: + - name: backend-sa-key + secret: + secretName: sample-game-backend-sa-key + containers: + - name: matchmaking-service + image: gcr.io/GCP_PROJECT_ID/matchmaking-service:latest # EDIT: Your GCP Project ID + ports: + - containerPort: 80 + volumeMounts: + - mountPath: /var/secrets + name: backend-sa-key + env: + - name: GOOGLE_APPLICATION_CREDENTIALS + value: /var/secrets/backend_sa_key.json + - name: SPANNER_PROJECT_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_project_id + - name: SPANNER_INSTANCE_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_instance_id + - name: SPANNER_DATABASE_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_database_id + - name: SERVICE_HOST + value: "0.0.0.0" + - name: SERVICE_PORT + value: "80" + resources: + requests: + cpu: "500m" + memory: "512Mi" + ephemeral-storage: "100Mi" + limits: + cpu: "500m" + memory: "512Mi" + ephemeral-storage: "100Mi" diff --git a/kubernetes-manifests/profile-service.yaml.tmpl b/kubernetes-manifests/profile-service.yaml.tmpl new file mode 100644 index 0000000..29e708a --- /dev/null +++ b/kubernetes-manifests/profile-service.yaml.tmpl @@ -0,0 +1,83 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +kind: Service +metadata: + name: profile +spec: + type: LoadBalancer + selector: + app: cymbal-games-profile + ports: + - port: 80 + targetPort: 80 +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: cymbal-games-profile +spec: + replicas: 2 # EDIT: Number of instances of deployment + selector: + matchLabels: + app: cymbal-games-profile + template: + metadata: + labels: + app: cymbal-games-profile + spec: + volumes: + - name: backend-sa-key + secret: + secretName: sample-game-backend-sa-key + containers: + - name: profile-service + image: gcr.io/GCP_PROJECT_ID/profile-service:latest # EDIT: Your GCP Project ID + ports: + - containerPort: 80 + volumeMounts: + - mountPath: /var/secrets + name: backend-sa-key + env: + - name: GOOGLE_APPLICATION_CREDENTIALS + value: /var/secrets/backend_sa_key.json + - name: SPANNER_PROJECT_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_project_id + - name: SPANNER_INSTANCE_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_instance_id + - name: SPANNER_DATABASE_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_database_id + - name: SERVICE_HOST + value: "0.0.0.0" + - name: SERVICE_PORT + value: "80" + resources: + requests: + cpu: "1" + memory: "1Gi" + ephemeral-storage: "100Mi" + limits: + cpu: "1" + memory: "1Gi" + ephemeral-storage: "100Mi" diff --git a/kubernetes-manifests/spanner_config.yaml.tmpl b/kubernetes-manifests/spanner_config.yaml.tmpl new file mode 100644 index 0000000..97823e3 --- /dev/null +++ b/kubernetes-manifests/spanner_config.yaml.tmpl @@ -0,0 +1,8 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: spanner-config +data: + spanner_project_id: PROJECT_ID # EDIT: Your GCP Project ID + spanner_instance_id: INSTANCE_ID # EDIT: Your Spanner Instance ID + spanner_database_id: DATABASE_ID # EDIT: Your Spanner Database ID diff --git a/kubernetes-manifests/tradepost-service.yaml.tmpl b/kubernetes-manifests/tradepost-service.yaml.tmpl new file mode 100644 index 0000000..2fb6375 --- /dev/null +++ b/kubernetes-manifests/tradepost-service.yaml.tmpl @@ -0,0 +1,83 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +apiVersion: v1 +kind: Service +metadata: + name: tradepost +spec: + type: LoadBalancer + selector: + app: cymbal-games-tradepost + ports: + - port: 80 + targetPort: 80 +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: cymbal-games-tradepost +spec: + replicas: 1 # EDIT: Number of instances of deployment + selector: + matchLabels: + app: cymbal-games-tradepost + template: + metadata: + labels: + app: cymbal-games-tradepost + spec: + volumes: + - name: backend-sa-key + secret: + secretName: sample-game-backend-sa-key + containers: + - name: tradepost-service + image: gcr.io/GCP_PROJECT_ID/tradepost-service:latest # EDIT: Your GCP Project ID + ports: + - containerPort: 80 + volumeMounts: + - mountPath: /var/secrets + name: backend-sa-key + env: + - name: GOOGLE_APPLICATION_CREDENTIALS + value: /var/secrets/backend_sa_key.json + - name: SPANNER_PROJECT_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_project_id + - name: SPANNER_INSTANCE_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_instance_id + - name: SPANNER_DATABASE_ID + valueFrom: + configMapKeyRef: + name: spanner-config + key: spanner_database_id + - name: SERVICE_HOST + value: "0.0.0.0" + - name: SERVICE_PORT + value: "80" + resources: + requests: + cpu: "500m" + memory: "512Mi" + ephemeral-storage: "100Mi" + limits: + cpu: "500m" + memory: "512Mi" + ephemeral-storage: "100Mi" diff --git a/scripts/build.sh b/scripts/build.sh new file mode 100755 index 0000000..46dfe78 --- /dev/null +++ b/scripts/build.sh @@ -0,0 +1,35 @@ +#!/bin/bash +# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# A convenience script to help developers easily submit service builds to Cloud Build +if [ -z "${PROJECT_ID}" ]; then + echo "[ERROR] PROJECT_ID environment variable must be set" >&2 + exit 1 +fi + +basedir=`pwd` + +# Submit a build command to +for service in profile-service matchmaking-service item-service tradepost-service; do + cd "${basedir}/src/golang/${service}" + echo "[INFO] Building ${service}" + gcloud builds submit --tag gcr.io/$PROJECT_ID/$service . + + if [ $? -ne 0 ]; then + echo "[ERROR] Build failed...stopping further builds" >&2 + exit 1 + fi +done diff --git a/scripts/deploy.sh b/scripts/deploy.sh new file mode 100755 index 0000000..f045047 --- /dev/null +++ b/scripts/deploy.sh @@ -0,0 +1,39 @@ +#!/bin/bash +# +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# A convenience script to help developers easily deploy services to GKE cluster +if [ -z "${PROJECT_ID}" ]; then + echo "[ERROR] PROJECT_ID environment variable must be set" >&2 + exit 1 +fi + +basedir=`pwd` + +cd "${basedir}/kubernetes-manifests" + +# Submit a kubectl apply for each deployment file +for service in profile-service matchmaking-service item-service tradepost-service; do + echo "[INFO] Configuring ${service}" + sed "s/GCP_PROJECT_ID/${PROJECT_ID}/" "${service}.yaml.tmpl" > "${service}.yaml" + + echo "[INFO] Deploying ${service}" + kubectl apply -f "${service}.yaml" + + if [ $? -ne 0 ]; then + echo "[ERROR] Deploy failed...stopping further deploys" >&2 + exit 1 + fi +done diff --git a/schema.bash b/scripts/schema.sh similarity index 100% rename from schema.bash rename to scripts/schema.sh diff --git a/src/golang/item-service/Dockerfile b/src/golang/item-service/Dockerfile index b7b11c0..3d47ec2 100644 --- a/src/golang/item-service/Dockerfile +++ b/src/golang/item-service/Dockerfile @@ -18,8 +18,11 @@ COPY . . RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o item-service -ldflags="-w -s" . -FROM scratch +FROM alpine:latest as certs +RUN apk --update add ca-certificates +FROM scratch +COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt COPY --from=builder /app/item-service /usr/bin/ EXPOSE 80 diff --git a/src/golang/matchmaking-service/Dockerfile b/src/golang/matchmaking-service/Dockerfile index 0d5fe8c..11b7b26 100644 --- a/src/golang/matchmaking-service/Dockerfile +++ b/src/golang/matchmaking-service/Dockerfile @@ -18,8 +18,11 @@ COPY . . RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o matchmaking-service -ldflags="-w -s" . -FROM scratch +FROM alpine:latest as certs +RUN apk --update add ca-certificates +FROM scratch +COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt COPY --from=builder /app/matchmaking-service /usr/bin/ EXPOSE 80 diff --git a/src/golang/profile-service/Dockerfile b/src/golang/profile-service/Dockerfile index 050613c..ab3f483 100644 --- a/src/golang/profile-service/Dockerfile +++ b/src/golang/profile-service/Dockerfile @@ -18,8 +18,11 @@ COPY . . RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o profile-service -ldflags="-w -s" . -FROM scratch +FROM alpine:latest as certs +RUN apk --update add ca-certificates +FROM scratch +COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt COPY --from=builder /app/profile-service /usr/bin/ EXPOSE 80 diff --git a/src/golang/tradepost-service/Dockerfile b/src/golang/tradepost-service/Dockerfile index 383bc21..94d7cc2 100644 --- a/src/golang/tradepost-service/Dockerfile +++ b/src/golang/tradepost-service/Dockerfile @@ -18,8 +18,11 @@ COPY . . RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o tradepost-service -ldflags="-w -s" . -FROM scratch +FROM alpine:latest as certs +RUN apk --update add ca-certificates +FROM scratch +COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt COPY --from=builder /app/tradepost-service /usr/bin/ EXPOSE 80