Closes #2 - Adds GKE Autopilot to infrastructure (#36)

* Closes #2

README.md

@@ -46,6 +46,10 @@ The Cloud Spanner schema that supports the backend services looks like this.
 
 ### Setup infrastructure
 
+You can either set up the Spanner infrastructure using the gcloud command line or Terraform. Instructions for both are below.
+
+> **NOTE:** The Terraform scripts also create a GKE Autopilot cluster.
+
 #### Gcloud command line
 
 To create the Spanner instance using gcloud, you must first [install and configure gcloud](https://cloud.google.com/sdk/docs/install-sdk).
@@ -61,9 +65,9 @@ gcloud config set project <PROJECT_ID>
 Now, create the Spanner instance and database:
 
 ```
-gcloud spanner instances create game-instance --config=regional-us-central1 --description=gaming-instance --processing-units=500
+gcloud spanner instances create sample-instance --config=regional-us-central1 --description=gaming-instance --processing-units=500
 
-gcloud spanner databases create --instance game-instance sample-game
+gcloud spanner databases create --instance sample-instance sample-game
 ```
 
 > **NOTE:** The above command will create an instance using the us-central1 [regional configuration](https://cloud.google.com/spanner/docs/instance-configurations) with a compute capacity of 500 processing units. Be aware that creating an instance will start billing your account unless you are under Google Cloud's [free trial credits](https://cloud.google.com/free).
@@ -73,7 +77,7 @@ A terraform file is provided that creates the appropriate resources for these sa
 
 Resources that are created:
 - Spanner instance and database based on user variables in main.tfvars
-- (TODO) GKE cluster to run the load generators
+- [GKE cluster](./docs/GKE.md) to run the services
 
 To set up the infrastructure, do the following:
 
@@ -83,41 +87,60 @@ To set up the infrastructure, do the following:
 
 ```
 cd infrastructure
+terraform init
 cp terraform.tfvars.sample terraform.tfvars
 vi terraform.tfvars # modify variables
 
+# Authenticate to gcloud services so Terraform can make changes
+gcloud auth application-default login
+
 terraform apply
 ```
 
 ### Setup schema
 Schema is managed by [Wrench](https://github.com/cloudspannerecosystem/wrench).
 
-After installing wrench, migrate the schema by running the `schema.bash` file (replace project/instance/database information with what was used in terraform file):
+After installing wrench, migrate the schema by running the `./scripts/schema.sh` file (replace project/instance/database information with what was used in terraform file):
 
 ```
-export SPANNER_PROJECT_ID=PROJECTID
-export SPANNER_INSTANCE_ID=INSTANCEID
-export SPANNER_DATABASE_ID=DATABASEID
-./schema.bash
+export SPANNER_PROJECT_ID=YOUR_PROJECT_ID
+export SPANNER_INSTANCE_ID=YOUR_INSTANCE_ID
+export SPANNER_DATABASE_ID=YOUR_DATABASE_ID
+./scripts/schema.sh
 ```
 
-### Player profile sample
+> **NOTE:** The schema must be in place for the services to work. Do not skip this step!
+
+### Deploy services
+You can deploy the services to the GKE cluster that was configured by Terraform, or you can deploy them locally.
+
+To deploy to GKE, follow the [instructions here](./docs/GKE.md).
+
+Otherwise, follow the local deployment instructions for player profile and tradepost.
+
+Once the services are deployed you can use the generators to [run workloads](./generators/README.md).
+
+Then follow the README to clean up based on whether you deployed with gcloud or Terraform.
+
+#### Local player profile deployment
+
+> **NOTE:** Skip this section if you deployed the services using [GKE](./docs/GKE.md)
 
 - Configure [`profile-service`](src/golang/profile-service) either by using environment variables or by copying the `profile-service/config.yml.template` file to `profile-service/config.yml`, and modify the Spanner connection details:
 
 ```
-# environment variables
-export SPANNER_PROJECT_ID=PROJECTID
-export SPANNER_INSTANCE_ID=INSTANCEID
-export SPANNER_DATABASE_ID=DATABASEID
+# environment variables. change the YOUR_* values to your information
+export SPANNER_PROJECT_ID=YOUR_PROJECT_ID
+export SPANNER_INSTANCE_ID=YOUR_INSTANCE_ID
+export SPANNER_DATABASE_ID=YOUR_DATABASE_ID
 ```
 
 ```
-# config.yml spanner connection details
+# config.yml spanner connection details. change the YOUR_* values to your information
 spanner:
-  project_id: YOUR_GCP_PROJECT_ID
-  instance_id: YOUR_SPANNER_INSTANCE_ID
-  database_id: YOUR_SPANNER_DATABASE_ID
+  project_id: YOUR_PROJECT_ID
+  instance_id: YOUR_INSTANCE_ID
+  database_id: YOUR_DATABASE_ID
 
 ```
 
@@ -132,17 +155,17 @@ go run .
 
 ```
 # environment variables
-export SPANNER_PROJECT_ID=PROJECTID
-export SPANNER_INSTANCE_ID=INSTANCEID
-export SPANNER_DATABASE_ID=DATABASEID
+export SPANNER_PROJECT_ID=YOUR_PROJECT_ID
+export SPANNER_INSTANCE_ID=YOUR_INSTANCE_ID
+export SPANNER_DATABASE_ID=YOUR_DATABASE_ID
 ```
 
 ```
 # config.yml spanner connection details
 spanner:
-  project_id: YOUR_GCP_PROJECT_ID
-  instance_id: YOUR_SPANNER_INSTANCE_ID
-  database_id: YOUR_SPANNER_DATABASE_ID
+  project_id: YOUR_PROJECT_ID
+  instance_id: YOUR_INSTANCE_ID
+  database_id: YOUR_DATABASE_ID
 
 ```
 
@@ -153,25 +176,25 @@ cd src/golang/matchmaking-service
 go run .
 ```
 
-- [Generate load](generators/README.md).
+#### Local player trading deployment
 
-### Player trading sample
+> **NOTE:** Skip this section if you deployed the services using [GKE](./docs/GKE.md)
 
 - Configure [`item-service`](src/golang/item-service) either by using environment variables or by copying the `item-service/config.yml.template` file to `item-service/config.yml`, and modify the Spanner connection details:
 
 ```
 # environment variables
-export SPANNER_PROJECT_ID=PROJECTID
-export SPANNER_INSTANCE_ID=INSTANCEID
-export SPANNER_DATABASE_ID=DATABASEID
+export SPANNER_PROJECT_ID=YOUR_PROJECT_ID
+export SPANNER_INSTANCE_ID=YOUR_INSTANCE_ID
+export SPANNER_DATABASE_ID=YOUR_DATABASE_ID
 ```
 
 ```
 # config.yml spanner connection details
 spanner:
-  project_id: YOUR_GCP_PROJECT_ID
-  instance_id: YOUR_SPANNER_INSTANCE_ID
-  database_id: YOUR_SPANNER_DATABASE_ID
+  project_id: YOUR_PROJECT_ID
+  instance_id: YOUR_INSTANCE_ID
+  database_id: YOUR_DATABASE_ID
 
 ```
 
@@ -186,17 +209,17 @@ go run .
 
 ```
 # environment variables
-export SPANNER_PROJECT_ID=PROJECTID
-export SPANNER_INSTANCE_ID=INSTANCEID
-export SPANNER_DATABASE_ID=DATABASEID
+export SPANNER_PROJECT_ID=YOUR_PROJECT_ID
+export SPANNER_INSTANCE_ID=YOUR_INSTANCE_ID
+export SPANNER_DATABASE_ID=YOUR_DATABASE_ID
 ```
 
 ```
 # config.yml spanner connection details
 spanner:
-  project_id: YOUR_GCP_PROJECT_ID
-  instance_id: YOUR_SPANNER_INSTANCE_ID
-  database_id: YOUR_SPANNER_DATABASE_ID
+  project_id: YOUR_PROJECT_ID
+  instance_id: YOUR_INSTANCE_ID
+  database_id: YOUR_DATABASE_ID
 
 ```
 
@@ -207,31 +230,7 @@ cd src/golang/tradepost-service
 go run .
 ```
 
-- [Generate load](generators/README.md).
-
-
-### Generator dependencies
-
-The generators are run by Locust.io, which is a Python framework for generating load.
-
-There are several dependencies required to get the generators to work:
-
-- Python 3.7+
-- Locust
-
-Assuming python3.X is installed, install dependencies via [pip](https://pypi.org/project/pip/):
-
-```
-# if pip3 is symlinked to pip
-pip install -r requirements.txt
-
-# if pip3 is not symlinked to pip
-pip3 install -r requirements.txt
-```
-
-> **NOTE:** To avoid modifying existing pip libraries on your machine, consider a solution like [virtualenv](https://pypi.org/project/virtualenv/).
-
-## How to build the services
+## How to build the services locally
 
 A Makefile is provided to build the services. Example commands:
 
@@ -272,7 +271,7 @@ make test-all
 If the Spanner instance was created using the gcloud command line, it can be delete using gcloud:
 
 ```
-gcloud spanner instances delete game-instance
+gcloud spanner instances delete sample-instance
 ```
 
 ### Terraform

docs/GKE.md

@@ -0,0 +1,76 @@
+# GKE
+
+This repository provides support for running the backend applications on GKE.
+
+## Setup
+
+### Terraform
+The provided Terraform [gke.tf](../infrastructure/gke.tf) will provision a [GKE Autopilot](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview) cluster. This method of provisioning GKE will automatically manage the nodes for the cluster as the backend applications are added.
+
+These terraform [services.tf](../infrastructure/services.tf) file enables the required cloud API services, such as `cloudbuild.googleapis.com` and `container.googleapis.com`.
+
+Additionally, there is a Cloud Spanner service account created for the backend services.
+
+### Kubectl
+To interact with the GKE cluster, ensure kubectl is installed.
+
+Once that is done, authenticate to GKE with the following commands:
+
+```
+export USE_GKE_GCLOUD_AUTH_PLUGIN=True
+export GKE_CLUSTER=sample-game-gke # change this based on the terraform configuration
+gcloud container clusters get-credentials $GKE_CLUSTER --region us-central1
+kubectl get namespaces
+```
+
+If there are no issues with the kubectl commands, kubectl is properly authenticated.
+
+### Create Cloud Build images
+Each service has a dockerfile that needs to be deployed to a container registry, such as [Google Container Registry](https://cloud.google.com/container-registry). The appropriate Cloud API services will have been enabled by the Terraform scripts found in the infrastructure folder.
+
+To deploy the services, run the [`scripts/build.sh`](../scripts/build.sh) script. This will submit all service images to Cloud Build.
+
+```
+export PROJECT_ID=<YOUR_PROJECT_ID>
+./scripts/build.sh
+```
+
+> **NOTE:** This will take some time time to complete all builds
+
+### Spanner configuration and secrets
+
+Once the images have been built, it is time to deploy the [kubernetes manifests](../kubernetes-manifests). Each backend application provides a LoadBalance service and a deployment.
+
+Create the secret for the service account that will connect to the Spanner instance:
+
+```
+export SERVICE_ACCOUNT=sample-game-backend
+gcloud iam service-accounts keys create backend_sa_key.json \
+    --iam-account=${SERVICE_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com
+
+kubectl create secret generic sample-game-backend-sa-key \
+ --from-file=backend_sa_key.json=./backend_sa_key.json
+```
+
+Create a config map for the Spanner instance
+
+```
+sed -e "s/PROJECT_ID/$PROJECT_ID/" \
+    -e "s/INSTANCE_ID/$SPANNER_INSTANCE_ID/" \
+    -e "s/DATABASE_ID/$SPANNER_DATABASE_ID/" \
+    spanner_config.yaml.tmpl > ./kubernetes-manifests/spanner_config.yaml
+
+kubectl apply -f spanner_config.yaml
+```
+
+### Deploy the manifests
+Once you have the kubernetes secret and config map established to connec to Cloud Spanner, the only thing left is to deploy the manifests. A [`scripts/deploy.sh`](../scripts/deploy.sh) file has been created to assist with this process.
+
+```
+export PROJECT_ID=<google cloud project id>
+./scripts/deploy.sh
+```
+
+## What's next
+
+Once the services are deployed you can use the generators to [run workloads](../generators/README.md).

generators/README.md

@@ -4,15 +4,34 @@ These generators leverage the [Locust](https://locust.io) python framework for g
 
 The generators can be run via the command line, or a web interface.
 
-If using the web interface, when you run the `locust` command for each service, you can point your web browser to the exposed port to determine
-concurrency of the load, in terms of "users". Then the load runs until the test is stopped in the browser.
+If using the web interface, when you run the `locust` command for each service, you can point your web browser to the exposed port to determine concurrency of the load, in terms of "users". Then the load runs until the test is stopped in the browser.
 
 Various charts are provided by the web interface to indicate the performance of the load test.
 
 If you do not want to use the web interface, the command line options specify the user concurrency, as well as a run time. Statistics are printed on the
 command line for the test.
 
+### Generator dependencies
 
+There are several dependencies required to get the generators to work:
+
+- Python 3.7+
+- Locust
+
+Assuming python3.X is installed, install dependencies via [pip](https://pypi.org/project/pip/):
+
+```
+# if pip3 is symlinked to pip
+pip install -r requirements.txt
+
+# if pip3 is not symlinked to pip
+pip3 install -r requirements.txt
+```
+
+> **NOTE:** To avoid modifying existing pip libraries on your machine, consider a solution like [virtualenv](https://pypi.org/project/virtualenv/).
+
+
+## Using the generators
 Provided generators do the following:
 
 - _authentication\_server.py_: mimics player signup and player retrieval by UUID. Login is not handled currently due to the necessity to track password creation.

generators/match_server.py

@@ -24,7 +24,7 @@ class GameMatch(HttpUser):
     leveraging the matchmaking-service
     """
 
-    @task(2)
+    @task(1)
     def create_game(self):
         """Task to create a new game"""