chore(examples): Tracetest x Tyk Gateway example (#3668)

* chore(examples): Tracetest x Tyk Gateway example * chore(examples): Adding recipe for tyk * chore(examples): Adding recipe for tyk * chore(examples): PR comment fixes
kubeshop · Feb 20, 2024 · dfa3a52 · dfa3a52
1 parent 471fc43
commit dfa3a52
Show file tree

Hide file tree

Showing 18 changed files with 1,226 additions and 27 deletions.
diff --git a/docs/docs/examples-tutorials/img/tyk-results.png b/docs/docs/examples-tutorials/img/tyk-results.png
diff --git a/docs/docs/examples-tutorials/recipes.mdx b/docs/docs/examples-tutorials/recipes.mdx
@@ -40,6 +40,10 @@ These recipes show how to run tests against Serverless Functions with Tracetest.
 - [Testing AWS Lambda Functions (Serverless Framework) with OpenTelemetry and Tracetest](/examples-tutorials/recipes/testing-lambda-functions-with-opentelemetry-tracetest)
 - [Testing Cloudflare Workers with OpenTelemetry](/examples-tutorials/recipes/testing-cloudflare-workers-with-opentelemetry-tracetest)
 
+### Tyk
+
+- [Testing Distributed Services Proxied by Tyk Gateway (API Gateway) with OpenTelemetry and Tracetest](/examples-tutorials/recipes/testing-distributed-services-with-tyk-opentelemetry-tracetest)
+
 ## Trace Data Stores
 
 These recipes show integrations with trace data stores and tracing vendors/providers.

diff --git a/...rials/recipes/testing-distributed-services-with-tyk-opentelemetry-tracetest.mdx b/...rials/recipes/testing-distributed-services-with-tyk-opentelemetry-tracetest.mdx
@@ -0,0 +1,378 @@
+---
+id: testing-distributed-services-with-tyk-opentelemetry-tracetest
+title: Testing Distributed Services Proxied by Tyk Gateway (API Gateway) with OpenTelemetry and Tracetest
+description: Quickstart on how to configure Tyk Gateway with OpenTelemetry and Tracetest to execute trace-based tests against your Distributed Services.
+hide_table_of_contents: false
+keywords:
+  - tracetest
+  - trace-based testing
+  - observability
+  - distributed tracing
+  - testing
+  - end to end testing
+  - end-to-end testing
+  - integration testing
+  - tyk
+  - api gateway
+image: https://res.cloudinary.com/djwdcmwdz/image/upload/v1698686403/docs/Blog_Thumbnail_14_rsvkmo.jpg
+---
+
+:::note
+[Check out the source code on GitHub here.](https://github.com/kubeshop/tracetest/tree/main/examples/quick-start-tyk)
+:::
+
+[Tracetest](https://tracetest.io/) is a testing tool based on [OpenTelemetry](https://opentelemetry.io/) that allows you to test your distributed application. It allows you to use data from distributed traces generated by OpenTelemetry to validate and assert if your application has the desired behavior defined by your test definitions.
+
+[Tyk Gateway](https://tyk.io/) is an open-source API gateway and management platform designed to help developers control and manage their APIs. It serves as an intermediary layer between client applications and backend services, providing functionalities like authentication, rate limiting, access control, analytics, and traffic management.
+
+## Why is this important?
+
+Testing Distributed Services behind API Gateways has been a pain point for years. Not having visibility into the infrastructure and not knowing where a test fails causes the MTTR to be higher than for other tools. Including OpenTelemetry in your stack, allows you to expose telemetry from the tools you use like Tyk Gateway and your services that you can use for both production visibility and trace-based testing.
+
+This sample shows how to run integration tests against a Node.js API behind Tyk Gateway, using [OpenTelemetry](https://opentelemetry.io/) and Tracetest.
+
+The Node.js Services will fetch data from an external API, transform the data, and insert it into a Postgres table. This particular flow has two failure points that are difficult to test.
+
+1. Validating that an external API request from the worker function is successful.
+2. Validating that the Postgress insert operation is successful.
+
+## Prerequisites
+
+**Tyk Gateway Example:**
+
+Clone the [Tracetest GitHub Repo](https://github.com/kubeshop/tracetest) to your local machine, and open the quick start tyk quick start example app.
+
+```bash title=Terminal
+git clone https://github.com/kubeshop/tracetest.git
+cd tracetest/examples/quick-start-tyk
+```
+
+**Tracetest Account**:
+
+- Copy the `.env.template` file to `.env`.
+- Log into **the** [Tracetest app](https://app.tracetest.io/).
+- This example is configured to use the OpenTelemetry Collector. Ensure the environment you will be utilizing to run this example is also configured to use the OpenTelemetry Tracing Backend by clicking on Settings, Tracing Backend, OpenTelemetry, and Save.
+- Configure your environment to use [the agent](https://docs.tracetest.io/concepts/agent), then click the Settings link, and from the Agent tab select the "Run Agent Locally" option.
+- Fill out the [token](https://docs.tracetest.io/concepts/environment-tokens) and [agent api key](https://docs.tracetest.io/concepts/agent) details by editing your .env file. You can find these values in the Settings area for your environment.
+
+## Project Structure
+
+This is a Docker Compose project you can find the setup in the `docker-compose.yml` file
+
+### 1. The Tyk Gateway
+
+In the `docker-compose.yml` file you can find the Tyk Gateway setup.
+
+```yaml title=docker-compose.yml
+# Tyk Gateway
+tyk-gateway:
+  image: tykio/tyk-gateway:v5.2.1
+  ports:
+    - 8080:8080
+  environment:
+    - TYK_GW_OPENTELEMETRY_ENABLED=true
+    - TYK_GW_OPENTELEMETRY_EXPORTER=grpc
+    - TYK_GW_OPENTELEMETRY_ENDPOINT=otel-collector:4317
+  volumes:
+    - ./deployments/tyk-gateway/apps:/opt/tyk-gateway/apps
+    - ./deployments/tyk-gateway/tyk.conf:/opt/tyk-gateway/tyk.conf
+  depends_on:
+    - tyk-redis
+
+tyk-redis:
+  image: redis:6.0.4
+  volumes:
+    - tyk-redis-data:/data
+```
+
+Adding the configuration for OpenTelemetry to the Tyk Gateway is as simple as setting the environment variables `TYK_GW_OPENTELEMETRY_ENABLED`, `TYK_GW_OPENTELEMETRY_EXPORTER`, and `TYK_GW_OPENTELEMETRY_ENDPOINT`.
+
+You can find the Tyk Gateway apps and configuration in the `deployments/tyk-gateway` file.
+
+```yaml title=deployments/tyk-gateway/apps/pokeshop-demo.json
+{
+  "name": "pokeshop",
+  "api_id": "1",
+  "org_id": "default",
+  "active": true,
+  "use_keyless": false,
+  "detailed_tracing": true,
+  "definition": { "location": "header", "key": "version" },
+  "auth": { "auth_header_name": "authorization" },
+  "version_data": { "not_versioned": true, "versions": { "Default": { "name": "Default" } } },
+  "proxy":
+    {
+      "listen_path": "/pokeshop/",
+      "target_url": "http://api:8081/pokemon/",
+      "strip_listen_path": true,
+      "preserve_host_header": true,
+    },
+}
+```
+
+### 2. Tracetest
+
+The tracetest setup is composed by the `tracetest-client` and the `tracetest-agent` services under the `docker-compose.yml` file.
+
+```yaml title=docker-compose.yml
+# Tracetest
+tracetest-client:
+  build: ./tracetesting
+  environment:
+    TRACETEST_API_TOKEN: ${TRACETEST_API_TOKEN}
+    POKESHOP_DEMO_URL: ${POKESHOP_DEMO_URL}
+  depends_on:
+    api:
+      condition: service_healthy
+
+tracetest-agent:
+  environment:
+    TRACETEST_DEV: ${TRACETEST_DEV}
+    TRACETEST_API_KEY: ${TRACETEST_AGENT_API_KEY}
+    TRACETEST_SERVER_URL: ${TRACETEST_SERVER_URL}
+  image: kubeshop/tracetest-agent:latest
+```
+
+### 3. Services under Test
+
+As a testing ground, the example uses Tracetest's own Pokeshop Demo APP which includes the `api` and `worker` services.
+
+```yaml title=docker-compose.yml
+# Demo services
+
+# pokeshop demo services
+postgres:
+  image: postgres:14
+  environment:
+    POSTGRES_USER: postgres
+    POSTGRES_PASSWORD: postgres
+    POSTGRES_DB: postgres
+  healthcheck:
+    test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"]
+    interval: 1s
+    timeout: 5s
+    retries: 60
+
+cache:
+  image: redis:6
+  healthcheck:
+    test: ["CMD", "redis-cli", "ping"]
+    interval: 1s
+    timeout: 3s
+    retries: 60
+
+queue:
+  image: rabbitmq:3.12
+  restart: unless-stopped
+  healthcheck:
+    test: rabbitmq-diagnostics -q check_running
+    interval: 1s
+    timeout: 5s
+    retries: 60
+
+api:
+  image: kubeshop/demo-pokemon-api:latest
+  restart: unless-stopped
+  pull_policy: always
+  environment:
+    REDIS_URL: cache
+    DATABASE_URL: postgresql://postgres:postgres@postgres:5432/postgres?schema=public
+    RABBITMQ_HOST: queue
+    POKE_API_BASE_URL: https://pokeapi.co/api/v2
+    COLLECTOR_ENDPOINT: http://otel-collector:4317
+    NPM_RUN_COMMAND: api
+  healthcheck:
+    test: ["CMD", "wget", "--spider", "localhost:8081"]
+    interval: 1s
+    timeout: 3s
+    retries: 60
+  depends_on:
+    postgres:
+      condition: service_healthy
+    cache:
+      condition: service_healthy
+    queue:
+      condition: service_healthy
+
+worker:
+  image: kubeshop/demo-pokemon-api:latest
+  restart: unless-stopped
+  pull_policy: always
+  environment:
+    REDIS_URL: cache
+    DATABASE_URL: postgresql://postgres:postgres@postgres:5432/postgres?schema=public
+    RABBITMQ_HOST: queue
+    POKE_API_BASE_URL: https://pokeapi.co/api/v2
+    COLLECTOR_ENDPOINT: http://otel-collector:4317
+    NPM_RUN_COMMAND: worker
+  depends_on:
+    postgres:
+      condition: service_healthy
+    cache:
+      condition: service_healthy
+    queue:
+      condition: service_healthy
+```
+
+## Set up Environment Variables
+
+Copy the `.env.template` and create a `.env` file in the same directory. Add token and Cloud Agent endpoint.
+
+```txt title=.env
+TRACETEST_API_TOKEN=<my_token_with_engineer_access>
+TRACETEST_AGENT_API_KEY=<my_agent_api_key>
+POKESHOP_DEMO_URL=http://tyk-gateway:8080
+```
+
+## The Tracetest Script
+
+The `tracetest/index.ts` file contains the script that will execute the trace-based tests against the tyk endpoint.
+
+### Steps Executed by the Script
+1. Create a new key in the Tyk Gateway.
+2. Create a new test definition in Tracetest (using the async import service from the Pokeshop Demo).
+3. Run the test with the variables defined.
+
+```typescript title=tracetesting/tracetest.ts
+import Tracetest from "@tracetest/client";
+import fetch from "node-fetch";
+import { TestResource } from "@tracetest/client/dist/modules/openapi-client";
+import { config } from "dotenv";
+
+config();
+
+const { TRACETEST_API_TOKEN = "", POKESHOP_DEMO_URL = "" } = process.env;
+
+const params = {
+  headers: {
+    "Content-Type": "application/json",
+    "x-tyk-authorization": "28d220fd77974a4facfb07dc1e49c2aa",
+    "Response-Type": "application/json",
+  },
+};
+
+const setup = async () => {
+  const alias = "website";
+
+  const data = {
+    alias,
+    expires: -1,
+    access_rights: {
+      1: {
+        api_id: "1",
+        api_name: "pokeshop",
+        versions: ["Default"],
+      },
+    },
+  };
+
+  const res = await fetch("http://tyk-gateway:8080/tyk/keys/create", {
+    ...params,
+    method: "POST",
+    body: JSON.stringify(data),
+  });
+
+  const { key } = (await res.json()) as { key: string };
+
+  return key;
+};
+
+const definition: TestResource = {
+  type: "Test",
+  spec: {
+    id: "ZV1G3v2IR",
+    name: "Import Pokemon",
+    trigger: {
+      type: "http",
+      httpRequest: {
+        method: "POST",
+        url: "${var:ENDPOINT}/pokeshop/import",
+        body: '{"id": ${var:POKEMON_ID}}',
+        headers: [
+          {
+            key: "Content-Type",
+            value: "application/json",
+          },
+          {
+            key: "Authorization",
+            value: "${var:AUTHORIZATION}",
+          },
+        ],
+      },
+    },
+    specs: [
+      {
+        selector: 'span[tracetest.span.type="database"]',
+        name: "All Database Spans: Processing time is less than 100ms",
+        assertions: ["attr:tracetest.span.duration < 100ms"],
+      },
+      {
+        selector: 'span[tracetest.span.type="http"]',
+        name: "All HTTP Spans: Status  code is 200",
+        assertions: ["attr:http.status_code = 200"],
+      },
+      {
+        selector: 'span[tracetest.span.type="http" name="GET" http.method="GET" net.peer.name = "pokeapi.co"]',
+        name: "The request matches the pokemon Id",
+        assertions: ['attr:http.url   =  "https://pokeapi.co/api/v2/pokemon/${var:POKEMON_ID}"'],
+      },
+    ],
+  },
+};
+
+const main = async () => {
+  const tracetest = await Tracetest(TRACETEST_API_TOKEN);
+
+  const key = await setup();
+
+  const test = await tracetest.newTest(definition);
+  await tracetest.runTest(test, {
+    variables: [
+      {
+        key: "ENDPOINT",
+        value: POKESHOP_DEMO_URL.trim(),
+      },
+      {
+        key: "POKEMON_ID",
+        value: `${Math.floor(Math.random() * 100) + 1}`,
+      },
+      {
+        key: "AUTHORIZATION",
+        value: key,
+      },
+    ],
+  });
+  console.log(await tracetest.getSummary());
+};
+
+main();
+```
+
+The output from the `tracetest/index.ts` script will show the test results with links to the Tracetest App.
+
+```bash title=Terminal
+2024-02-20 13:11:06 
+2024-02-20 13:11:06 Successful: 1
+2024-02-20 13:11:06 Failed: 0
+2024-02-20 13:11:06 
+2024-02-20 13:11:06 [✔️ Import Pokemon] #11 - https://app.tracetest.io/organizations/ttorg_ced62e34638d965e/environments/ttenv_b42fa137465c6e04/test/ZV1G3v2IR/run/11
+2024-02-20 13:11:06 
+```
+
+### Tracetest App Results
+
+![Tracetest App Results](../img/tyk-results.png)
+
+### Running the Example
+
+Spin up the deployment and test execution.
+
+```bash title=Terminal
+docker-compose up
+```
+
+This will trigger the Docker Compose setup and immediately run the [trace-based tests using the Tracetest Typescript integration](../../tools-and-integrations/typescript.mdx) as part of the `tracetest-client` service.
+
+## Learn More
+
+Feel free to check out our [examples in GitHub](https://github.com/kubeshop/tracetest/tree/main/examples) and join our [Slack Community](https://dub.sh/tracetest-community) for more info!