Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc(ingestion): add examples of running in docker and Kubernetes #4519

Merged
merged 1 commit into from
Mar 30, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 25 additions & 1 deletion metadata-ingestion/sink_docs/datahub.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ is that any errors can immediately be reported.

Check out the following recipe to get started with ingestion! See [below](#config-details) for full configuration options.

For general pointers on writing and running a recipe, see our [main recipe guide](../README.md#recipes).
For general pointers on writing and running a recipe, see our [main recipe guide](../README.md#recipes). This should point to the GMS server.

```yml
source:
Expand All @@ -28,6 +28,30 @@ sink:
server: "http://localhost:8080"
```

If you are running the ingestion in a container in docker and your [GMS is also running in docker](../../docker/README.md) then you should use the internal docker hostname of the GMS pod. Usually it would look something like

```yml
source:
# source configs
sink:
type: "datahub-rest"
config:
server: "http://datahub-gms:8080"
```

If GMS is running in a kubernetes pod [deployed through the helm charts](../../docs/deploy/kubernetes.md) and you are trying to connect to it from within the kubernetes cluster then you should use the Kubernetes service name of GMS. Usually it would look something like

```yml
source:
# source configs
sink:
type: "datahub-rest"
config:
server: "http://datahub-datahub-gms.datahub.svc.cluster.local:8080"
```

If you are using [UI based ingestion](../../docs/ui-ingestion.md) then where GMS is deployed decides what hostname you should use.

### Config details

Note that a `.` is used to denote nested fields in the YAML recipe.
Expand Down